<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Niyi Ayinde on Medium]]></title>
        <description><![CDATA[Stories by Niyi Ayinde on Medium]]></description>
        <link>https://medium.com/@niyi.ayinde1?source=rss-c38de887a73e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*2S7kLs2UJA4LosCycYo6Vg.jpeg</url>
            <title>Stories by Niyi Ayinde on Medium</title>
            <link>https://medium.com/@niyi.ayinde1?source=rss-c38de887a73e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 14:30:56 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@niyi.ayinde1/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Built a Serverless AWS Cost-Optimization Pipeline That Detects, Acts, and Explains Itself.]]></title>
            <link>https://medium.com/@niyi.ayinde1/i-built-a-serverless-aws-cost-optimization-pipeline-that-detects-acts-and-explains-itself-c33f9cc74942?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/c33f9cc74942</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Mon, 10 Aug 2026 02:30:07 GMT</pubDate>
            <atom:updated>2026-08-10T04:28:12.818Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A serverless AWS pipeline that identifies idle cloud resources using defined thresholds, safely takes action, converts results into executive-level cost insights, and built as infrastructure-as-code, with production deployed and staging prepared for a CI/CD rollout.</em></p><p><a href="https://github.com/NiyiNeo/cloud-cost-optimization-pipeline">https://github.com/NiyiNeo/cloud-cost-optimization-pipeline</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*WZl5sWrpYSh-cBuXNsVMIw.png" /></figure><h4><strong>The Problem</strong></h4><p><strong><em>LevelUp Retail</em></strong><em>, a fast‑growing e‑commerce startup, has been overspending on unused cloud resources with a concrete version of that problem: EC2 instances left running at near-zero utilization, S3 buckets sitting empty for months. The leadership team approached me on finding a solution in a way they can understand the appropriate way to utilize their resources.</em></p><p><em>I noted that cost optimization is one of the pillars of the AWS Well-Architected Framework, and in most real environments it fails the same way: idle resources quietly accumulate.</em></p><p><em>The goal wasn’t to wire together a tutorial. It was to build a system I could deploy, defend, and hand to the leadership of LevelUp Retail and understand every decision well enough to explain the why behind it.</em></p><p><strong>Technology stack:</strong></p><p><strong>Terraform: </strong><em>IaC tool used to build and define the AWS services to construct the cost optimization tool.</em></p><p><strong>Amazon EC2:</strong> <em>the scalable resource the scanner lambda will identify, and the action lambda will take action on within defined threshold to optimize cost savings.</em></p><p><strong>Amazon S3:</strong> <em>stores the scan reports each run produces and the AI-Generated summaries: also, one of the resources the scanner audits for idle resources.</em></p><p><strong>Amazon DynamoDB:</strong> <em>The database to save the results from the Scanner Lambda and allow the Action Lambda to act on once stored.</em></p><p><strong>AWS Lambda</strong>: <em>Serverless tool that will be used to perform the action needed to optimization the resources in the LeveUp Retail cloud environment.</em></p><p><strong>AWS Step Functions: </strong><em>orchestrate the three Lambdas in sequence, passing data between them with retry and failure handling, so the pipeline runs automatically.</em></p><p><strong>Amazon SNS:</strong> Sends the notification when a pipeline run completes, delivering the AI summary and run status so the outcome reaches a human instead of sitting in a log.</p><p><strong>Docker</strong>: Packages each Lambda’s code and dependencies into a container image, so the function that runs in AWS is the exact one I built and tested.</p><p><strong>Amazon ECR:</strong> Stores the three Lamba container images that AWS pulls from at deploy time; the private registry the pipeline builds to.</p><p><strong>GitHub Actions:</strong> The CI/CD layer that builds the container images and applies the Terraform automatically, so deployments are repeatable.</p><p><strong>Python</strong> — <em>Used to build the functionality of the three lambdas.</em></p><p><strong>AWS Identity and Access Management (IAM) User</strong>: To<em> manage the permissions needed to allow roles to be assumed to execute tasks within the AWS environment.</em></p><h4><strong><em>The Architecture and the Decisions Behind It</em></strong></h4><p>The solution is structured as a four-phase pipeline: resource detection, safe action, AI-assisted summarization, and scheduled orchestration. Terraform with production deployed and staging structured for rollout, with a clear separation between shared foundational infrastructure and environment-specific application stacks. The foundation layer contains the DynamoDB runs table, ECR repositories, and S3 reports bucket, while the env layers contain the Lambda functions, policies, triggers, and other runtime-specific resources.</p><p>I chose this split because foundational resources and application resources do not share the same lifecycle. Core shared components such as the image registry, reporting bucket, and historical run data need to outlive any single application deployment or environment reset. Keeping them in a separate foundation layer prevents an environment teardown from wiping out durable state or shared assets, while also making deployments cleaner, safer, and easier to reason about.</p><h4><strong>Detect: defining the “idle” instances</strong></h4><p>The Scanner Lambda doesn’t flag instances that “look” idle. It applies the Well-Architected definition: EC2 instances averaging under 5% CPU for 7 consecutive days (pulled from CloudWatch), and S3 buckets empty for more than 30 days. The threshold <em>is</em> the product “idle” without numbers is an opinion.</p><pre>    # ---- EC2 scan ----<br>    instance_ids = list_running_instances()<br>    idle_instances: List[Dict[str, Any]] = []<br><br>    for iid in instance_ids:<br>        avgs = cpu_daily_average(iid, CPU_WINDOW_DAYS)<br>        if is_idle_cpu(avgs, CPU_IDLE_PCT, CPU_WINDOW_DAYS):<br>            recent = avgs[-CPU_WINDOW_DAYS:] if avgs else []<br>            cpu_avg_7d = round(sum(recent) / max(1, len(recent)), 2)<br>            monthly = round(24 * 30 * FALLBACK_EC2_USD_H, 2)<br><br>            idle_instances.append({<br>                &quot;resource_id&quot;: iid,<br>                &quot;resource_type&quot;: &quot;ec2&quot;,<br>                &quot;ENV&quot;: ENV,<br>                &quot;last_seen&quot;: ts_iso,<br>                &quot;idle_reason&quot;: f&quot;cpu&lt;{CPU_IDLE_PCT}% for {CPU_WINDOW_DAYS}d&quot;,<br>                &quot;metrics_snapshot&quot;: {&quot;cpu_avg_7d&quot;: Decimal(str(cpu_avg_7d))},<br>                &quot;estimated_monthly_cost_savings&quot;: Decimal(str(monthly)),<br>                &quot;risk_flags&quot;: [],<br>            })</pre><p>Each finding is written to DynamoDB with the fields that make it actionable downstream: the resource ID, why it was flagged, a metrics snapshot, and an estimated monthly saving.</p><h4><strong>Act: the decision of protecting every change</strong></h4><p>The Action Lambda is the only component that touches live infrastructure, so it’s the one place a mistake is expensive. A cost tool that stops the <em>wrong</em> instance is worse than no tool at all. So I created a guardrail for every mutation:</p><ol><li>A <strong>DRY_RUN flag</strong> enforced at the mutation point: in dry-run, the function tags and stops <em>nothing</em> and logs dry-run instead.</li><li>An <strong>IAM condition</strong> that only permits ec2:StopInstances on resources tagged cost:managed = true, the permission itself can’t touch an unmanaged instance.</li><li>A <strong>protect-tag</strong> (Cleanup:Protected) the code honors, so a flagged-but-protected instance is never stopped.</li></ol><pre>def stop_instance(instance_id: str) -&gt; Tuple[bool, str]:<br>    try:<br>        if DRY_RUN:<br>            return True, &quot;dry-run&quot;<br>        ec2.stop_instances(InstanceIds=[instance_id])<br>        return True, &quot;stopped&quot;<br>    except ClientError as e:<br>        return False, str(e)<br><br>  statement {<br>    sid       = &quot;StopInstancesIfManaged&quot;<br>    actions   = [&quot;ec2:StopInstances&quot;]<br>    resources = [&quot;*&quot;]<br>    condition {<br>      test     = &quot;StringEquals&quot;<br>      variable = &quot;ec2:ResourceTag/cost:managed&quot;<br>      values   = [&quot;true&quot;]<br>    }</pre><h4><strong>Translate: the layer of communication that makes the work understandable.</strong></h4><p>Detection and action produce machine output: stopped: 3, estimated_monthly_savings: $X. The AI-Insights Lambda reads that structured result and uses Amazon Bedrock to generate a <strong>natural-language executive summary</strong> — the kind a VP reads without knowing what a Lambda is. This is the translation layer, and it’s the difference between a tool engineers use and a tool a business acts on.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/973/1*Ab33e_QSlDyNXH4yQcvRTQ.png" /></figure><p>The pipeline ran and correctly found nothing to act on. The environment is new enough that no resource has crossed the idle threshold yet. The summary reads “Status: Green: no action required”. The empty result is the system working appropriately and not failing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*C2hGSJT5vyMH0xuQIvSrDA.png" /></figure><p>So, to show what the pipeline would produce when waste existed vs a tool reporting no findings, I created a sample populated run. The same summary format, with real findings, savings and recommended actions rendered in language that comprehended and acted upon.</p><pre>def stop_instance(instance_id: str) -&gt; Tuple[bool, str]:<br>    try:<br>        if DRY_RUN:<br>            return True, &quot;dry-run&quot;<br>        ec2.stop_instances(InstanceIds=[instance_id])<br>        return True, &quot;stopped&quot;<br>    except ClientError as e:<br>        return False, str(e)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/975/1*meUjxd9Q7-tCnTNmZ4C1yA.png" /></figure><h4><strong>Orchestrate: why Step Functions, not direct invocation</strong></h4><p>The three Lambdas don’t call each other, that was intentional. A Step Functions state machine runs them in sequence, threading each run’s ID and report location from one stage to the next, and an EventBridge schedule fires it at 7:00 every morning. I chose that over having each Lambda directly invoke the next . Direct invocation couples the functions together and hides failures. For a pipeline that runs unattended, orchestration is worth more than the simplicity of a direct call, for three specific reasons:</p><p><strong>Visibility:</strong></p><p>Step Functions gives me a historical map of the whole run. If one stage fails, I can see which one stalled and why, right in the execution graph. With direct Lambda-to-Lambda invocation, that same failure is scattered across separate CloudWatch log groups, and I’m searching each one by hand to reconstruct what happened.</p><p><strong>Built-in retry and error handling:</strong></p><p>Step Functions handles this declaratively: I define retry behavior on transient failures and route anything that still fails to a dedicated Fail state, without writing that logic myself. With direct invocation, I’d have to hand-code retry and failure handling into every Lambda.</p><p><strong>Decoupling:</strong></p><p>The Lambdas aren’t aware of each other. The state machine passes the data between them, so in this case the run_id and report_prefix that link one run’s output to the next stage’s input. Because the functions are decoupled, one engineer can work on and test a single stage without touching the rest of the pipeline.</p><pre># Scanner captures run_id + report_prefix from its output...<br>ResultSelector = {<br>  &quot;run_id.$&quot;        = &quot;$.Payload.run_id&quot;<br>  &quot;report_prefix.$&quot; = &quot;$.Payload.report_prefix&quot;<br>}<br>ResultPath = &quot;$.scanner&quot;<br><br># ...and Action reads them from the previous stage as its input:<br>Payload = {<br>  &quot;run_id.$&quot;        = &quot;$.scanner.run_id&quot;<br>  &quot;report_prefix.$&quot; = &quot;$.scanner.report_prefix&quot;<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/974/1*8IWWNwGx7M_shPjAkK1ClQ.png" /></figure><h4><strong>The challenges and the solutions:</strong></h4><p><strong>1.</strong> <strong>The IAM permissions: reading the error, not speculating</strong></p><p><strong>The problem:</strong> Deploying the state machine failed repeatedly with AccessDenied on actions I thought my tf.admin user already had. The trap was in the near-identical names: states:ListStateMachines (which I had) and states:ListStateMachineVersions (which I didn’t) look almost the same, one word apart.</p><pre>User: arn:aws:iam::123456789012:user/tf.admin is not authorized to perform:<br>states:ListStateMachineVersions on resource: …:stateMachine:levelupretail-costopt-prod<br>because no identity-based policy allows the states:ListStateMachineVersions action</pre><p><strong>The second layer:</strong> After I fixed the tf.admin permissions and deployed, the Action Lambda failed with a cloudwatch:PutMetricData denial. My first instinct was to edit tf.admin again, but the error named a <em>different</em> identity: assumed-role/levelupretail-action-prod. That was the Lambda’s execution role, not my deploy user. Two separate identity tiers: tf.admin provisions the infrastructure, while execution roles govern what the running code can do. The error tells me <em>which</em> identity failed, which told me <em>which</em> policy to fix. Chasing the wrong tier would have meant changing a policy that was never the problem.</p><p><strong>How I worked it:</strong> I used AI throughout this build, and I stayed in control of every decision it touched. I reviewed each output, understood every permission I deployed, and could debug from first principles. In 2026, an AI assistant is like a backup camera in a car, it gives me visibility, but I still must interpret what I see and know how to drive. My practice was to read each error in VS Code and form my own explanation first: “I think this is denying X because Y”, then use the assistant to confirm or challenge that reasoning. On this one, my first hypothesis was wrong: I saw ListStateMachineVersions and assumed it was a permission I already had. Being corrected is what taught me to read the action name all the way to the last word.</p><p><strong>The lesson:</strong> The fix wasn’t “add more permissions.” It was to read the error down to the exact action name <em>and</em> the exact identity, grant only what was missing, and discover the provider’s real required permissions empirically instead of falling back on an overbroad wildcard.</p><p><strong>2. The DRY_RUN accounting bug and correctness of output</strong></p><p><strong>The setup:</strong> I wanted to verify my own safety gate before letting the pipeline run on a schedule: to confirm DRY_RUN prevented modifications. It did.</p><p><strong>What the review turned up:</strong> The review turned up something subtler: stop_instance() returns True in dry-run (meaning “the call didn’t error”), and the handler treated that as “the instance was stopped.” So, it increased the counter <em>and</em> accrued realized savings for a stop that never happened.</p><p><strong>Why it mattered:</strong> Nothing unsafe occurred. But the run summary, the DynamoDB record, and the AI executive summary would all report dollars that were never actually saved. An assessor could flag immediately: “The demo states $400 was saved, but DRY_RUN was on” and that strips the <strong>credibility </strong>from every other number in the report.</p><p><strong>The fix + code block:</strong> I wanted to separate “the call succeeded” from “the change actually happened.”</p><pre># Before any successful call accrued savings<br>if ok_stop:<br>    totals[&quot;ec2&quot;][&quot;stopped&quot;] += 1<br>    realized_savings += _money(ec2_plan.get(&quot;estimated_monthly_cost_savings&quot;, 0))<br># After only a real stop accrues realized savings<br>did_stop = ok_stop and stop_msg != &quot;dry-run&quot;<br>if ok_stop:<br>    totals[&quot;ec2&quot;][&quot;stopped&quot;] += 1<br>if did_stop:<br>    realized_savings += _money(ec2_plan.get(&quot;estimated_monthly_cost_savings&quot;, 0))</pre><p><strong>The lesson:</strong> A function returning successfully isn’t the same merit as the system doing what it claims. For a cost tool, the headline number <em>is</em> the purpose of the product: so a reliable zero is worth more than an impressive fiction.</p><p><strong>3. Designing a verification strategy that isolated failure domains</strong></p><p>T<strong>he problem:</strong> Three independent functions: Scanner, Action, AI — Insights, that also have to work together as a chain. I needed to verify that actually did, not just assume it.</p><p><strong>What I did:</strong> I ran Scanner alone and confirmed it produced its output; the run_id and report location. I fed that exact output into Action and confirmed it ran green. I fed that into AI-Insights and confirmed the same. After each function passed on its own, I wired them into Step Functions, where the full run validated what the standalone tests could not: the handoff between stages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3tfZoI6sTo41Jf4lHGiLfQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1LzEeIIIXZmb4DY3T7EuXA.png" /></figure><p><strong>The lesson: </strong>To isolate failure domains before integrating. If I’d wired the chain first and it broke, I wouldn’t know whether the bug was in a function’s logic or in the orchestration handoff. This prevented me from debugging two unknowns at once. I verified the pieces work, then prove the seams.</p><h4>Conclusion:</h4><p>What I set out to build was a cost-optimization pipeline. This system detects idle resources against measured thresholds, takes actions with integrated safety guardrails, and translates the outcome to a non-technical audience. I built a CI/CD pipeline using GitHub Actions and OIDC, so deployments run automatically on merge to main without long-lived credentials.</p><p>The pipeline works, but what I would emphasize in a conversation is not the architecture itself, it is the judgment behind it. I learned to read IAM errors down to the exact action and identity, recognize that a successful function call does not always mean the system is reporting the truth, and validate each component before trusting the full workflow.</p><p>I built this to show that I can solve a real cloud problem and defend every decision behind the solution.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c33f9cc74942" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Auto-Scaling Group Deployment on AWS with Terraform.]]></title>
            <link>https://medium.com/@niyi.ayinde1/auto-scaling-group-deployment-on-aws-with-terraform-70d655be8694?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/70d655be8694</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Thu, 21 Aug 2025 03:44:38 GMT</pubDate>
            <atom:updated>2025-08-21T03:44:38.647Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/727/1*jQeF8zOAs7osH6HkCiLG0Q.jpeg" /></figure><p>GitHub Link: <a href="https://github.com/NiyiNeo/tf-asg-01">https://github.com/NiyiNeo/tf-asg-01</a></p><h3><strong>Introduction:</strong></h3><p>Hello everyone. I am back again to assist with an e-commerce company, Holiday Haven. During the holiday season, they receive the most amount of traffic as it is their busiest time of the year and keep up with the customers’ request and maintain ideal response times.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Holiday Haven’s goal is to ensure that the website remains available during these times of high traffic to handle the capacity as well as minimize the costs when the traffic is low. Also, Holiday Haven wants to ensure that the infrastructure for their website is reliable and easy managed, this way the can easily roll back when necessary.</p><h3><strong>Solution Strategy:</strong></h3><p>My solution is to build an Auto Scaling group using Terraform that spans two subnets within a default VPC. By doing this, this solution will ensure the high availability that they need as well as fault tolerance, giving Holiday Haven the ability to scale up or down based on the traffic to the website. With the Auto Scaling group, I will create a security group that will allow traffic from the internet and that associates with the EC2 instances in the auto scaling group. The auto scaling group will have a minimum of two instances and a maximum of five instances. Along with this, I will launch an Apache web server for the website to be available to all their customers. Finally, I will create an S3 bucket and set it as the remote backend for version control.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Terraform: </strong>is an infrastructure as code tool that lets one define both cloud and on-premises resources in human-readable configuration files that one can version, reuse, and share.</p><p><strong>Auto-Scaling Group</strong>- contains a collection of EC2 instances that are treated as logical grouping for the purpose of automatic scaling and management.</p><p><strong>Amazon VPC</strong>: define and launch AWS resources in a logically isolated virtual network</p><p><strong>Amazon EC2:</strong> on-demand, scalable computing capacity in AWS Cloud</p><p><strong>Amazon S3:</strong> an object storage service that offers industry-leading scalability, data availability, security and performance.</p><p><strong>Apache web server</strong>: open-source and free web server, commonly utilized for web hosting and development.</p><p><strong>Visual Studio (VS) Code</strong>: a free and lightweight source code editor.</p><p>My project layout will be:</p><pre>tf-asg-01/<br>├─ main.tf<br>├─ data.tf<br>├─ variables.tf<br>├─ outputs.tf<br>├─ providers.tf<br>├─ versions.tf<br>├─ backend.tf             <br>├─ backend.hcl               # local only (ignored by git)<br>├─ user_data_apache.sh<br>├─ .gitignore<br>└─ .terraform.lock.hcl</pre><h3><strong>Implementation Steps:</strong></h3><p><strong><em>Building out the Terraform (.tf) files:</em></strong></p><p>I added the versions.tf file.</p><p>This file defines the required versions of Terraform and its the provider plugin for my projects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/387/1*fwHbEM_z2W0Sg0Et269vTw.png" /></figure><p>Next, I built out my different terraform documents for this project. The first one is the provider.tf. This file is used to define the providers I am going to use and which services Terraform will engage with.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/619/1*Ct_zGgDL_qRUQvQqKP40Ug.png" /></figure><p><strong><em>Data sources construction:</em></strong></p><p>My goal was to target a previous created VPC, jenkins-vpc, as this VPC has already been constructed in a previous project.</p><p>I also added a second subnet to this section as well.</p><pre># Look up the existing Jenkins VPC by Name tag<br>data &quot;aws_vpc&quot; &quot;jenkins&quot; {<br>  filter {<br>    name   = &quot;tag:Name&quot;<br>    values = [var.vpc_name]<br>  }<br>}<br><br># Look up existing public subnet A by Name tag and scope to that VPC<br>data &quot;aws_subnet&quot; &quot;public_a&quot; {<br>  filter {<br>    name   = &quot;tag:Name&quot;<br>    values = [var.public_subnet_a_name]<br>  }<br>  filter {<br>    name   = &quot;vpc-id&quot;<br>    values = [data.aws_vpc.jenkins.id]<br>  }<br>}<br><br># Get the route table currently associated to subnet A <br>data &quot;aws_route_table&quot; &quot;public_rt_for_a&quot; {<br>  subnet_id = data.aws_subnet.public_a.id<br>}<br><br># Create NEW public subnet B in the same VPC<br>resource &quot;aws_subnet&quot; &quot;public_b&quot; {<br>  vpc_id                  = data.aws_vpc.jenkins.id<br>  cidr_block              = var.public_b_cidr<br>  availability_zone       = var.public_b_az<br>  map_public_ip_on_launch = true<br><br>  tags = {<br>    Name = var.public_subnet_b_name<br>  }<br>}<br><br># Associate subnet B to the *same* public route table as subnet A<br>resource &quot;aws_route_table_association&quot; &quot;public_b_assoc&quot; {<br>  subnet_id      = aws_subnet.public_b.id<br>  route_table_id = data.aws_route_table.public_rt_for_a.id<br>}<br><br># Convenience local: pass these to your ASG&#39;s vpc_zone_identifier<br>locals {<br>  asg_subnet_ids = [<br>    data.aws_subnet.public_a.id,<br>    aws_subnet.public_b.id,<br>  ]<br>}</pre><p>Followed the same steps to build out the variables.tf. This is again to avoid placing hardcoded data within the code.</p><p>I added the HTTP/HTTPS traffic from the internet and SSH connection with my IP.</p><pre>variable &quot;aws_region&quot; {<br>  description = &quot;AWS region to deploy resources&quot;<br>  type        = string<br>  default     = &quot;us-east-1&quot;<br>}<br><br>#Ecom User for Auto Scaling Group<br>variable &quot;aws_profile&quot; {<br>  description = &quot;The aws profile for this project&quot;<br>  type        = string<br>  default     = &quot;tf.admin&quot;<br>}<br><br>variable &quot;key_name&quot; {<br>  description = &quot;Key value for the EC2 instance&quot;<br>  type        = string<br>  default     = &quot;tf-admin-key&quot;<br>}<br><br><br>#EC2 Instance Details<br>variable &quot;ami_id&quot; {<br>  description = &quot;AMI of the EC2 instance&quot; #Ubuntu<br>  type        = string<br>  default     = &quot;ami-0a7d80731ae1b2435&quot;<br>}<br><br>variable &quot;instance_type&quot; {<br>  description = &quot;Type of EC2 Instance&quot;<br>  type        = string<br>  default     = &quot;t3.micro&quot;<br>}<br><br><br>#VPC Setup<br>variable &quot;vpc_name&quot; {<br>  description = &quot;The default VPC for Auto Scaling Group&quot;<br>  type        = string<br>  default     = &quot;jenkins-vpc&quot;<br>}<br><br>variable &quot;public_subnet_a_name&quot; {<br>  description = &quot;The first public subnet, already exist &quot;<br>  type        = string<br>  default     = &quot;jenkins-public-jsn&quot;<br>}<br><br>variable &quot;public_subnet_b_name&quot; {<br>  description = &quot;The second public subnet, new subnet &quot;<br>  type        = string<br>  default     = &quot;jenkins-public-jsn-b&quot;<br>}<br><br>variable &quot;public_b_cidr&quot; {<br>  description = &quot;Addning a second subnet to VPC, (non-overlapping)&quot;<br>  type        = string<br>  default     = &quot;10.0.12.0/24&quot;<br>}<br><br>variable &quot;public_b_az&quot; {<br>  description = &quot;AZ for subnet b &quot;<br>  type        = string<br>  default     = &quot;us-east-1b&quot;<br>}<br><br>#Security Group CIDRs<br>variable &quot;ssh_cidr&quot; {<br>  description = &quot;Secure access to EC2&quot;<br>  type        = string<br>  default     = &quot;YOUR-IP ADDRESS/32&quot;<br>}<br><br>variable &quot;http_cidrs&quot; {<br>  description = &quot;CIDR blocks allowed to HTTP (80)&quot;<br>  type        = list(string)<br>  default     = [&quot;0.0.0.0/0&quot;]<br>}<br><br>variable &quot;https_cidrs&quot; {<br>  description = &quot;CIDR blocks allowed to HTTPS (443)&quot;<br>  type        = list(string)<br>  default     = [&quot;0.0.0.0/0&quot;]<br>}<br><br>#Auto Scaling Sizes<br>variable &quot;asg_min&quot; {<br>  description = &quot;Minimum capacity of EC2 instances for ASG&quot;<br>  type        = number<br>  default     = 2<br>}<br><br>variable &quot;asg_desired&quot; {<br>  description = &quot;Desired capacity of EC2 instances for ASG&quot;<br>  type        = number<br>  default     = 2<br>}<br><br>variable &quot;asg_max&quot; {<br>  description = &quot;Maximum capacity of EC2 instances for ASG&quot;<br>  type        = number<br>  default     = 5<br>}<br><br>#Tags<br>variable &quot;app_name&quot; {<br>  description = &quot;Apache Web Server&quot;<br>  type        = string<br>  default     = &quot;apache-asg&quot;<br>}<br><br></pre><p><strong><em>Launch Apache Webserver with user data script:</em></strong></p><p>This script is a one-time bootstrap script that cloud-init runs on the first boot of each EC2 instance. For the Auto-Scaling Group, every new or replacement configures on by itself within having to log into the instance. The script will refresh the packages, install the Apache, start or enable the service and renders a landing page to show my hostname, EC2 instance ID, and any other added information.</p><pre># 1) Use bash to interpret this file<br>#!/bin/bash<br><br># 2) Fail fast, show commands, catch pipe errors                         <br>set -euxo pipefail <br><br># 3) Suppress interactive apt prompts<br>export DEBIAN_FRONTEND=noninteractive<br><br># 4) Refresh package indices<br>apt-get update -y<br><br># 5) Install Apache + curl (curl used below)              <br>apt-get install -y apache2 curl<br><br># 6) Start Apache now and on boot<br>systemctl enable --now apache2       <br><br># 7) Dynamic data for the landing page<br>HOSTNAME_FQDN=&quot;$(hostname -f || hostname)&quot;<br>INSTANCE_ID=&quot;$(curl -s http://169.254.169.254/latest/meta-data/instance-id || echo unknown)&quot;<br><br># 8) Generate a very simple landing page<br>cat &gt;/var/www/html/index.html &lt;&lt;EOF<br>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>  &lt;head&gt;&lt;meta charset=&quot;utf-8&quot;/&gt;&lt;title&gt;Apache on Ubuntu (ASG)&lt;/title&gt;&lt;/head&gt;<br>  &lt;body&gt;<br>    &lt;h1&gt;Hello from ${HOSTNAME_FQDN}&lt;/h1&gt;<br>    &lt;p&gt;Instance ID: ${INSTANCE_ID}&lt;/p&gt;<br>    &lt;hr/&gt;<br>    &lt;h2&gt;Season Greetings!&lt;/h2&gt;<br>    &lt;p&gt;Nothing beats a Sale from Holiday Haven!&lt;/p&gt;<br>  &lt;/body&gt;<br>&lt;/html&gt;<br>EOF<br><br>chown www-data:www-data /var/www/html/index.html || true  # 9) Set ownership (don’t fail if it’s already correct)</pre><p><strong><em>Output.tf file construction</em></strong></p><p>I also put together the output.tf file, which defines the output values that shows the information about the infrastructure.</p><pre># Networking Ouputs<br>output &quot;vpc_id&quot; {<br>  value       = data.aws_vpc.jenkins.id<br>  description = &quot;VPC used for stack&quot;<br>}<br><br>output &quot;public_subnet_a_id&quot; {<br>  value       = data.aws_subnet.public_a.id<br>  description = &quot;Existing public subnet A&quot;<br>}<br><br>output &quot;public_subnet_b_id&quot; {<br>  value       = aws_subnet.public_b.id<br>  description = &quot;New public subnet B&quot;<br>}<br><br>output &quot;subnet_a_az&quot; {<br>  value       = data.aws_subnet.public_a.availability_zone<br>  description = &quot;AZ of subnet A&quot;<br>}<br><br>output &quot;subnet_b_az&quot; {<br>  value       = aws_subnet.public_b.availability_zone<br>  description = &quot;AZ of subnet B&quot;<br>}<br><br>output &quot;public_route_table_id&quot; {<br>  value       = data.aws_route_table.public_rt_for_a.id<br>  description = &quot;Public route table (0.0.0.0/0 -&gt; IGW)&quot;<br>}<br><br>output &quot;asg_subnet_ids&quot; {<br>  value       = local.asg_subnet_ids<br>  description = &quot;Subnets the ASG spans (2 subnets)&quot;<br>}<br><br># Security Group Outputs<br>output &quot;instance_security_group_id&quot; {<br>  value       = aws_security_group.web_sg.id<br>  description = &quot;Security group applied to ASG instances&quot;<br>}<br><br># Launch Template &amp; ASG Outputs<br>output &quot;launch_template_id&quot; {<br>  value       = aws_launch_template.web.id<br>  description = &quot;Launch Template ID used by the ASG&quot;<br>}<br><br>output &quot;asg_name&quot; {<br>  value       = aws_autoscaling_group.web.name<br>  description = &quot;Auto Scaling Group name&quot;<br>}<br><br>output &quot;asg_arn&quot; {<br>  value       = aws_autoscaling_group.web.arn<br>  description = &quot;Auto Scaling Group ARN&quot;<br>}<br><br>output &quot;asg_capacity&quot; {<br>  value = {<br>    min     = aws_autoscaling_group.web.min_size<br>    desired = aws_autoscaling_group.web.desired_capacity<br>    max     = aws_autoscaling_group.web.max_size<br>  }<br>  description = &quot;ASG capacity settings&quot;<br>}</pre><p><strong><em>The launch template, Auto-Scaling Group, and the Main.tf construction</em></strong>:</p><p>I moved forwarded with constructing the main.tf.</p><p>The launch template defines how to build one instance within the auto-scaling group.</p><p>The auto-scaling group defines the quantity of many instances are desired, the minimum and the maximum amount.</p><p>Lastly, added a security group that determines the “traffic talk” with the instances ( HTTP/HTTPS internet traffic, and the SSH connection with my IP).</p><pre><br># Security Group for ASG instances<br>resource &quot;aws_security_group&quot; &quot;web_sg&quot; {<br>  name        = &quot;${var.app_name}-sg&quot;<br>  description = &quot;Allow HTTP/HTTPS from the Internet and SSH from my IP&quot;<br>  vpc_id      = data.aws_vpc.jenkins.id<br><br>  # outbound anywhere<br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br><br>  # HTTP 80 from Internet<br>  ingress {<br>    description = &quot;HTTP from Internet&quot;<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = var.http_cidrs<br>  }<br><br>  # HTTPS 443 from Internet <br>  ingress {<br>    description = &quot;HTTPS from Internet&quot;<br>    from_port   = 443<br>    to_port     = 443<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = var.https_cidrs<br>  }<br><br>  # SSH for admin profile<br>  ingress {<br>    description = &quot;SSH from admin IP&quot;<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [var.ssh_cidr]<br>  }<br><br>  tags = {<br>    Name = &quot;${var.app_name}-sg&quot;<br>  }<br>}<br><br><br># Launch Template (Ubuntu + Apache user_data)<br>resource &quot;aws_launch_template&quot; &quot;web&quot; {<br>  name_prefix   = &quot;${var.app_name}-lt-&quot;<br>  image_id      = var.ami_id<br>  instance_type = var.instance_type<br>  key_name      = var.key_name<br><br><br>  network_interfaces {<br>    associate_public_ip_address = true<br>    security_groups             = [aws_security_group.web_sg.id]<br>  }<br><br>  # User data installs Apache and serves a simple page<br>  user_data = base64encode(file(&quot;${path.module}/user_data_apache.sh&quot;))<br><br>  update_default_version = true<br><br>  # tag EC2 instances launched by this template<br>  tag_specifications {<br>    resource_type = &quot;instance&quot;<br>    tags = {<br>      Name = var.app_name<br>    }<br>  }<br><br>  lifecycle {<br>    create_before_destroy = true<br>  }<br>}<br><br><br># Auto Scaling Group<br>resource &quot;aws_autoscaling_group&quot; &quot;web&quot; {<br>  name             = &quot;${var.app_name}-asg&quot;<br>  min_size         = var.asg_min<br>  desired_capacity = var.asg_desired<br>  max_size         = var.asg_max<br><br>  # Use the two subnets (A from data source, B created earlier)<br>  vpc_zone_identifier       = local.asg_subnet_ids<br>  health_check_type         = &quot;EC2&quot;<br>  health_check_grace_period = 120<br>  capacity_rebalance        = true<br><br>  launch_template {<br>    id      = aws_launch_template.web.id<br>    version = &quot;$Latest&quot;<br>  }<br><br>  tag {<br>    key                 = &quot;Name&quot;<br>    value               = var.app_name<br>    propagate_at_launch = true<br>  }<br><br>  lifecycle {<br>    create_before_destroy = true<br>  }<br>}<br><br>terraform {<br>  backend &quot;s3&quot; {}<br>}</pre><p><strong><em>Terraform commands (init, fmt, validate and apply):</em></strong></p><p>Now, I have all of the files constructed and ready to run terraform.</p><p>First, I used terraform init.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*r0aJ9nczAfRBE91z22Nwsw.png" /></figure><p>Next, I used terraform fmt and terraform validate. The fmt command forms the configuration and validate checks the syntax of my files to help catch errors prior to deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*pVSdZuxGh4VqaT0kDJeTFg.png" /></figure><p>This was successful and now I ran terraform plan -out=tfplan. This shows me the execution plan for the infrastructure changes and resources that will be added.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*okoWfrni98mc5r-PR7ctAg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*VhxBSeO-8KQYNWMwWaDtZA.png" /></figure><p>Next step is to finally run terraform apply tfplan. This command execute the planned changes and add the defined resources in the configuration files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jvV750l_S4kn1imjYY01fg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/679/1*mhHl1Fgy7NWzRq_SnFZQHw.png" /></figure><p>After this was completed, I went to the AWS Console and verify the resources were added.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*rWoOuHLf3m2N3EWVE7M4WA.png" /><figcaption>The routing table shows the 2 subnets.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*vfoOO1A6EMZ_ZewlCnTuFw.png" /><figcaption>I have the original subnet and added a new subnet for the auto-scaling group.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/596/1*nHWi7LyTgKdvVIecTekbAw.png" /><figcaption>The security group was added.</figcaption></figure><p><strong><em>Testing the Auto-Scaling Group:</em></strong></p><p>With the resources now added, I first make sure I could access Apache from the web browser. I tested both EC2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*p0imxXqOCZNTN9Z-n3APoQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*ENTsCNBcMDf9UpeirRGU7w.png" /></figure><p>The next step was to terminate one of the EC2 instances, and the auto-scaling group should spin up another EC2 per my configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*cJHnIj2bz5FRkr2ZXm4B6A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/1*ahdItGGGvxUYNusoPBQSpQ.png" /></figure><p>Once this EC2 was terminated, I refreshed the page and the auto-scaling group spin up another instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IhzFRr5f1tFHzy7QjKX9kw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IhzFRr5f1tFHzy7QjKX9kw.png" /></figure><p>Then tested the browser to ensure I can still access Apache.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*LlSBmJE5rreWf3CwlBEw4Q.png" /><figcaption>Success!</figcaption></figure><p><strong><em>Creating a remote backend:</em></strong></p><p>Terraform keeps a state file that maps out the terraform resources to the AWS objects that has been created. This file is within the Terraform memory, and it’s within the desktop/laptop. Creating a remote backend in S3 allows this memory to be used for more than the creator and provides a versioned history to rollback if needed.</p><p>I created an S3 bucket to start the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/1*e8NRkzSHVg8IrUBDRNqlZQ.png" /></figure><p>Then I added a backend code block to configuration file. So, if the local file is lost of out-of-date, the S3’s versioning provides a safety net.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*Y4vMjMgfvoesGfUNm5BQog.png" /></figure><p>Finally, I push this project to GitHub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*IBShp85oZ08XIfptUtxidQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*RC6RB9E5dEF-rVW1vdBZJA.png" /></figure><p><strong>Challenges and Solutions:</strong></p><p><strong>Launch Template + Security Groups (the “facepalm” moment).</strong><br> I initially defined both a network_interfaces block <em>and</em> a top-level vpc_security_group_ids value in the Launch Template. AWS rejects that combination with: <em>“When a network interface is provided, the security groups must be a part of it.”</em> It felt like a small detail, but it completely blocked ASG creation. The fix was simple once I understood the rule: if you use network_interfaces, you must put the security groups <strong>inside that block</strong> and remove vpc_security_group_ids. After that, the ASG created cleanly.</p><p><strong>Security Group “description” causes recreation.</strong><br> I learned that changing an SG’s <strong>description</strong> forces Terraform to recreate the security group, which can ripple into downtime if you’re not careful. The better practice is to keep descriptive, human-friendly notes in <strong>tags</strong> instead. Tags are mutable and don’t force a replacement.</p><p><strong>Subnet and route table alignment .</strong><br> To span two AZs, I created a second public subnet and associated it with the same route table that points default traffic to the Internet Gateway. Missing that association leaves an instance stranded without public egress. Being explicit in Terraform (create the subnet, then associate it to the known public route table) kept the network predictable.</p><h3>Conclusion Insights:</h3><p>I now have a self-healing web tier: an Auto Scaling Group spanning two AZs that serves traffic from Apache, replaces failed nodes automatically, and is fully defined as code. Moving Terraform state to a versioned S3 backend felt like crossing a reliability threshold — collaboration-ready, rollbacks possibilities.</p><p>I got sharper at debugging real-world Infrastructure as Code issues: reading ASG/LT error messages and keeping repos clean (no giant binaries, no secrets). Honestly, seeing a terminated instance get replaced moments later never stops being satisfying.</p><p>Learning Terraform + AWS, this project caused some stress, but I learned durable lessons about networking, bootstrapping, and state management.</p><p>Thanks again for your time, see you soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70d655be8694" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying A Jenkins Server With Terraform.]]></title>
            <link>https://medium.com/@niyi.ayinde1/deploying-a-jenkins-server-with-terraform-5eb05461ce35?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5eb05461ce35</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Thu, 14 Aug 2025 06:46:38 GMT</pubDate>
            <atom:updated>2025-08-17T17:33:11.381Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1LSq5_rqKK9ls7-3po8JFA.png" /></figure><h3><strong>Introduction:</strong></h3><p>Hello everyone. I am back again to assist my DevOps team to build our CI/CD tool to create pipelines for DevOps projects.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Our goal is to create a Jenkins Server using Terraform so that it can be used in other environments and if any changes happen within the environment, they can be better tracked and monitored.</p><h3><strong>Solution Strategy:</strong></h3><p>My solution is to build a VPC with a subnet, internet gateway, route table and associated with an EC2 instance that will bootstrap a Jenkins server. I will also create a security group one for remote ssh connection and another for the Jenkins server. Next, I will add a S3 bucket for all Jenkins artifacts as well as block public access and enable versioning. Lastly, I will verify I can reach Jenkins through the browser, push the code to GitHub, and create a Terraform layout with the providers, variables and outputs split into separate files.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Terraform: </strong>is an infrastructure as code tool that lets one define both cloud and on-premises resources in human-readable configuration files that one can version, reuse, and share.</p><p><strong>Amazon VPC</strong>: define and launch AWS resources in a logically isolated virtual network.</p><p><strong>Amazon EC2:</strong> on-demand, scalable computing capacity in AWS Cloud.</p><p><strong>Jenkins</strong>: a free open-source tool that helps developers automate the process of building, testing and deploying code.</p><p><strong>Amazon S3:</strong> an object storage service that offers industry-leading scalability, data availability, security and performance.</p><p><strong>Visual Studio (VS) Code</strong>: a free and lightweight source code editor.</p><p><strong>GitHub</strong>: is web-based platform that hosts Git repositories, providing developers with tools for version control and collaboration.</p><p>At the end, my project layout will have all of these components:</p><pre>tf-jenkins-01/<br>├─ main.tf<br>├─ providers.tf<br>├─ versions.tf<br>├─ variables.tf<br>├─ outputs.tf<br>├─ user_data.sh<br>└─ .gitignore</pre><h3>Implementation Steps:</h3><p><strong>AWS Resources Setup</strong></p><p>First, I setup my Terraform with a main.tf. I wanted to first create this file with all of the AWS resources I would need such the VPC, subnet, EC2, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*KQQFVzF8a442U9eDt7ktpQ.png" /><figcaption>One Main.tf is known as a monolith, later I will show the main.tf without hardcoded data.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*KZtwo7a4N8oBe-AbeRDX8g.png" /></figure><p>I am generated and imported a key pair for the EC2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*cwBdQQatRLuGXJYBJuUPWA.png" /></figure><p>Once I completed this, I moved to the terminal and ran Terraform init.</p><p><em>Note: The </em>terraform init<em> command initializes a working directory containing Terraform configuration files.</em></p><p>Then I used the terraform fmt and terraform validate to ensure the main.tf file is format correct and authenticates the file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/1*p7T-dnxumtUm-MlzoxdyVQ.png" /></figure><p><em>Note</em>: <em>The </em>terraform fmt <em>command formats Terraform configuration file contents so that it matches the canonical format and style.</em></p><p><em>The </em>terraform validate <em>command validates the configuration files in a directory.</em></p><p>After this, I used terraform plan to review what changes are going to happen prior to executing the main.tf.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*Td3-cU0b4b_QfvDOVfbcMg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/608/1*qlVy9Y0aQv7WoTygo0d66Q.png" /></figure><p><em>Note: The </em>terraform plan <em>command creates an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure.</em></p><p>Once I see that it shows Plan: 1 to add, 0 to change, 0 to destroy, I can move forward with the final command of terraform apply, which implements the plan.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*BsQFcEUAkXw1qJlOA5dpPw.png" /></figure><p><em>Note: The</em> terraform apply <em>command executes the actions proposed in a Terraform plan.</em></p><p><strong><em>Review AWS Console to validate Resources:</em></strong></p><p>Now, I went into the AWS console just to ensure that all the resources are now there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/1*2KgNE0cKpLeP5eoQMfPjcw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*1-7VIKSLVF0Me8IaIY_GIg.png" /></figure><p><strong><em>Creating the user_data.sh file to load and start Jenkins:</em></strong></p><p>To add Jenkins to my EC2, I used the Jenkins official documentation to finding my distribution type. I actually tried to embed the script inside of the main.tf, that did not work. VS code did not agree with this action, mainly because Terraform is very picky about quotes and line endings.</p><p>When the EC2 instance first boots, <strong>cloud-init</strong> parses user_data and runs it as <strong>root</strong> during its final stage.</p><ol><li>I logged into the EC2, updated all the packages.</li><li>Install Java (Jenkins runtime dependency).</li><li>Set up the official Jenkins apt repository (signed).</li><li>Install Jenkins and start it as a systemd service.</li><li>Surface the initial admin password so I could unlock the UI.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*UubfmKEtIobZKwXEWFjKjw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HobCeEpDd_FPduiI8bfA0w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1013/1*0UgIRIn9HnKGEJIcQxMnHA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*IqjgglXPYlEqhI6mQVpZ0g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*QE5miiKj4G-yyAO8ZC-AWA.png" /></figure><p>Then added this to the main.tf.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*zRzEL7fDhR0aN27g1lF4EQ.png" /></figure><p>I also verified that my security group. The security group allows port 22 and port 8080.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dPhmCnbfEizafV6VepgZMg.png" /></figure><p>Next, I verified that my S3 Bucket exists, and it blocks all public access and I enabled versioning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*BjX5RyLQOQ0sDKRYVzRryA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*GBvSUNajqW6qh4Muj9mznQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*Pt0j4whER97djKcdoox2QA.png" /></figure><p><strong><em>Verify Jenkins:</em></strong></p><p>To access Jenkins, I obtained the admin password that was generate from the user_data.sh.</p><p>I used this command to grab the password:</p><pre># over SSH<br>sudo cat /var/lib/jenkins/secrets/initialAdminPassword</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*meyZqgGBdyB2SHwsiFAzkw.png" /></figure><p>Then, headed over to the browser and used this to log into Jenkins with the public IP.</p><p>I setup the account and now Jenkins is running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*sm-5roolJKpYz9iNYRJWDw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*7PU86_gq91CJ80G3qr_v5Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/1*RWSBebV-9hxMhVtSUji4ag.png" /></figure><p><strong><em>GitHub Push:</em></strong></p><p>Link: <a href="https://github.com/NiyiNeo/terraform.jenkins.aws">https://github.com/NiyiNeo/terraform.jenkins.aws</a></p><p>Next, I pushed my code and documents into a GitHub Repo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1016/1*MMtUaoZhHN05625iIfCLbg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*WmMM__ND98ZSIEYz21qzxg.png" /></figure><p><em>Note: I also add a .gitignore to keep state and keys out of GitHub.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/793/1*OjXMABeOdFf-gdTpw_55hQ.png" /></figure><p><strong><em>Added Variables.tf, Providers.tf, Outputs.tf files</em></strong></p><p>The first part of the project, I used hardcoded data to complete the main.tf. Best practice is always to not hardcoded data into any script. With Terraform, I can create different documents to hold the hardcode data and update the main.tf with “variables”.</p><p>I separated each document out as such:</p><p>providers.tf</p><pre>provider &quot;aws&quot; {<br>  region  = var.aws_region<br>  profile = var.aws_profile<br>}</pre><p>versions.tf</p><pre>terraform {<br>  required_version = &quot;&gt;= 1.5.0&quot;<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 5.0&quot;<br>    }<br>  }<br>}</pre><p>variables.tf</p><pre>variable &quot;aws_region&quot; {<br>  description = &quot;AWS region to deploy resources&quot;<br>  type        = string<br>  default     = &quot;us-east-1&quot;<br>}<br><br>variable &quot;aws_profile&quot; {<br>  description = &quot;The aws profile for this project&quot;<br>  type        = string<br>  default     = &quot;tf.admin&quot;<br>}<br><br>#VPC Setup<br>variable &quot;vpc_cidr&quot; {<br>  description = &quot;The CIDR block for the VPC&quot;<br>  type        = string<br>  default     = &quot;10.0.0.0/16&quot;<br>}<br><br>variable &quot;public_subnet_cidr&quot; {<br>  description = &quot;The CIDR block for the public subnet&quot;<br>  type        = string<br>  default     = &quot;10.0.1.0/24&quot;<br>}<br><br>variable &quot;availability_zone&quot; {<br>  description = &quot;Availability Zone&quot;<br>  type        = string<br>  default     = &quot;us-east-1a&quot;<br>}<br><br>#Security Groups<br>variable &quot;jenkins_cidr&quot; {<br>  description = &quot;Access into the Jenkins server&quot;<br>  type        = string<br>  default     = &quot;Your-Ip address/32&quot;<br>}<br><br>variable &quot;ssh_cidr&quot; {<br>  description = &quot;Secure access to EC2&quot;<br>  type        = string<br>  default     = &quot;Your-Ip address/32&quot;<br>}<br><br>#EC2 Instance Details<br>variable &quot;ami_id&quot; {<br>  description = &quot;AMI of the EC2 instance&quot;<br>  type        = string<br>  default     = &quot;ami-0a7d80731ae1b2435&quot;<br>}<br><br>variable &quot;instance_type&quot; {<br>  description = &quot;Type of EC2 Instance&quot;<br>  type        = string<br>  default     = &quot;t2.micro&quot;<br>}<br><br>variable &quot;key_name&quot; {<br>  description = &quot;Key value for the EC2 instance&quot;<br>  type        = string<br>  default     = &quot;tf-admin-key&quot;<br>}<br><br>variable &quot;instance_name&quot; {<br>  description = &quot;Name of the EC2 instance&quot;<br>  type        = string<br>  default     = &quot;jenkins-ec2&quot;<br>}<br><br><br>#S3 bucket Details<br>variable &quot;bucket_name&quot; {<br>  description = &quot;Unique Global name of the S3 bucket for Jenkins artifacts&quot;<br>  type        = string<br>  default     = &quot;jenkins-artifacts-tf-22005&quot;<br>}<br><br>variable &quot;enable_bucket_versioning&quot; {<br>  description = &quot;enabling s3 bucket versioning&quot;<br>  type        = bool<br>  default     = true<br>}</pre><p>outputs.tf</p><pre>#EC2 Instance<br>output &quot;instance_public_ip&quot; {<br>  description = &quot;The public IP address of the EC2 instance&quot;<br>  value       = aws_instance.jenkins_ec2.public_ip<br>}<br><br>#Network Output<br>output &quot;public_dns&quot; {<br>  description = &quot;The public dns for this project&quot;<br>  value       = aws_instance.jenkins_ec2.public_dns<br>}<br><br>output &quot;instance_id&quot; {<br>  description = &quot;The ID of the EC2 instance&quot;<br>  value       = aws_instance.jenkins_ec2.id<br>}<br><br>output &quot;instance_az&quot; {<br>  description = &quot;The availability zone of the EC2 instance&quot;<br>  value       = aws_instance.jenkins_ec2.availability_zone<br>}<br><br>output &quot;jenkins_url&quot; {<br>  description = &quot;Convenient URL to Jenkins (HTTP)&quot;<br>  value       = &quot;http://${aws_instance.jenkins_ec2.public_ip}:8080&quot;<br>}<br><br>output &quot;public_subnet_id&quot; {<br>  description = &quot;THe ID of the public subnet&quot;<br>  value       = aws_subnet.public_jsn.id<br>}<br><br>output &quot;public_security_group_id&quot; {<br>  description = &quot;The ID of the public security group&quot;<br>  value       = aws_security_group.jenkins_sg.id<br>}<br><br>#S3 Bucket<br>output &quot;s3_bucket_name&quot; {<br>  description = &quot;Name of the S3 Bucket for Jenkins artifacts&quot;<br>  value       = aws_s3_bucket.jenkins_s3.bucket<br>}<br><br>output &quot;s3_bucket_arn&quot; {<br>  description = &quot;S3 bucket ARN for Jenkins Artifacts&quot;<br>  value       = aws_s3_bucket.jenkins_s3.arn<br>}</pre><p>Last but not least, the update main.tf</p><pre>#  VPC Networking <br>resource &quot;aws_vpc&quot; &quot;jenkins_vpc&quot; {<br>  cidr_block           = var.vpc_cidr<br>  enable_dns_support   = true<br>  enable_dns_hostnames = true<br>  tags                 = { Name = &quot;jenkins-vpc&quot; }<br>}<br><br>resource &quot;aws_subnet&quot; &quot;public_jsn&quot; {<br>  vpc_id                  = aws_vpc.jenkins_vpc.id<br>  cidr_block              = var.public_subnet_cidr<br>  availability_zone       = var.availability_zone<br>  map_public_ip_on_launch = true<br>  tags                    = { Name = &quot;jenkins-public-jsn&quot; }<br>}<br><br>resource &quot;aws_internet_gateway&quot; &quot;igw&quot; {<br>  vpc_id = aws_vpc.jenkins_vpc.id<br>  tags   = { Name = &quot;jenkins-igw&quot; }<br>}<br><br>resource &quot;aws_route_table&quot; &quot;public&quot; {<br>  vpc_id = aws_vpc.jenkins_vpc.id<br>  route {<br>    cidr_block = &quot;0.0.0.0/0&quot;<br>    gateway_id = aws_internet_gateway.igw.id<br>  }<br>  tags = { Name = &quot;jenkins-rt-public&quot; }<br>}<br><br>resource &quot;aws_route_table_association&quot; &quot;public_jsn&quot; {<br>  subnet_id      = aws_subnet.public_jsn.id<br>  route_table_id = aws_route_table.public.id<br>}<br><br><br># Security Group <br>resource &quot;aws_security_group&quot; &quot;jenkins_sg&quot; {<br>  name        = &quot;jenkins-sg&quot;<br>  description = &quot;Allow Jenkins on (8080) and SSH (22) from my public IP&quot;<br>  vpc_id      = aws_vpc.jenkins_vpc.id<br><br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br><br>  ingress {<br>    description = &quot;Jenkins UI&quot;<br>    from_port   = 8080<br>    to_port     = 8080<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [var.jenkins_cidr]<br>  }<br><br>  ingress {<br>    description = &quot;SSH&quot;<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [var.ssh_cidr]<br>  }<br><br>  tags = { Name = &quot;jenkins-sg&quot; }<br>}<br><br><br># Create the EC2 Instance<br>resource &quot;aws_instance&quot; &quot;jenkins_ec2&quot; {<br>  ami                         = var.ami_id<br>  instance_type               = var.instance_type<br>  subnet_id                   = aws_subnet.public_jsn.id<br>  vpc_security_group_ids      = [aws_security_group.jenkins_sg.id]<br>  associate_public_ip_address = true<br>  key_name                    = var.key_name<br><br>  user_data_replace_on_change = true<br>  user_data                   = file(&quot;${path.module}/user_data.sh&quot;)<br><br>  tags = { Name = var.instance_name }<br>}<br><br>#S3 Bucket Creation<br>resource &quot;aws_s3_bucket&quot; &quot;jenkins_s3&quot; {<br>  bucket = var.bucket_name<br><br>  tags = {<br>    Name = var.bucket_name<br>  }<br>}<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;S3_Policy&quot; {<br>  bucket                  = aws_s3_bucket.jenkins_s3.id<br>  block_public_acls       = true<br>  block_public_policy     = true<br>  ignore_public_acls      = true<br>  restrict_public_buckets = true<br>}<br><br>resource &quot;aws_s3_bucket_versioning&quot; &quot;jenkins_s3&quot; {<br>  bucket = aws_s3_bucket.jenkins_s3.id<br><br>  versioning_configuration {<br>    status = var.enable_bucket_versioning ? &quot;Enabled&quot; : &quot;Suspended&quot;<br>  }<br>}</pre><p>Now, my project layout is updated as expected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/349/1*kLHj4r2udu55wjAZCEiwKw.png" /></figure><h3><strong>Challenges and Solutions:</strong></h3><p>I hit several very real “why is this orange? why won’t Jenkins start?” moments. My first pass shoved a long heredoc into main.tf—VS Code lit it up, and Terraform refused to parse it. I stepped back, created a user_data.sh with LF endings, and pointed user_data = file(...) at it. That one change eliminated a whole class of quoting problems.</p><p>Next, I tripped over my own names: I typed public_a in some places while I had defined public_jsn. Terraform’s “undeclared resource” error was brutally honest; I searched for the mismatch and fixed it everywhere. I corrected the bootstrap with the new Jenkins keyring path and the service finally came up on port 8080. The last mistake was self-inflicted: I tweaked the Security Group description and Terraform wanted to replace the whole SG. That taught me AWS treats SG descriptions as immutable, so I reverted the string and moved on.</p><h3><strong>Conclusion Insights:</strong></h3><p>With this project, I designed and deployed a complete Jenkins environment on AWS using Terraform, starting with a clean VPC foundation (VPC, public subnet, internet gateway, route table + association) and a least-privilege Security Group that only permits SSH (22) and the Jenkins UI (8080) from my /32. I provisioned an Ubuntu 22.04 EC2 instance and fully automated the Jenkins install with a user_data.sh bootstrap script: it adds the signed Jenkins APT repo via a keyring, installs OpenJDK 17 and Jenkins, enables and starts the systemd service, and logs the initial admin password to syslog for quick retrieval. I verified the deployment end-to-end by reaching http://&lt;public-ip&gt;:8080, unlocking Jenkins, and confirming the service status on the instance. To store build artifacts securely, I created a private S3 bucket with “Block Public Access” enforced and versioning enabled.</p><p>I then refactored the codebase to follow Terraform best practices: providers and version constraints in dedicated files, all environment-specific values parameterized in variables.tf, clear outputs.tf, and a concise main.tf that wires resources together. The workflow is reproducible (fmt → init → validate → plan → apply) and state-safe; changing user_data intentionally replaces only the instance. The finished project demonstrates practical Terraform fluency, secure AWS networking, reliable bootstrap automation, and disciplined iteration, and is published to GitHub with an appropriate .gitignore to keep state and keys out of version control.</p><p>Thanks once again for reading!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5eb05461ce35" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying a Scalable WordPress Site and MySQL with Docker Swarm on AWS.]]></title>
            <link>https://medium.com/@niyi.ayinde1/deploying-a-scalable-wordpress-site-and-mysql-with-docker-swarm-on-aws-5c5d6cf553c4?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c5d6cf553c4</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Mon, 04 Aug 2025 14:28:42 GMT</pubDate>
            <atom:updated>2025-08-04T14:28:42.504Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6OVM40NajDIKvDHtcKRJLQ.png" /></figure><h3><strong>Introduction:</strong></h3><p>Greetings all. I have been asked by a client, Level Up Publications, to provide a solution for their company. Level Up Publications is an online publication that posts blogs about the hottest updates and trends in IT. Level Up Publications wants to transition their web infrastructure into a containerized environment using Docker Swarm.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Level Up Publications wants to leverage the use of Docker Swarm for a few reasons. With this implementation, it significantly simplifies the process of managing and scaling applications. Next, Level Up Publications can easily scale up to meet peak times and scale down during off-peak times, ensuring efficient use of resources and cost savings. Also, this implementation can improve the resilience of the system. If a container fails, Docker Swarm can automatically replace it, minimizing downtime.</p><h3><strong>Solution Strategy:</strong></h3><p>My solution is to deploy a 3 node Docker Swarm cluster with two networks, one for the front end and the back end. After this, I will create two services, MySQL and WordPress. The MySQL services will be associated with backend network with 1 replica, and I will use a Docker Volume to ensure the database service is resilient and the data persistency. For the WordPress service, I will associate this with the front end network, create 3 replicas, and associate it with MySQL.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Docker</strong>: an open platform for developing, shipping, and running applications. Docker enables the user to separate applications from the infrastructure, so the user can deliver software quickly.</p><p><strong>Docker Swarm:</strong> a container orchestration tool that allows a user to manage a cluster of Docker nodes as a single virtual system</p><p><strong>Amazon EC2 instances</strong> <em>(Ubuntu): </em>on-demand, scalable computing capacity in AWS Cloud.</p><p><strong>Visual Studio (VS) Code</strong>: a free and lightweight source code editor.</p><p><strong>MySQL: </strong>an open source, relational database management system that uses Structured Query Language (SQL) to manage and manipulate data.</p><p><strong>WordPress:</strong> a free and open-source content management system (CMS) that allows users to create and manage websites or blogs.</p><h3><strong>Implementation Steps:</strong></h3><p><strong><em>Setup EC2 Instances:</em></strong></p><p>First, I setup the infrastructure for Docker Swarm with 3 EC2 instances and ensure that they are all inside the same VPC, subnet, and the same key pair.</p><p>I also tagged each instance as lup-swarm-manager, lup-swarm-worker1, and lup-swarm-worker2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/1*e_nxG2HsLimjtMcoS6Femg.png" /></figure><p>Once the instances are up and running I logged into each instance with VS Code, updated all the packages, and then installed docker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ynDTCDSvu5A01h8VoOiPLQ.png" /></figure><p>After this, I ensure that docker was running.</p><pre>sudo systemctl enable docker<br>sudo systemctl start docker </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YumS0PpRuCVeLnIgOU2JHA.png" /></figure><p><strong><em>Setting Up Docker Swarm:</em></strong></p><p>Then I updated all the names of the nodes to make everything easier to track with this command.</p><pre>sudo hostnamectl set-hostname &lt;name of host&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*edbQ5cVrWn3g7zn_EPmw8Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aacgvbT878ByO-nAbK8_fQ.png" /></figure><p>I went back to the manager node to generate the rsa key for connectivity for all of the nodes.</p><pre>ssh-keygen -t rsa -b 4096 </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*P1moNruw5kAwJ2KyfVQWSA.png" /></figure><p>Next, I mapped all the private IPs to the /etc/host file for the 3 nodes prior to setting up the swarm and pinged each worker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*ugs0VRSxVyA9ZRqB6Oj9Ug.png" /></figure><p>Now, I initialized the Swarm and went into the manager node and use this command:</p><pre>docker swarm init --advertise-addr &lt;manager-private-IP&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*1_bOZW_6g7qKn4UNuxR4dQ.png" /></figure><p>Then I joined the worker nodes into the swarm using this command</p><pre>docker swarm join --token &lt;worker-token&gt; &lt;manager-private-IP&gt;:2377</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*eruhIU7TsvRs4eZEJxtfQw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*3ZMMPhSfFnWOcpuOoHGhTg.png" /></figure><p>Then I verified all the nodes where inside the swarm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/555/1*aePhP0G6jC9NswGKy8CWlQ.png" /></figure><p><strong><em>Creating an Overlay Network:</em></strong></p><p>Docker swarm uses overlay networks to allow service on different nodes to communicate securely across the cluster.</p><p>I created two networks, and these networks are associated with WordPress and MySQL services.</p><p>I ran this command:</p><pre><br>docker network create --driver overlay frontend-net<br><br>docker network create --driver overlay backend-net</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*O7_IhFHcAi6foJFcg08N7A.png" /></figure><p><em>Note: driver overlay = This creates a multi-host network across the Swarm.</em></p><p><em>Attachable = allows standalone containers (not just services) to connect to the network if needed.</em></p><p>Then verified both networks were created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/449/1*8w4pz9zp2OT-RcZN--DNFA.png" /></figure><p><strong><em>Deploy the MySQL Service with a Named Volume:</em></strong></p><p>Now, this part came with various challenges from connection issues to having no access to database at times.</p><p>I had to associated this with the backend network, create 1 replica and use Docker Volume to ensure the database service is resilient and data is persistent.</p><p>I will highlight some of these challenges toward the end, but through lot of patience, energy drinks, tea, and more tea, I was able to get a successful output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/615/1*HMGCwCJFAlOXwTCsyWGiRw.png" /></figure><p><strong><em>Deploying the Wordpress Service:</em></strong></p><p>I had similar struggles with the WordPress service.</p><p>I had to associated with the frontend network, with port 80, have 3 replicas, associate with MySQL service, and finally have WordPress accessible in the browser.</p><p><em>Note: I used 3 replicas because it simulates high availability and distributed loads, demonstrating Swarm’s service orchestration.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/477/1*kb-B9ChCpp4IkRklPx9AsA.png" /></figure><p>Then, I went back into MySQL to verify that the database was there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/610/1*rouVwmY01VNoFhrSFChSQQ.png" /></figure><p>Now, after all this, I tested the browser to see if WordPress setup page would be launched with the node’s IP address.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*zVQDKBrF_qlV9L7x0Dappw.png" /></figure><p>I completed the login with the Level Up Publication information and WordPress was installed successfully!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/939/1*Ic5YvMQSX6V_UxoNkHiCxg.png" /></figure><h3><strong>Major Troubleshooting Hurdles &amp; Solutions:</strong></h3><p>As rewarding as this project turned out to be, it was far from a smooth ride. Each success was hard-earned through repeated testing, digging through documentation, and navigating moments of uncertainty. There were several points where I felt stuck, frustrated, and even questioned whether I had taken the right approach, but with each challenge, I committed to pushing forward, staying professional, and finding a solution.</p><p>One of the earliest and most frustrating errors was seeing the message:<br> <strong>ERROR 2003 (HY000): Can&#39;t connect to MySQL server on &#39;mysql-db&#39; (110)</strong>.<br> At first glance, it felt like a normal connectivity error, but it ended up having multiple underlying issues. I began troubleshooting by checking whether the MySQL container was even running—using docker ps and docker service ps mysql-db. To my surprise, it was running—but not where I expected. The container had landed on a Swarm worker node rather than the manager, which I had been using to run test connections.</p><p>It took some time to realize that the container placement was random across the Swarm cluster. This explained why some of my docker run tests failed, because they were occurring on nodes where the MySQL container wasn’t present. I added a node constraint (--constraint &#39;node.hostname == lup-swarm-manager&#39;) to ensure that the MySQL service would always deploy on the manager node. Once I did that and confirmed its presence via docker ps, breathed a sigh of relief.</p><p>Another headache, the database password. I kept encountering authentication errors even though I was certain I had set the environment variable correctly. After rechecking the syntax and re-creating the service more than once, I realized that environment variables can be cached in certain cases. The fix was to fully remove and recreate the service while being intentional with the password settings. This experience reinforced for me how important it is to be precise and consistent when working with Docker Swarm services.</p><p>There was also a moment where I suspected security group misconfiguration might be the root cause. I double-checked the inbound rules for my EC2 security group and confirmed that MySQL (TCP port 3306) was open to the correct CIDR block (10.0.0.0/16). While this wasn&#39;t the cause in this case, it reminded me that cloud networking misconfigurations are always worth validating early in the troubleshooting process.</p><p>Probably the number one of all of the frustrating issues involved using the mysql:5.7 Docker image for connectivity testing. I wanted to run a debug container to manually connect to mysql-db, but the image lacked common tools like ping, which threw me off at first. Instead of wasting time installing tools into the container, I learned to rely on the MySQL CLI to validate connectivity directly. That small pivot saved me hours.</p><p>What made these hurdles even more taxing was the repetition. After what felt like the tenth failed attempt, I started to second-guess myself. Was my Swarm broken? Did I misconfigure something at the VPC level? Still, I forced myself to take breaks, return to the Docker and AWS docs, and re-approach each issue with a fresh mind. I reminded myself that these moments of pain were where real learning happens.</p><p>Through all of this, I maintained professionalism, documented my steps, and stayed calm, even though I felt overwhelmed or doubted whether I was getting anywhere. This determination ultimately led to the success of this project.</p><h3><strong>Conclusion Insights:</strong></h3><p><strong><em>What I achieved:</em></strong></p><ul><li>Built and orchestrated a multi-node Swarm cluster</li><li>Deployed stateful MySQL database using volumes</li><li>Integrated and exposed WordPress service publicly</li><li>Debugged complex inter-node networking issues</li><li>Applied real-world troubleshooting strategies under pressure</li></ul><p>This one was a test of grit. I faced issues that weren’t Google-able in one shot. I learned to slow down, verify container placement, analyze logs, revisit assumptions, and keep pushing forward.</p><p>There were moments I considered blowing everything away and starting over.</p><p>But I stuck with it, and this was important reminder:</p><p>“Never confuse a single defeat with a final defeat.” — F. Scott Fitzgerald</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c5d6cf553c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[There is a Swarm coming: Building and Deploying a Docker Swarm Cluster on AWS EC2]]></title>
            <link>https://medium.com/@niyi.ayinde1/there-is-a-swarm-coming-building-and-deploying-a-docker-swarm-cluster-on-aws-ec2-8c752f8f4ae6?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/8c752f8f4ae6</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Sun, 27 Jul 2025 18:22:44 GMT</pubDate>
            <atom:updated>2025-07-27T18:22:44.695Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aZHRtH-f1KQ3A79oWa4MhA.png" /></figure><h3><strong>Introduction:</strong></h3><p>Hello, once again, I have been tasked by my company, NexaGlobal, to solve some issues we have had. Currently, NexaGlobal is having trouble with containerized workloads failing for our global shipping application, NG. The major concern is that there could be potential data loss as there is no alternative way to reschedule the containers that are failing and there is approximately one week to update the infrastructure.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>NexaGlobal wants to leverage the use of Docker Swarm. With this implementation, the data can be restored for the containers that are no longer in a healthy state. Some team members are not familiar with Docker Swarm, so I was asked to assist.</p><h3><strong>Solution Strategy:</strong></h3><p>I will put together a container orchestration deployment using <strong>Docker Swarm</strong> across <strong>three AWS EC2 Ubuntu instances. </strong>I will take the following steps to achieve this goal:</p><p>· Install Docker on all hosts</p><p>· Change the hostname on each node to easily identified, one as the manager node and the two worker nodes</p><p>· Ensure that there is an SSH Key on each node</p><p>· Run all the containers using the command-line interface (CLI)</p><p>· Deploy a container with <strong>highly available Nginx service</strong> across the cluster</p><p>· Document and validate every step as if onboarding a new team member</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Docker Engine (CLI) </strong>: an open-source containerization technology that allows you to build, run, and manage applications in lightweight containers. It consists of a server (daemon), APIs for communication, and a CLI for user interaction.</p><p><strong>Docker Swarm:</strong> a container orchestration tool that allows a user to manage a cluster of Docker nodes as a single virtual system.</p><p><strong>Amazon EC2 instances</strong> <em>(Ubuntu 20.04): </em>on-demand, scalable computing capacity in AWS Cloud.</p><p><strong>Nginx: </strong>open-source web server software that can also function as a reverse proxy, load balancer, and HTTP cache.</p><p><strong>OpenSSH</strong>: a tool for secure system administration, file transfers, and other communication across the Internet or other untrusted network.</p><h3><strong>Implementation Steps:</strong></h3><p><strong><em>Setting up AWS EC2 (Nodes) for Docker</em></strong></p><p>First, I used the AWS Console to create 3 Ubuntu 20.04 EC2 instances (t2 micro). Each host was put into the same VPC and security group. Lastly, I created .pem key to SSH into each instance.</p><p>Note: In a production deployment, each host would ideally have their own SSH key for better access control and auditing purposes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*7N2FpEG3G9sBRqwvZWItYQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*bkdWrIH9OG9TsRHWPIaggg.png" /></figure><p><strong><em>Install Docker on All Nodes:</em></strong></p><p>Once all nodes were setup, I went to VS Code to install Docker on all the nodes.</p><p><em>Note: A node is physical or virtual machine that is running a Docker Engine. This machine serves as a host for Docker containers, providing the necessary resources (CPU, memory, storage, network) for them to operate.</em></p><p>I first update all packages</p><pre>sudo apt update</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*5UeD6S-fZSNdnRM-7kbz-Q.png" /></figure><p>Then I installed docker.</p><pre>sudo apt install docker.io -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/929/1*EQbiUh5mLM0Ysd037QaCfQ.png" /></figure><p>After this, I enabled and started docker and rinsed and repeated for all 3 hosts.</p><pre>sudo systemctl start docker<br>sudo systemctl enable docker</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MCsDO_CEZ5gVQewOy8TUmQ.png" /></figure><p>Then I verfied that Docker is running on all the hosts.</p><pre>sudo systemctl status docker</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yn8slX1jG8oKEQySeUYZzQ.png" /></figure><p><strong><em>Change the host name for each node. (1 manager and 2 workers):</em></strong></p><p>Now, I changed the name of each host. This helps me easy identify who is the manager node and which are the worker nodes.</p><p><em>The manager node is responsible for handling the cluster manager tasks like maintaining the cluster state and scheduling services.</em></p><p><em>The worker nodes are the ones who execute containers.</em></p><p>I used the command below to update the host names and did this command for all 3 hosts:</p><pre>sudo hostnamectl set-hostname &lt;custom hostname&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1aNJP-zK8zOEpDXEwFO6wg.png" /></figure><p>I also ensure that all the worker nodes are the same security group as the manager node. I complete this task during the creation of the EC2 instances.</p><p><em>Note: This is important because Docker Swarm communication requires open ports among nodes and instances in the same security group makes things easier.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/1*UG2bR6N5ivmK3jZVkRpz5g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*7_SL7nKZmAHwiEgICrZKyw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*C5XPP5lqYrJQR8l4wB0yNA.png" /></figure><p><strong><em>Adding the SSH key to each node:</em></strong></p><p>The manager node does not have access to the worker nodes. So, I generated an SSH key so this will give the manage node access to both worker nodes.</p><pre>ssh-keygen -t rsa -b 4096</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/551/1*1HiK4lV-2grXIVCi-4NRlA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/354/1*xaJy2qnMzSJX38Iom_hwOA.png" /></figure><p>Now, I did have some troubles getting the manager node to access the other workers. I eventually had to obtain my .pem key and pair it will my SSH Key I created to assist in access the workers nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/1*IHj7etcPGbODbAzVQlKA2w.png" /></figure><p>After some troubleshooting, I was finally able to connect the nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*SjC-N8V4NWopreh7jt5gJg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*KShWvI20MG3jMnCRr-jcGA.png" /><figcaption>Manager node logging into worker 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/541/1*WUVQgtBMq-KwJTYdal1Puw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/503/1*VWhU_lpXNlWz3y_dL7pHoQ.png" /><figcaption>Manager node logging into worker 2</figcaption></figure><p><strong><em>Mapped Private IP address for Docker Swarm</em></strong></p><p>Mapping private IP addresses for all nodes in a Docker Swarm is said to be best practice for several reasons, primarily related to <strong>performance, security, and consistent internal communication </strong>for a distributed containerized application.</p><p>I used this command to add all the private IP addresses.</p><pre>sudo nano /etc/hosts</pre><p>Then I used this command to see if the nodes can be reached.</p><pre>ping -c 2 worker1<br>ping -c 2 worker2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*RdnIjwooSZ7KgYOZAFwE8g.png" /></figure><p><strong><em>Creating the Docker Swarm</em></strong></p><p><em>Note: Create the Docker Swarm cluster before running the containers, as the cluster setup is necessary for managing and orchestrating the containers effectively. Once the cluster is initialized, you can deploy your services and containers within it.</em></p><p><em>Initialize the Swarm on the Manager Node:</em></p><p>For the Docker Swarm, I have to set the Manager node. Once I do that I used this command to generate a token that will allow the other worker nodes to join.</p><pre>sudo docker swarm init --advertise-addr 10.0.1.219</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*Z25hXTSPeudX2V2tq54YbA.png" /></figure><p>Next step is to join the worker nodes to the swarm.</p><p>I first SSH into worker1 node.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/465/1*a_DbOjQ345lkAn2buko4HQ.png" /></figure><p>Then I used the token that was generate for each worker node to join the Swarm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/566/1*1S-FxeusE-pUxWwtjUF7WA.png" /></figure><p>Rinse and Repeat for worker2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*pqHJfskLnDYf9DPCu_frGg.png" /></figure><p>Then have worker2 join the Swarm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*e-OMdFGWt4pKkLoUzsqTYQ.png" /></figure><p>Then I use the command below to verify the status.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*1qDtrvMxRSaLceKZltd5bw.png" /><figcaption>.</figcaption></figure><p><em>Note: The line m8xzsn9tckbs9e2mquycf3888 * shows your current node is the </em><strong><em>Swarm Manager</em></strong><em> (* symbol confirms it), and its status is Leader. The nexa-swarm-worker1 and nexa-swarm-worker2 are listed with STATUS = Ready and AVAILABILITY = Active.</em></p><p><strong><em>Deploy NGINX</em></strong></p><p>Now that your Swarm is initialized and both workers have successfully joined, it’s time to deploy a real service.</p><p>I use <strong>Nginx</strong> because it’s widely used in real-world production as a reverse proxy, load balancer, and web server. Also lightweight and perfect for showcasing Swarm features like replication, failover, and load distribution.</p><p>To test this, I deployed Nginx with 3 replicas:</p><pre>sudo docker service create --name ngnix-web --publish 80:80 --replicas 3 nginx</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*vYXibun1qhdF1q-YRgyAPg.png" /><figcaption>Yes, I had a typo. I must always double-check! I just move forward with this as ngnix!</figcaption></figure><p>Then I verified the deployment with the following:</p><pre>sudo docker service ls<br>sudo docker service ps ngnix-web</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/560/1*DE5b9epfUbTrZD45YKkQhQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*FFif9UQZsntdw_dJdboe_w.png" /></figure><p>Lastly, I test the browser using the public IP Address for each node in the Swarm.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*D45WMpTV9M3TOt2H1AeO0Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*g7tQc90MaXNkoK9natrgrg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*Icz5O2UDHC5I3IpylrIoGQ.png" /></figure><p><strong><em>Status of the Docker nodes.</em></strong></p><p>The final piece was to show team members the status of the docker nodes and now they are all reachable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/562/1*0oLV7bmjJMnV75lT-uGOlw.png" /></figure><h3><strong><em>Challenges &amp; Solutions:</em></strong></h3><p><strong><em>SSH Key Propagation</em></strong></p><p><strong>Challenge:</strong> Couldn’t SSH between nodes without re-entering passwords.<br><strong>Solution:</strong> Used cat ~/.ssh/id_rsa.pub | ssh ... with proper permissions to propagate keys securely.</p><p><strong><em>Docker Swarm Permission Denied</em></strong></p><p><strong>Challenge:</strong> Received connect: permission denied to Docker socket.<br> <strong>Solution:</strong> Used sudo consistently on all Docker commands.</p><p><strong><em>Incorrect Swarm Join</em></strong></p><p><strong>Challenge:</strong> Accidentally ran swarm join from the manager while logged in as a worker.<br> <strong>Solution:</strong> Regenerated token on manager, properly exited SSH sessions, and joined from correct node.</p><p><strong><em>Remote CA Fingerprint Mismatch</em></strong></p><p><strong>Challenge:</strong> Mismatch in expected CA fingerprint.<br> <strong>Solution:</strong> Generated a <strong>fresh token</strong> using</p><h3><strong>Conclusion Insights:</strong></h3><p>This project pushed me to think like a real Cloud DevOps Engineer. Not just running scripted commands, but a problem-solver building toward automation, availability, and scale. I had to pause, reflect, research, and re-attempt several steps.</p><p>It forced me to simulate how a container orchestrator thinks in production environments. How do services recover? How do nodes communicate securely? How do I know if a deployment actually worked?</p><p>The CLI-centric approach gave me muscle memory and built confidence in the terminal. Networking configs, SSH tunneling, Swarm logic, and all of it becomes more real and less theoretical. I walked away with deeper insight into how infrastructure is automated and how services are built to withstand failure.</p><p>More importantly, I proved to myself that even in unfamiliar territory, I can figure it out and I can explain the process clearly to others.</p><p>Thanks again for your time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8c752f8f4ae6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Continuous Development and Deployment with Docker and Jenkins.]]></title>
            <link>https://medium.com/@niyi.ayinde1/continuous-development-and-deployment-with-docker-and-jenkins-168926cddfdb?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/168926cddfdb</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Sun, 20 Jul 2025 19:01:03 GMT</pubDate>
            <atom:updated>2025-07-21T13:57:27.400Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/627/1*w8lELqYLoUIuezl0E6rtOA.png" /></figure><h3><strong>Introduction:</strong></h3><p>Greetings all. Once again, I have been requested by my client, Level Up Solutions, to assist with a new project. Level Up Solutions wanted to continue with adopting tools and methodologies that can streamline their operations, improve their productivity and enhance the quality of their applications.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Level Up Solutions wants to use Docker for creating and managing Jenkins containers, expressing the implementation of persistent volumes. They are looking to implement a strategy that has a robust, scalable CI/CD pipeline, persistent data to implement resiliency and stability.</p><h3><strong>Solution Strategy:</strong></h3><p>I presented a solution that I will deploy a Jenkins CI/CD server using Docker on an EC2 instance, ensuring that configuration data persist even after the container is remove, with the use of Docker Volumes. This approach will not only optimize the company’s software development and deployment procedures, but also provide significant advantages, consistency, reliability, cost savings and efficiency.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p>Jenkins: a free open-source tool that helps developers automate the process of building, testing and deploying code.</p><p>Docker: an open platform for developing, shipping, and running applications. Docker enables the user to separate applications from the infrastructure, so the user can deliver software quickly.</p><p>Docker volume: volumes are persistent data stored in containers, created and managed by Docker.</p><p>Amazon EC2: on-demand, scalable computing capacity in AWS Cloud.</p><h3><strong>Implementation Steps:</strong></h3><p><strong><em>Log into EC2 and verify Docker:</em></strong></p><p>First I logged into the EC2 and once inside the terminal, I verified that my Docker is installed.</p><pre># Verified Docker Installation<br>docker --version<br><br># Verify image <br>sudo docker images</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y-mYkIJ6rgl0R4wK8SyJ2g.png" /></figure><p><strong><em>Pull the Jenkins image:</em></strong></p><p>I research the Jenkins documentation to find the latest image to use.</p><p>Below, I used the jenkins/jenkis:jdk17 for this project</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*k3DJf-P-0IHLKzBhxLicHw.png" /></figure><p>After this, I logged into Docker Hub. Again, I had to recreate a Personal Access Token (PAT), I successfully log into Docker Hub.</p><pre>sudo docker login -u &lt;dockerhub-username&gt; </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/943/1*CrNjX9trCGxDRRrX1mEcUg.png" /></figure><p>Next, I used the command below pull the Jenkins image into the container, then I verify the jenkins image was present.</p><pre>sudo docker pull jenkins/jenkins:lts-jdk17</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*qp8yNQC-kvGL0EfPG6xBkQ.png" /></figure><p><strong><em>Create Docker Volume for Jenkins:</em></strong></p><p>Once this was completed, I moved to the next phase in creating a Docker Volume.</p><pre>sudo docker volume create level-up-solutions-new</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*DAw_t0-W9sbK2UWRo4cFdw.png" /></figure><p>And of course, once created, I verified the volume was there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/469/1*cxptdrcDBUuOgF8ChSQV4Q.png" /></figure><p><strong><em>Run the Jenkins Container with Volume and Ports:</em></strong></p><p>Now, I can start Jenkins. I used the port mapping 8080:8080 (UI) and 50000:50000 (agents).</p><p>Then attached the Docker Volume created and mount to with /var/jenkins_home.</p><p>Then, I named the container: jenkins-level-up</p><pre>sudo docker run -d -p 8080: -p 50000:50000 -v level-up-solutions-new:var/jenkins_home --name jenkins-server jenkins/jenkins:lts-jdk17</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*7z08u3I3cCIfwH6FvKijKg.png" /></figure><p>And then I verified the configuration.</p><pre>sudo docker inspect jenkins-level-up </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/751/1*NKreffvEIJrSYWQnx8xd4Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/733/1*F9Xx7-aj0Tg685t-gzV73A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*LW26DQILZulCYeiSXiDy6g.png" /></figure><p><strong><em>Retrieve the Admin Password:</em></strong></p><p>The next step was to retrieve the Jenkins admin pass from inside the container. I searched online and I was able to find the command for this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/729/1*nFQINyXc49syyWKGcwU6ng.png" /></figure><pre>sudo docker exec -it jenkins-level-up cat /var/jenkins_home/secrets/initialAdminPassword</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k22JosiVhXnXUNy3kFwQ1g.png" /></figure><p><strong><em>Login to Jenkins and Create New User:</em></strong></p><p>I can now log into Jenkins and enter the admin password.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*hyVe8_-JLDAvUVYCBhHegQ.png" /></figure><p>After this, I installed the recommend plugins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*nge83LLFtYkZAdf1SaFJeQ.png" /></figure><p>Once the plugins were completed, I created a new user.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*cRqPMoHq7Udy7i0GBeYOoA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*XpzbRHukGuJhn_tGo7J4vg.png" /></figure><p><strong><em>Verify Data Persistence with New Container:</em></strong></p><p>The Jenkins data (such as the admin user) is stored inside the /var/jenkins_home, and since the path was mapped to the Docker Volume, the data should persist even when a container is removed.</p><p>I started a new container and attached the same volume. Now, Jenkins should pick this volume and I should see the Jenkins login page where I can log in with my new admin username and password that I created earlier.</p><p>If the data does not persist, I will be prompted with the initial set up wizard again.</p><p>First, I stopped and removed the original container.</p><p>Then verified the volume was still in one piece.</p><pre>#Stop the original Jenkins container<br>sudo docker stop jenkins-level-up<br><br>#Remove the original Jenkins container<br>sudo docker rm jenkins-level-up<br><br># Verified volume is intact <br>sudo docker volume ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/1*fGrTU29mHdMuhzFW9VOeeQ.png" /></figure><p>Now, I created a new container and attached the same volume.</p><pre>sudo docker run -d \<br>  -p 8080:8080 -p 50000:50000 \<br>  -v level-up-solutions-new:/var/jenkins_home \<br>  --name jenkins-level-up-new \<br>  jenkins/jenkins:lts-jdk17</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/707/1*t5L6jDVP1jSJ5u7nbIxRNw.png" /></figure><p>Now, I went back to the Jenkins in the browser to ensure I got login with the credentials of the user I created.</p><p>Success!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nvVo2D7S1NTtfYepWpGFig.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*MpF-JAAHzASjauaSYdGlkg.png" /></figure><h3><strong>Challenges &amp; How I Overcame Them:</strong></h3><p>Throughout this project, I encountered several challenges that tested my problem-solving skills.</p><p>At one point, I attempted to launch a new Jenkins container but got an error that port 8080 was already in use. I traced this back to another container still running on that port, so I stopped and removed the conflicting container before proceeding.</p><p>Similarly, when reusing the container name jenkins-level-up, I received an error saying the name was already taken. To resolve this, I simply renamed the new container, jenkins-level-up-new.</p><p>Finally, to confirm that Jenkins data persisted, I inspected the Docker volume and logged into the newly launched container with my previously created admin credentials — which worked exactly as expected.</p><p>Each of these obstacles taught me something valuable about container management, Docker volumes, and troubleshooting in real-world scenarios.</p><h3><strong>Conclusion Insights:</strong></h3><p>This project was a meaningful opportunity to both solidify my technical skills and appreciate the practical challenges of deploying reliable, stateful applications in containerized environments. By successfully deploying a Jenkins CI/CD server with Docker and ensuring data persistence through volumes, I validated my ability to deliver resilient infrastructure — a cornerstone of modern DevOps practices. Along the way, I navigated and overcame common but critical obstacles, including permission issues, Docker authentication hurdles, and port and container conflicts, each of which, enhanced my understanding of how to troubleshoot and maintain production-ready systems. Most importantly, this project not only reinforced my grasp of Docker fundamentals and Jenkins administration but also demonstrated the tangible impact of careful planning, verification, and persistence in achieving reliable, scalable results that can translate directly to real-world Cloud and DevOps environments.</p><p>Thank you again for your time!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FQAsBwSjx9zVKoGp9nr%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fthank-you-keanu-reeves-QAsBwSjx9zVKoGp9nr&amp;image=https%3A%2F%2Fmedia3.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExb2lldGYxcmo5YzJmamthZXR1YTliZ3R1a3JnYzdlZzF2eG9qY3JmNiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FQAsBwSjx9zVKoGp9nr%2Fgiphy_s.gif&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="313" frameborder="0" scrolling="no"><a href="https://medium.com/media/25ac8178604b5ac32df5fff6b32da132/href">https://medium.com/media/25ac8178604b5ac32df5fff6b32da132/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=168926cddfdb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Containerized Web Server Deployment with Docker]]></title>
            <link>https://medium.com/@niyi.ayinde1/containerized-web-server-deployment-with-docker-46f08535f9be?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/46f08535f9be</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Fri, 18 Jul 2025 03:19:38 GMT</pubDate>
            <atom:updated>2025-07-18T14:05:04.668Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*iFI-2uPAGt8WImuh.png" /></figure><p>DockerHub Link: <a href="https://hub.docker.com/repository/docker/nayinde125/lu-solutions/general">https://hub.docker.com/repository/docker/nayinde125/lu-solutions/general</a></p><h3><strong>Introduction:</strong></h3><p>Hello once again. I was recently requested to assist with a client, Level Up Solutions. They are a dynamic technology company that offers innovative web solutions. Level Up Solutions wants to adopt containerization using Docker to streamline their web applications deployments.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Level Up Solutions wants to use Docker containers because it provides a flexible and efficient way to package, distribute, and run their applications. Docker containers will also provide Level Up Solutions with consistency, portability, maximized resources, reproducibility, version control, scalability, and collaboration with the development and the operations teams.</p><h3><strong>Solution Strategy:</strong></h3><p>I informed Level Up solutions that I will create an Ubuntu-based Docker container running Apache2 and this will serve as a custom web page for Level Up Solutions. After this, I will push the image to Docker Hub. Once this I complete this, I will also pull the image back and run this on a different host port while still mapping to port 80 within the container.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Docker</strong>: an open platform for developing, shipping, and running applications. Docker enables the user to separate applications from the infrastructure, so the user can deliver software quickly.</p><p><strong>Docker container:</strong> a standard unit of software that packages up code and all its dependencies so the application runs quickly from one computing environment to another.</p><p><strong>Amazon EC2</strong>: on-demand, scalable computing capacity in AWS Cloud.</p><p><strong>Apache web server</strong>: open-source and free web server, commonly utilized for web hosting and development.</p><h3><strong>Implementation Steps:</strong></h3><p><strong><em>Set up environment:</em></strong></p><p>I first added an AWS EC2 instance (Ubuntu). After this, I logged into the terminal and logged into the instance.</p><p>After this, I then verified Docker was installed with this:</p><pre>docker --version</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*ZboKJLrgIMj8PFadzJ2O2Q.png" /></figure><p><strong><em>Run Ubuntu container:</em></strong></p><p>After this, I pulled the Ubuntu image and ran it in detached mode. I also mapped the host port 80 to the container port 80.</p><p><em>Run as detached note:</em></p><p><em>-d → detached</em></p><p><em>-i &amp; -t → interactive terminal</em></p><p><em>-p 80:80 → host port 80 → container port 80</em></p><p><em>— name → gives it an easy-to-reference name</em></p><pre>sudo docker run -dit -p 80:80 --name lusolutions ubuntu:latest</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*oXk2zT202pPDCVXbIH6fwg.png" /></figure><p><strong><em>Update &amp; Install Apache2:</em></strong></p><p>First, I have to attach the container and update the package list:</p><pre># Attach the container<br>sudo docker exec -it lusolutions bash<br><br># Update package list<br>apt-get update &amp;&amp; apt-get updated -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/866/1*fUdPlgvndhD5aUoB8Ucidw.png" /></figure><p>After this, I installed the Apache2.</p><pre>apt-get install appach2 -y</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UqYgDSOqIBUC-KAevgJztw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3q1OB2fj2IBQLonOudzbUg.png" /></figure><p>Next, I start the Apache. I used the “ServerName localhost” to help overcome any warnings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*ymzoQ767w1-T3v6EQFAi_w.png" /></figure><p><strong><em>Create a custom webpage:</em></strong></p><p>Once the Apache web server was added, I can now add the customer web page.</p><p>I added a custom web page for testing.</p><pre>cat &lt;&lt;EOF &gt; /var/www/html/index.html<br>&lt;!DOCTYPE html&gt;<br>&lt;html&gt;<br>&lt;head&gt;<br>  &lt;title&gt;LU Solutions&lt;/title&gt;<br>&lt;/head&gt;<br>&lt;body&gt;<br>  &lt;h1&gt;Welcome to LU Solutions!&lt;/h1&gt;<br>  &lt;p&gt;We deliver innovative web solutions for your business needs.&lt;/p&gt;<br>&lt;/body&gt;<br>&lt;/html&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*Y_zsgCrJitGPMpdrbkTvTA.png" /></figure><p>Then I verified that this will show by visiting a web browser with my EC2 Public IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nl0QwHx3vc2CIFYG3oTTVA.png" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FvvbGMpbhZMcHSsD50w%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Freaction-vvbGMpbhZMcHSsD50w&amp;image=https%3A%2F%2Fmedia0.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExcmpzZGpzMjR1czZwZzVvYzVzMGEyZWxudWQzYnBxaTMzYnh3c3diMCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FvvbGMpbhZMcHSsD50w%2Fgiphy_s.gif&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="326" frameborder="0" scrolling="no"><a href="https://medium.com/media/17160f2457982208470672882f286042/href">https://medium.com/media/17160f2457982208470672882f286042/href</a></iframe><h3>But wait, there is more!</h3><p>After building, pushing, and running the Level Up Solutions web server container on port 80, I moved forward to validate port remapping. I will ensure the Docker image was pulled from Docker Hub, and a new container will be ran with host port 8080 mapped to container port 80.</p><p>Also, verify that I can reach the webserver with the new port.</p><p><strong><em>Push to Docker Hub:</em></strong></p><p>First, I committed the container as a new image and then I verified the image was available.</p><pre># Commit the new image<br>sudo docker commit lusolutions nayinde125/lu-solutions:v1<br><br># Verify the image is there<br>sudo docker images</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*euYGNAGlFTrdYwKqREuq9g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*IBKmL4ja5zfrcnPQWf7NbA.png" /></figure><p><strong><em>Login to Docker Hub:</em></strong></p><p>I logged back into the DockerHub and created a repository for my image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/906/1*2reSu2NT_JpTuplfxj4Grg.png" /><figcaption>Note: if you used another service (i.e. Google) as your login, you can create personal access token to log in when in the terminal.</figcaption></figure><p><strong><em>Push Image to Docker Hub:</em></strong></p><p>After a number of attempts to log into my account, I finally setup everything to push the tagged image to Docker Hub:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/862/1*D2hV20MFDKhG4jklW00buA.png" /><figcaption>I did log into root with sudo -i to avoid session mismatch</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*BnZw4Ge0rJU-fWG_KaIR_w.png" /></figure><p><strong><em>Pull the Image &amp; updated host port:</em></strong></p><p>Now that the image is there, I proceeded to pull the image from the Docker Hub. The new mapping host port I choose was port 8080. Reason being that it’s a widely known conventional way to use development and test environments for applications. Also, it is easier to run multiple containers applications simultaneously, and avoid any conflicts with other services that are using port 80.</p><p>Below, I used this command to start a new container to map it to port 8080 from port 80.</p><pre>docker run -dit -p 8080:80 --name lusolutions-new nayinde125/lu-solutions:v1</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*2jIG-bUSSi3VABPJe3cKAQ.png" /></figure><p>Then, I attached the container and started Apache.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*laLLcwfqr1DkJOGt5SIV2A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/1*ymzoQ767w1-T3v6EQFAi_w.png" /></figure><p>Then I verified that the container is now mapped.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ykSIkPA28BbU70GNaoMx-Q.png" /></figure><p>Lastly, I just needed to the test the browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*hyF4ysCe2DbWVmo7EgfAag.png" /></figure><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fl0MYrguDYf6uvscak%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Ftrailblazers-blazers-portland-trail-l0MYrguDYf6uvscak&amp;image=https%3A%2F%2Fmedia1.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExZ2E2aG5iYzBrajd1Nnp5YmIxcmRsd2duamI1Njk0cW44M2k4ejl5YiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2Fl0MYrguDYf6uvscak%2Fgiphy_s.gif&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="355" frameborder="0" scrolling="no"><a href="https://medium.com/media/035dfce2cc85626e9b224bc665038111/href">https://medium.com/media/035dfce2cc85626e9b224bc665038111/href</a></iframe><h3>Challenges &amp; Solutions:</h3><p>One of the initial challenges was dealing with permission errors when trying to interact with the Docker daemon. Since Docker requires elevated privileges, I ensured all commands were executed either as root or prefixed with sudo, which resolved the access issues.</p><p>Another challenge arose when pushing the Docker image to Docker Hub. Because I logged into Docker Hub using Google authentication on the website, I initially struggled to log in via the CLI. After researching, I generated a personal access token on Docker Hub and used it in place of my password — an important real-world learning point about credential management.</p><p>Additionally, while running the container on port 8080, I mistakenly tried accessing it via https:// instead of <a href="http://,/">http://,</a> which resulted in connection errors. Recognizing that the Apache server was only configured for HTTP allowed me to correct the URL and confirm that the application was serving properly.</p><p>These challenges gave me the chance to deepen my understanding of Docker’s security model, authentication flows, container lifecycle, and networking — all of which are highly relevant for modern DevOps workflows. Overall, this project strengthened both my technical skills and my ability to troubleshoot and adapt when unexpected issues arose.</p><h3>Conclusion Insights:</h3><p>Through this project, I successfully designed, deployed, and validated a containerized Apache web server running a custom webpage for Level Up Solutions. Outside of just building and running the application, I also demonstrated how to publish the Docker image to Docker Hub, reuse it by pulling it on the same host, and run it with an alternate port mapping — a critical skill in scenarios where default ports may already be occupied.</p><p>Completing this project helped me understand Docker commands, key DevOps concepts like fixed infrastructure, container reuse, port mapping, and troubleshooting service availability. It also highlighted the importance of understanding how containers, host networking, and security group configurations interact when deploying to cloud instances.</p><p>I appreciate you for taking the time to read, I will see you again soon!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46f08535f9be" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building a CI/CD AI-Powered Email Pipeline with AWS Bedrock, S3, and GitHub Actions]]></title>
            <link>https://medium.com/@niyi.ayinde1/building-a-ci-cd-ai-powered-email-pipeline-with-aws-bedrock-s3-and-github-actions-306a35b71999?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/306a35b71999</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Thu, 10 Jul 2025 23:41:36 GMT</pubDate>
            <atom:updated>2025-07-11T14:36:44.962Z</atom:updated>
            <content:encoded><![CDATA[<h3>For educational purposes only: Building a CI/CD AI-Powered Pipeline with AWS Bedrock, S3, and GitHub Actions.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZHi-XcScnVdMSmBr4JJY-Q.jpeg" /></figure><p>GitHub Link: <a href="https://github.com/NiyiNeo/Pixel.Bedrock">https://github.com/NiyiNeo/Pixel.Bedrock</a></p><h3><strong>Introduction:</strong></h3><p>Hello once again. I am back assisting with my client, Pixel Learning company. The content team wants to create materials for students and instructors, drafting prompt templates for onboarding messages, study tips, and course summaries using foundation models. They aim to streamline this process by managing prompts and generation workflows.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Pixel Learning company stated that they want to implement a solution that integrates Artificial Intelligence (AI), and they aim to streamline this process by managing prompts and automatically publish generated content to a static website hosted in S3.</p><h3><strong>Solution Strategy:</strong></h3><p>My solution is to build and deploy an automated pipeline that creates personalized content with Amazon Bedrock, stores results in <strong>S3 buckets</strong>, and uses <strong>GitHub Actions workflows</strong> for deployments. I will walk through the setup, template design, GitHub secrets management, CI/CD process, and final deployment.</p><p>I will also include lessons learned from this project.</p><p><strong>Prerequisites/Technologies needed:</strong></p><ul><li><strong>AWS S3</strong> — Static website hosting for output.</li><li><strong>AWS IAM</strong> — Secure permissions for CI/CD and S3 access.</li><li><strong>Amazon Bedrock (Claude 3 Sonnet)</strong> — To generate natural language email content.</li><li><strong>GitHub Actions</strong> — To automate pipeline on Pull Request and Merge events.</li><li><strong>Python (boto3 &amp; Jinja2)</strong> — To orchestrate rendering, API calls, and uploads.</li><li><strong>GitHub Repository Secrets</strong> — To securely store credentials.</li></ul><h3>Implementation Steps:</h3><p><strong><em>Setting Up AWS Resources</em></strong></p><p><em>Amazon S3 Buckets</em></p><p>First, I setup the Amazon S3 Buckets. The first bucket will be the Beta bucket (pixel.br.beta) for testing and the second bucket will be for production (pixel.br.prod).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*o95bj8Fmh3h8_xBKqXkZxQ.png" /></figure><p>Next, I ensured that the static website hosting was enabled for both buckets. After this, I configured the bucket policies to allow public read access for objects, which is needed for website hosting.</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Sid&quot;: &quot;PublicReadGetObject&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;,<br>            &quot;Principal&quot;: &quot;*&quot;,<br>            &quot;Action&quot;: &quot;s3:GetObject&quot;,<br>            &quot;Resource&quot;: &quot;arn:aws:s3:::pixel.br.beta/*&quot;<br>        }<br>    ]<br>}</pre><p><em>IAM Roles and Policies:</em></p><p>Next, I also updated my IAM user, PL_Admin, to have access to the AWS services: S3 bucket permissions as well as Bedrock access.</p><p><em>Amazon Bedrock:</em></p><p>For this project, I am using Amazon Bedrock’s Claud 3 sonnet model. Bedrock provides a powerful foundation model with no infrastructure management. Bedrock also provides fast prompt execution, multi-model access, security and scalability, and no deployment overhead.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/653/1*TfUDaZt9ODDqzPgtto2LGg.png" /></figure><p><strong><em>Design Templates and Configurations:</em></strong></p><p>I created a prompt template in a form of an email from instructor to student, welcome_email.txt, and saved under my prompt_templates/ folder. The template included placeholders for the student_name, goal1, goal2, and goal3.</p><pre>Hello {{ student_name }}<br>Greetings from the Pixel Learning Company.<br>In your cloud journey, you have achieved:<br>1. {{ goal1 }}<br>2. {{ goal2 }}<br>3. {{ goal3 }}<br><br>We’re excited about your progress with Pixel Learning Company!</pre><p><strong><em>Variable configurations:</em></strong></p><p>Following that, I creating a corresponding JSON file that contains the description for the template. This was saved in my prompts folder named welcome_prompt.json</p><pre>{<br>  &quot;output_file&quot;: &quot;welcome_{{ student_name|lower }}.html&quot;,<br>  &quot;variables&quot;: {<br>    &quot;student_name&quot;: &quot;Jordan&quot;,<br>    &quot;goal1&quot;: &quot;Learned how to launch and configure an EC2 instance&quot;,<br>    &quot;goal2&quot;: &quot;Uploaded and hosted a static website on S3&quot;,<br>    &quot;goal3&quot;: &quot;Set up IAM roles and policies for secure access&quot;<br>  }<br>}</pre><p><strong><em>Breakdown of Python Script:</em></strong></p><p>I move then into creating a python script. Below is brief overview of the script is meant to:</p><ul><li><em>Load the JSON configuration.</em></li><li><em>Render the Jinja2 email template with variables.</em></li><li><em>Construct a well-crafted prompt for Claude (Bedrock).</em></li><li><em>Call Bedrock and receive the human-like response.</em></li><li><em>Save outputs (</em><em>.html and </em><em>.md) locally</em></li><li><em>Upload outputs to the correct S3 bucket (</em><em>beta or </em><em>prod).</em></li></ul><p>First, load environment variables. It uses GitHub secrets as environment variables and prevents any hardcoding into the script.</p><pre>S3_BUCKET_BETA = os.getenv(&#39;S3_BUCKET_BETA&#39;)<br>S3_BUCKET_PROD = os.getenv(&#39;S3_BUCKET_PROD&#39;)<br>DEPLOY_ENV = os.getenv(&#39;DEPLOY_ENV&#39;, &#39;beta&#39;)<br>FILENAME = os.getenv(&#39;FILENAME&#39;)<br>AWS_REGION = os.getenv(&#39;AWS_REGION&#39;, &#39;us-east-1&#39;)</pre><p>Next, chose the target bucket based on deployment environment, (beta or prod).</p><pre>S3_BUCKET = S3_BUCKET_BETA if DEPLOY_ENV == &#39;beta&#39; else S3_BUCKET_PROD</pre><p>Now, I load the JSON configuration, read variables, and out file template from the JSON config I mentioned earlier.</p><pre>json_path = prompts_dir / f&#39;{FILENAME}.json&#39;<br>with open(json_path, &#39;r&#39;, encoding=&#39;utf-8&#39;) as f:<br>    config = json.load(f)</pre><p>Afterwards, I render or deliver the Jinja2 template and this fills in the .txt template with the variables from the JSON to create the email.</p><pre>template = Template(template_content)<br>rendered_prompt = template.render(**config[&#39;variables&#39;])</pre><p>Then I built the Bedrock Prompt and I carefully crafted the Human: prompt to inform Claude to rewrite the draft naturally.</p><pre>return {<br>        &quot;anthropic_version&quot;: &quot;bedrock-2023-05-31&quot;,<br>        &quot;max_tokens&quot;: max_tokens,<br>        &quot;messages&quot;: [<br>            {<br>                &quot;role&quot;: &quot;user&quot;,<br>                &quot;content&quot;: (<br>                    f&quot;&quot;&quot;Human: You are a professional instructor at Pixel Learning Company.</pre><p>Now, I called Bedrock, send the prompt to Bedrock, and decipher the JSON response.</p><pre>    response = bedrock_client.invoke_model(<br>        modelId=&quot;anthropic.claude-3-sonnet-20240229-v1:0&quot;,<br>        contentType=&quot;application/json&quot;,<br>        accept=&quot;application/json&quot;,<br>        body=json.dumps(body)<br>    )</pre><p>Following this, I save the output locally and the output saves as both .html and .md files for debugging and hosting.</p><pre>html_path.write_text(html_content.strip(), encoding=&#39;utf-8&#39;)<br>md_path.write_text(final_output, encoding=&#39;utf-8&#39;)</pre><p>Finally, I upload both files to their designated S3 bucket under the prefix and sets content types.</p><pre>    s3_client.upload_file(<br>        str(html_path),<br>        S3_BUCKET,<br>        f&quot;{DEPLOY_ENV}/outputs/{html_filename}&quot;,<br>        ExtraArgs={&#39;ContentType&#39;: &#39;text/html&#39;}<br>    )<br><br>    s3_client.upload_file(<br>        str(md_path),<br>        S3_BUCKET,<br>        f&quot;{DEPLOY_ENV}/outputs/{md_filename}&quot;,<br>        ExtraArgs={&#39;ContentType&#39;: &#39;text/markdown&#39;}<br>    )</pre><p>Full the full script, please see here: <a href="https://github.com/NiyiNeo/Pixel.Bedrock/blob/main/process_prompt.py">https://github.com/NiyiNeo/Pixel.Bedrock/blob/main/process_prompt.py</a></p><p><strong><em>Implementing Git Hub Secrets CI/CD Pipelines:</em></strong></p><p>Inside of the GitHub Repository, I added the GitHub Secrets for the workflow YAML files. Again, this good coding practice to prevent hardcoding in the python script.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*FfaTuxXZ9SaMg0sX74U_kQ.png" /></figure><p><strong><em>Implementing CI/CD Pipelines:</em></strong></p><p>For the GitHub Actions Workflows, there are two workflows to be tested.</p><p>Each workflow installed Python dependencies (boto3 and Jinja2), run the full Python script and upload the crafted Bedrock generated HTML to their corresponding bucket.</p><p>The first is the on pull request, to deploy the beta and review within the S3 bucket for review.</p><pre>name: Call Amazon Bedrock (Beta)<br><br>on:<br>  pull_request:<br><br>jobs:<br>  call_bedrock:<br>    runs-on: ubuntu-latest<br><br>    env:<br>      S3_BUCKET_BETA: ${{ secrets.S3_BUCKET_BETA }}<br>      S3_BUCKET_PROD: ${{ secrets.S3_BUCKET_PROD }}<br>      FILENAME: ${{ secrets.FILENAME }}<br>      AWS_REGION: ${{ secrets.AWS_REGION }}<br>      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>      DEPLOY_ENV: beta<br><br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v3<br>        <br>      - name: Install dependencies<br>        run: |<br>          pip install --upgrade boto3 jinja2<br><br>      - name: Set up AWS CLI<br>        uses: aws-actions/configure-aws-credentials@v3<br>        with:<br>          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>          aws-region: ${{ secrets.AWS_REGION }}<br><br>      - name: Call Amazon Bedrock (beta)<br>        run: |<br>          python process_prompt.py</pre><p>Once uploaded, I got sucess!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ta3u3aiFcvZeCwy8v9xMhw.png" /></figure><p>Once this was completed, I checked to ensure that the S3 beta bucket output has been uploaded and I can access the S3 Static website.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/923/1*2n5ub6ljG8f5TaOINFYYLA.png" /></figure><p>After many attempts, I got the S3 Static website to generate an output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bcu4Zb544GeElsKws1gkYw.png" /></figure><p>The next workflow is the on merge to main branch and deploys the results to my S3 Bucket for production.</p><pre>name: Deploy to Production (Bedrock)<br><br>on:<br>  push:<br>    branches:<br>      - main<br><br>jobs:<br>  call_bedrock:<br>    runs-on: ubuntu-latest<br><br>    env:<br>      S3_BUCKET_PROD: ${{ secrets.S3_BUCKET_PROD }}<br>      FILENAME: ${{ secrets.FILENAME }}<br>      AWS_REGION: ${{ secrets.AWS_REGION }}<br>      DEPLOY_ENV: prod<br><br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v3<br><br>      - name: Set up AWS CLI<br>        uses: aws-actions/configure-aws-credentials@v3<br>        with:<br>          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>          aws-region: ${{ secrets.AWS_REGION }}<br><br>      - name: Set up Python<br>        uses: actions/setup-python@v4<br>        with:<br>          python-version: &#39;3.11&#39;<br><br>      - name: Install dependencies<br>        run: |<br>          python -m pip install --upgrade pip<br>          pip install -r requirements.txt  # or list them explicitly if no file<br><br>      - name: Call Bedrock (Prod)<br>        run: |<br>          python process_prompt.py</pre><p>After this, I check to see if the S3 Prod Bucket had the output inside there and tested the S3 Static Website for the prompt, success!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*yG1nx0H-5kiiLXAFb_x_Ng.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xLnq1enZk-CzoTStpmHk0Q.png" /></figure><p><strong><em>Adding the README.md:</em></strong></p><p>Finally, I added a file README.md to the repository. The README is designed to assist anyone to help understand the setup and how to run the project.</p><p>This included:</p><ul><li>Project overview: with a detailed explanation of workflow</li><li>AWS setup: How to add the S3 buckets, create the IAM roles and enable Amazon Bedrock</li><li>GitHub Secrets: Setting up this in the GitHub repository for the python environment</li><li>Workflow execution: How to trigger the CI/CD pipeline by creating a pull request or merge YAML file to the main branch.</li><li>Reviewing the Output: Review the results from .html and .md files for the S3 static website endpoint.</li></ul><h3>Conclusion Insights:</h3><p>This one nearly broke me! Seriously! But I can say this, I was able to design, build and deliver a cloud-native automation pipeline with AWS services, Git Action CI/CD and generation AI.</p><p>I accomplished the following:</p><p>· Designed and implemented a <strong>GitHub‑based CI/CD pipeline</strong> that automates the generation of personalized content using Amazon Bedrock and deploys the results to S3 static websites.</p><p>· Set up two <strong>S3 buckets </strong>(Beta &amp; Prod) with static website hosting and proper bucket policies.</p><p>· Created <strong>IAM role</strong>, configured <strong>GitHub Secrets</strong> securely, and orchestrated workflows with best practices (PR triggers for Beta and Merge for Prod).</p><p>· Authored reusable <strong>Jinja2 templates</strong> and structured .json configuration files for dynamic and maintainable prompts.</p><p>· Wrote a <strong>Python script</strong> that loads configurations, renders the prompt, calls <strong>Amazon Bedrock</strong> (Claude 3 Sonnet), formats the output (.html &amp; .md), and uploads it with proper metadata to S3.</p><p>· Provided a comprehensive <strong>README.md</strong> and workflow instruction..</p><h3><strong>Challenge(s) &amp; How I Overcame Them:</strong></h3><p>One of the most significant challenges I faced was crafting a prompt in the Python script that consistently generated <em>natural, human‑like</em> responses from Claude 3 Sonnet via Amazon Bedrock.<br>At first, Claude would frequently respond as an AI assistant, stating things like: “As an AI language model, I don’t …” even though the intention was to produce a human instructor’s email.</p><p><strong>To address this:</strong></p><p>· I experimented with the <em>Human:</em> prefix in the Bedrock request payload, refining the context to clearly instruct Claude to behave as a human instructor and not refer to itself as AI.</p><p>· I rewrote the prompt instructions to explicitly say: <em>“Do NOT mention being an AI, a model, or an assistant. You are a professional instructor at Pixel Learning Company writing to a student …”</em>.</p><p>· I tested multiple combinations of template and .json configurations to ensure that the variables were passed correctly and that the rendered prompt matched expectations.</p><p>· I also wrapped the outputs in proper HTML and ensured the correct Content‑Type so the static website displayed the email properly instead of downloading the file.</p><p>This iterative debugging and prompt engineering taught me valuable lessons about working with generative AI in production pipelines: <em>be explicit in your instructions, test edge cases, and validate output against user expectations</em>.</p><p>Again, I want to Thank you for reading, I will see you on the next one!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=306a35b71999" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[For English press one, para español, oprima dos.]]></title>
            <link>https://medium.com/@niyi.ayinde1/for-english-press-one-para-espa%C3%B1ol-oprima-dos-88392f7de272?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/88392f7de272</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Sun, 29 Jun 2025 14:26:49 GMT</pubDate>
            <atom:updated>2025-06-30T14:06:51.934Z</atom:updated>
            <content:encoded><![CDATA[<h3>For English press one, <strong>para español, oprima dos</strong>. A review in multilingual audio processing workflow using AWS Services and GitHub Actions.</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w3TeeDADSvePJ2Ye4knxFg.png" /></figure><p><a href="https://github.com/NiyiNeo/Pixel.Translate.1">GitHub - NiyiNeo/Pixel.Translate.1</a></p><h3><strong>Introduction:</strong></h3><p>Greetings everyone. I am back with one of my favorite clients, you guess it, Pixel Learning company. They reached out to me to help them solve another business issue they had. They have a repository of their educational content they use for their course materials, and they want to automate the instructor voice messages into multiple languages.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Pixel Learning company stated that they want to implement a solution that will take their .mp3 audio files with educational content, transcribe the audio file, translate the text into another language and then regenerate speech into a target language for international listeners. They also requested the process to be efficient, consistent, and fully automated with their GitHub-based content pipeline.</p><h3><strong>Solution Strategy:</strong></h3><p>I informed Pixel Learning company that I can leverage the AWS services such as Amazon Transcribe, Amazon Translate, Amazon S3, along with GitHub Action to automate a multilingual audio process. The workflow will process a MP3 file, transcribe this file, audio to text, then translate this text into another language, from English to Spanish, and then translate the convert text into generate speech content. After this, all of the outputs from this process will be stored within Amazon S3. Using the AWS AI services offers quick and accessible language transformation within custom model training. This also provides accurate extract text from voice due to the Amazon’s Transcribe’s speech recognition engine, built-in translation the supports dozens of languages, and all services are fully hosted, secure, and scalable without infrastructure provisioning.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Amazon Transcribe:</strong> an automatic speech recognition service that use machine learning models to convert audio to text.</p><p><strong>Amazon Translate: </strong>a high-quality neural machine translation service that uses advanced machine learning technologies to provide fast language translation of text-based content and enables the development of applications that provide multilingual user experiences.</p><p><strong>Amazon Polly</strong>: a cloud service that converts text into speech and supports multiple languages.</p><p><strong>Python</strong> — an interpreted, object-oriented, programming language that is known for is readability and versatility and is widely used for web development, data analysis, machine learning, etc.</p><p><strong>Amazon S3</strong>: is an object storage service that offers scalability, data availability, security, and performance.</p><p><strong>GitHub Actions:</strong> A platform that allows users to automate software development workflows, including continuous integration and continuous delivery (CI/CD) that can build, test, and deploy code.</p><p><strong>AWS Identity and Access Management (IAM) User</strong>: securely manage identities and access to AWS services and resources.</p><h3><strong>Implementation Steps:</strong></h3><p>First, I setup the GitHub Repository for Pixel Learning Co. After this, I created the S3 buckets for the results of converting the speech to text, the translation of the text from English to Spanish and then generating speech speech from that translation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*bsxbqC8G6QTSR177IKZ95Q.png" /></figure><p>I also updated the IAM user, PL_Admin to have access to the AWS AI services, so this user has the necessary permissions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JvxZIPA0qEMJzlj1KWRd4Q.png" /></figure><p><strong><em>Python Script Breakdown</em></strong><em>:</em></p><p>First section focuses on the AWS session using the GitHub. This part of the script helps setup a session to call AWS Services and each code is setup in GitHub Secrets to authenticate AWS access to prevent hardcoding.</p><pre>session = boto3.Session(<br>    aws_access_key_id=os.environ[&#39;AWS_ACCESS_KEY_ID&#39;],<br>    aws_secret_access_key=os.environ[&#39;AWS_SECRET_ACCESS_KEY&#39;],<br>    region_name=os.environ[&#39;AWS_REGION&#39;]<br>)</pre><p>The next section is about AWS clients and the environment variables. The clients allow communications with AWS services and these are also kept in GitHub Secrets.</p><pre>s3 = session.client(&#39;s3&#39;)<br>transcribe = session.client(&#39;transcribe&#39;)<br>translate = session.client(&#39;translate&#39;)<br>polly = session.client(&#39;polly&#39;)<br><br>prod_bucket = os.environ[&#39;S3_BUCKET_PROD&#39;]<br>beta_bucket = os.environ[&#39;S3_BUCKET_BETA&#39;]<br>filename = os.environ[&#39;FILENAME&#39;]<br>source_lang = os.environ[&#39;SOURCE_LANG&#39;]<br>translate_lang = os.environ[&#39;TRANSLATE_LANG&#39;]<br>polly_voice = os.environ[&#39;POLLY_VOICE&#39;]</pre><p>This section are the file paths and job names. This helps generate unique names and structured paths for processing artifacts and the output is stored in S3 bucket with an key strings as prod/audio/Pixel_Learning.mp3.</p><pre>timestamp = datetime.now().strftime(&#39;%Y%m%d%H%M%S&#39;)<br>job_name = f&quot;PixelLearn-{timestamp}&quot;<br>audio_file = f&quot;prod/audio/{filename}.mp3&quot;<br>audio_uri = f&quot;s3://{prod_bucket}/{audio_file}&quot;<br>transcript_key = f&quot;beta/transcripts/{filename}.txt&quot;<br>translated_key = f&quot;beta/translations/{filename}_{translate_lang}.txt&quot;<br>final_audio_key = f&quot;prod/audio_outputs/{filename}_{translate_lang}.mp3&quot;</pre><p>This section is very self-explanatory, it is to upload the .mp3 file to the S3 bucket. Next, I used Amazon Transcribe and this initiates a transcription process taking the MP3 that stored in the S3 bucket and produced a .json file that has the transcript.</p><pre>s3.upload_file(f&quot;{filename}.mp3&quot;, prod_bucket, audio_file)<br><br>transcribe.start_transcription_job(<br>    TranscriptionJobName=job_name,<br>    Media={&#39;MediaFileUri&#39;: audio_uri},<br>    MediaFormat=&#39;mp3&#39;,<br>    LanguageCode=source_lang,<br>    OutputBucketName=beta_bucket<br>)</pre><p>This part of the script is wait for the transcription job to finish. This is to ensure that the script does not proceed until the job is finished, and it will stop if the job fails.</p><pre>while True:<br>    status = transcribe.get_transcription_job(TranscriptionJobName=job_name)<br>    job_status = status[&#39;TranscriptionJob&#39;][&#39;TranscriptionJobStatus&#39;]<br>    if job_status in [&#39;COMPLETED&#39;, &#39;FAILED&#39;]:<br>        if job_status == &#39;FAILED&#39;:<br>            raise Exception(&quot;Transcription failed.&quot;)<br>        break<br>    time.sleep(5)</pre><p>Now, once that completed, this section will download the Transcript from the JSON file, extract the text, and upload transcript text file to S3.</p><pre>transcript_file_key = f&quot;{job_name}.json&quot;<br>local_transcript_file = &quot;/tmp/transcript.json&quot;<br>s3.download_file(beta_bucket, transcript_file_key, local_transcript_file)<br><br>with open(local_transcript_file, &#39;r&#39;) as f:<br>    transcript_json = json.load(f)<br><br>transcript_text = transcript_json[&#39;results&#39;][&#39;transcripts&#39;][0][&#39;transcript&#39;]<br><br>s3.put_object(Bucket=beta_bucket, Key=transcript_key, Body=transcript_text.encode())</pre><p>Ok now, this section actually translate the transcript that has been upload into Spanish, and uploads this translation into the S3 bucket again.</p><pre>translation = translate.translate_text(<br>    Text=transcript_text,<br>    SourceLanguageCode=source_lang.split(&quot;-&quot;)[0],<br>    TargetLanguageCode=translate_lang<br>)<br>translated_text = translation[&#39;TranslatedText&#39;]<br><br>s3.put_object(Bucket=beta_bucket, Key=translated_key, Body=translated_text.encode())</pre><p>Lastly, the script uses the Amazon Polly generate the new mp3 file of generated speech of the translated file and stores this in S3 Prod Bucket.</p><pre>polly_response = polly.synthesize_speech(<br>    OutputFormat=&#39;mp3&#39;,<br>    Text=translated_text,<br>    VoiceId=polly_voice,<br>    LanguageCode=f&quot;{translate_lang}-ES&quot;<br>)<br><br>with open(f&quot;{filename}_{translate_lang}.mp3&quot;, &#39;wb&#39;) as f:<br>    f.write(polly_response[&#39;AudioStream&#39;].read())<br><br>s3.upload_file(f&quot;{filename}_{translate_lang}.mp3&quot;, prod_bucket, final_audio_key)</pre><p>I tested this locally, and I achieved success.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*stRFcN97JMxDkIdACpHlWQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1015/1*2l52TSRuMWHDrOlpmWOaZw.png" /></figure><p><strong><em>GitHub Secrets Setup</em></strong>:</p><p>As stated before, avoid hardcoding credentials or sensitive data in Python.</p><p>In the GitHub repository, navigate to Settings, scroll to Secrets and variables, and click Actions. Add each secret name and value here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*cCfkxYITSD1EQxF1eqRaeg.png" /></figure><p><strong><em>GitHub Actions Deployment Workflow:</em></strong></p><p>After this, I upload the mp3 file into the GitHub Repo.</p><p>I started working on the pull request with the on_pull_request.yml file. This will trigger the following:</p><ul><li>upload the prod/audio file</li><li>start the transcription job</li><li>Download the transcript from S3 bucket</li><li>Translate to “target language” of Spanish</li><li>Synthesizes speech via Amazon Polly</li><li>Saves the results to S3 bucket.</li></ul><p>After debugging some errors, I got the pull request green button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XQ5t6nlirerzrqGFYl99Mg.png" /></figure><p>I also verified that the items were save inside the S3 Bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wVE0-6_NeQcnfTJd_wrXOw.png" /></figure><p>Next, I worked on the on_merge.yml file. This will re-run the pipeline to confirm the production deployment and download the final mp3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Za4JOglAQ90_I_FPHBekBA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gu9WD5uCIDrU4FZgxJxZzQ.png" /></figure><p><strong><em>Structure for this GitHub:</em></strong></p><p>Now, the GitHub repo setup has the final folder structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/657/1*6NF7qQUsva-_Zt1DSkYT3g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*IaSJS49JBVXTX8u6w77i4w.png" /></figure><p>I finally downloaded the mp3 file to verify the conversion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CL2Z7oCUWy4SPDlJqfXyJA.png" /></figure><p>See below:</p><p><a href="https://github.com/NiyiNeo/Pixel.Translate.1/blob/main/pixel.language_es.mp3">https://github.com/NiyiNeo/Pixel.Translate.1/blob/main/pixel.language_es.mp3</a></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FSnM6oxhwCISic%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2FSnM6oxhwCISic&amp;image=https%3A%2F%2Fmedia4.giphy.com%2Fmedia%2Fv1.Y2lkPTc5MGI3NjExb3VhN3JwdDR6ZTQwaXRpNXY5aWZoNHJmOHozd3dhbDgwaHF5MjJvbCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw%2FSnM6oxhwCISic%2Fgiphy_s.gif&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="435" frameborder="0" scrolling="no"><a href="https://medium.com/media/25175971192cfb679bf5347f05ed24ac/href">https://medium.com/media/25175971192cfb679bf5347f05ed24ac/href</a></iframe><h3><strong>Conclusion Insights:</strong></h3><p>This was tough, but I got through it. Now, there is a serverless, cloud-native audio processing pipeline. Also, I have used GitHub Actions for multi-stage deployment (Beta + Prod) workflows. I also, did some debugging from failed workflows. Updated the GitHub Secrets to safely pass environment variables to avoid hardcoding. Finally, document this project within GitHub as README.md file.</p><p>Thank you for reading, be blessed!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=88392f7de272" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Do you have any Rekognition of this?]]></title>
            <link>https://medium.com/@niyi.ayinde1/do-you-have-any-rekognition-of-this-48e7ad404840?source=rss-c38de887a73e------2</link>
            <guid isPermaLink="false">https://medium.com/p/48e7ad404840</guid>
            <dc:creator><![CDATA[Niyi Ayinde]]></dc:creator>
            <pubDate>Sun, 15 Jun 2025 17:35:18 GMT</pubDate>
            <atom:updated>2025-06-16T00:21:48.421Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Do you have any Rekognition of this? A </strong>complete CI/CD pipeline that automates image analysis using <strong>Amazon Rekognition</strong>, stores results in <strong>DynamoDB</strong>, and runs entirely from a <strong>GitHub-based workflow.</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/270/1*teqgDz3_EqS2pOQW3b1UnA.jpeg" /></figure><p>GitHub Repository Link: <a href="https://github.com/NiyiNeo/Pixel_Rekogntion">https://github.com/NiyiNeo/Pixel_Rekogntion</a></p><h3><strong>Introduction:</strong></h3><p>Greetings all. I am once again working with Pixel Learning company. They reached out to help solve another business issue . They have a repository of various educational images they use for their course materials and other images for their marketing assets, and they want to automate the tagging and classification of their image content.</p><h3><strong>The Goal/Objective of this Project:</strong></h3><p>Pixel Learning company stated that they want to implement a solution that integrates an Artificial Intelligence (AI) labeling system for their images in a GitHub repository. They would like a solution in which they are not required to manage the servers, streamline review the process and improve content organization of their images.</p><h3><strong>Solution Strategy:</strong></h3><p>I informed Pixel Learning company that I can leverage the AWS AI services, such as Amazon Rekongition, and other AWS services like S3, and DynamoDB to detect and classify images or objects automatically when changes occur in the repository. The results of these labels will be in a DynamoDB Tables. Also, I will use GitHub Actions. GitHub Action will help with automation, support workflows that are triggered by a pull requests, and will run directly into the workflow where the files reside. This will give quality assurance prior to any release and increase accessibility. Lastly, this will minimize the overhead costs, since the AWS services are fast, lightweight, and cost-effective.</p><h3><strong>Prerequisites/Technologies needed:</strong></h3><p><strong>Amazon Rekognition:</strong> an image and video analysis services that makes it easy to add advanced computer vision capabilities to applications.</p><p><strong>Python</strong> — an interpreted, object-oriented, programming language that is known for is readability and versatility and is widely used for web development, data analysis, machine learning, etc.</p><p><strong>Amazon S3</strong>: is an object storage service that offers scalability, data availability, security, and performance.</p><p><strong>Amazon DynamoDB: </strong>a managed NoSQL database service that supports key-value and document data structures and is designed for high performance, scalability, and store and retrieve data without server management.</p><p><strong>GitHub Actions:</strong> A platform that allows users to automate software development workflows, including continuous integration and continuous delivery (CI/CD) that can build, test, and deploy code.</p><p><strong>AWS Identity and Access Management (IAM) User</strong>: securely manage identities and access to AWS services and resources.</p><h3><strong>Implementation Steps:</strong></h3><h4>AWS Resources Setup:</h4><p><strong><em>Create an S3 Bucket:</em></strong></p><p>I first created an S3 bucket for the images. As always, make sure the bucket name is unique.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8bCF6nxRPDcfXrt1EJgFOg.png" /></figure><p>After this, I created a GitHub Repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/893/1*6aTDyvxA0kyTSasxNO1jvA.png" /></figure><p><strong><em>DynamoDB Tables:</em></strong></p><p>I created two tables in DynamoDB. One table is the beta table. This test will post the results of the images. The results will have name, structured labels with confidence ratings, timestamp and the branch.</p><p>The second table will be the Production environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5fCII_zCiCOLi2oLPBkACw.png" /></figure><p><strong><em>IAM User creation:</em></strong></p><p>I created an user, PL_Admin, and gave this user the permissions of Amazon DynamoDB, AmazonS3FullAccess, and Amazon Rekognition Access.</p><p>Downloaded and securely store the following: Access Key ID and Secret Access Key. I will use these later for GitHub Secrets.</p><p><em>Note: For this user, the full access permissions for the AWS Services can be amended based on the scope of their role.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*E_pXDIufHSb-ZiLmPiFlpA.png" /></figure><h4>Create the Python Script:</h4><p>The python script I created will accomplish the following:</p><ul><li>Uploads images from images/ folder to S3</li><li>Calls Rekognition detect_labels</li><li>Logs results in appropriate DynamoDB table</li><li>Reads all credentials from <strong>GitHub Secrets</strong></li></ul><p>This one wasn’t easy, but here is the full script.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lmDmPaRS-SAqPfQCiE6FlQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/809/1*uTSSjkH3iv0LNjI3Xo-Nbw.png" /></figure><h4>Configure GitHub Secrets:</h4><p>It is important to not hardcode credentials or any sensitive names in the python. The script will pull from GitHub Secrets to execute.</p><p>Within the GitHub repository, find Settings. After this, scroll down and find Secrets and variables. Next, click actions. Once there add each secret name and their value. This is where I add the IAM user’s Access Key ID and Secret Access Key and the other items needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*jiVFVht3ats-vR2HDVrLvw.png" /></figure><h4>Creating the GitHub Workflows:</h4><p>The next step in the solution was for me to create GitHub Actions. I created two different YML files. The on_pull_request.yml to trigger a pull request that is a target to the main branch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/1*MvCYqeXiiv5k5tXPqJ3mRg.png" /></figure><p>After I corrected a few errors, I got the green checkmark!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rn-isHFaPzIT4pHp9V7acw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3ECIV2hJGAZSEGiDyIqn_w.png" /></figure><p>The on_merge.yml is the target for the main branch in GitHub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/1*GQ2JSz0XYBTJ2sTWLmWtPQ.png" /></figure><p>It merged succesfully, again after correcting errors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z0unQ7qAS6BXjIRoJ7J9rw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*IwEFtiuccqzgvmPGKSlH4w.png" /></figure><p>Now, both workflows have ran successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XbDl85QEwJflhyBs36jofg.png" /></figure><h4>Verified the results are in S3 and DynamoDB:</h4><p>Lastly, I verify that the images are now within the S3 bucket and DynamoDB table.</p><p><em>S3 Bucket:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FNuu_WhYWsflYmjLUbNefA.png" /></figure><p><strong><em>DynamoDB Table:</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/798/1*6HaJUxzwYPAkngpIFjbclA.png" /></figure><p><strong><em>Add README.md:</em></strong></p><p>After this, I then added a README.md file. This is text file and it informs the next individual on the scope of the project, how to build it, the necessary code and other ways to help recreate:</p><p>Here is the link to my README.md below:</p><p><a href="https://github.com/NiyiNeo/Pixel_Rekogntion/blob/main/README.md">https://github.com/NiyiNeo/Pixel_Rekogntion/blob/main/README.md</a></p><h3>Challenges encountered/How it was addressed:</h3><p><strong><em>Challenge:</em></strong></p><p>One of the biggest challenge when running my python script was the environment variables (S3 BUCKET or DYNAMODB), were not being processed correctly.</p><p><strong><em>Solution:</em></strong></p><p>Double check the GitHub Secrets. I actually went back and wrote each secret one character at a time. Once I did this, my script run just fine. Another lesson on just taking my time and things will be fine!</p><h4>Conclusion Insights:</h4><p>The hours spent on this, I won’t say but it took me a while. A lot of the time was spent correcting errors I made whether it was in the script or within GitHub Secrets. I want the project to come out perfectly, but there isn’t such a thing. But the beauty in this is the lessons learned within the process. I was able to complete the following:</p><ul><li>Practical understanding of <strong>GitHub Actions</strong> for CI/CD</li><li>End-to-end integration of <strong>AWS Rekognition</strong>, <strong>S3</strong>, and <strong>DynamoDB</strong></li><li>How to safely manage <strong>GitHub Secrets</strong> and AWS credentials</li><li>CI/CD design that separates <strong>beta testing</strong> from <strong>production deployment</strong></li></ul><p>It has been fun. Thank you again fo your time, see you on the next one!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48e7ad404840" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>