<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hadianoor on Medium]]></title>
        <description><![CDATA[Stories by Hadianoor on Medium]]></description>
        <link>https://medium.com/@hadianoor16?source=rss-588e4115da9f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*N3z1lg5VX1twd9Nu</url>
            <title>Stories by Hadianoor on Medium</title>
            <link>https://medium.com/@hadianoor16?source=rss-588e4115da9f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 04:37:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hadianoor16/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[AWS infrastructure deployment with Terraform modules and GitHub Actions]]></title>
            <link>https://medium.com/@hadianoor16/aws-infrastructure-deployment-with-terraform-modules-and-github-actions-72e48f585f13?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/72e48f585f13</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Sat, 18 Oct 2025 07:00:28 GMT</pubDate>
            <atom:updated>2025-10-18T07:11:51.883Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey everyone, this is another fun project that I was working on from few weeks. In this project, I am deploying Jenkins server on EC2 instance deployed in a two tier AWS network. To create VPC with multiple subnets, public and private route tables, defining subnet association, security groups; it is sometimes possible to make an error specially when you are navigating across multiple tabs in UI. Terraform modules simplify infrastructure management by enabling code reusability, consistency, and reduced redundancy.. Honestly, I was so scared of using Terraform modules before but with this project I am very confident about it :)</p><h4>Key Features of Terraform modules:</h4><ol><li><strong>Reusability</strong> : You can write infrastructure code once and reuse it across projects or environments.</li><li><strong>Organization &amp; Maintainability:</strong> The modular structure is easy to maintain and read</li><li><strong>Consistency &amp; Best Practices:</strong> It helps organization with maintaining standards like naming convention, tagging strategies etc.</li><li><strong>Easier Testing &amp; Debugging:</strong> You can test each module independently.</li><li><strong>Reduced Human Error :</strong> By reusing verified modules, you avoid typos, wrong resource arguments, or misconfigured dependencies.</li><li><strong>Scalability :</strong> When you scale infrastructure (e.g., add another environment, region, or service), you just reuse existing modules — no need to copy-paste or modify tons of .tf files.</li></ol><p>Another twist was to deploy all this infrastructure using CI/CD via GitHub Actions. GitHub Actions is very user friendly, as you don’t need separate tool/software to download or create an account. This is all done in GitHub account within Actions tab. It enables teams to build, test, and deploy applications seamlessly.</p><h4>Key Features of GitHub Actions</h4><ol><li><strong>Native Integration with GitHub</strong>: No need for external CI/CD tools — everything runs inside your existing GitHub environment.</li><li><strong>Continuous Integration and Deployment: </strong>GitHub Actions automatically builds, tests, and deploys code whenever changes are made.</li><li><strong>Secrets and Security:</strong> GitHub Actions lets you securely store and use secrets (API keys, credentials, etc.) for use in workflows — ensuring sensitive information is never exposed in code.</li><li><strong>Visibility and Collaboration: </strong>Workflows show up directly in pull requests, commits, and repository Actions tabs, making it easy to see what succeeded of failed, who ran the workflow manually or what triggered the workflow to run.</li><li><strong>Infrastructure as Code Automation:</strong> When used with Terraform, GitHub Actions enables full infrastructure automation like provisioning AWS resources, validate and apply terraform plans.</li></ol><blockquote>For better understanding on TF modules, I suggest watching this video by Rahul Wagh<br><a href="https://www.youtube.com/watch?v=otQqd7GRVK0&amp;list=PLm27MOigQ0WbZy9vs7cyDJpsi--0tekPl&amp;index=7">https://www.youtube.com/watch?v=otQqd7GRVK0&amp;list=PLm27MOigQ0WbZy9vs7cyDJpsi--0tekPl&amp;index=7</a></blockquote><p>With all these features in mind, lets start building our project.</p><p><strong>Steps:</strong></p><ol><li>Create a separate repository and open in VS Code. Include .gitignore file for Terraform in it.</li><li>Open this repo in VS Code and create terraform files like main.tf, variables.tf, terraform.tfvars, provider.tf.</li><li>First keep your infrastructure in mind and start creating modules.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1UkFHMYSBrBNEeB2ymefzw.png" /></figure><h4><strong>Main.tf:</strong></h4><p>The main file will be referencing all the modules that we will create in this project. I created these modules:</p><ol><li>Networking</li><li>Security group</li><li>Jenkins</li><li>Target group</li><li>Application load balancer</li><li>Hosted Zone</li><li>SSL certificate</li></ol><h4>Networking module:</h4><p>In main.tf file you will define all the modules. Now notice that first line under module is <strong>“source = ./networking” </strong>. Source defines the location of the TF code, this module is referencing to. You can also define the variables along with the values that you need to pass to the source folder needed to create the VPC, subnets, route tables for you. Now you must be wondering where is the actual TF code to create infrastructure ? That code lies in another main.tf file under “networking” folder (i.e source). Hence you need to create a folder in root directory with name “networking” that has another main.tf file under it.</p><pre>module &quot;networking&quot;{<br>    source= &quot;./networking&quot;<br>    vpc_cidr= var.vpc_cidr<br>    vpc_name = var.vpc_name<br>    availability_zone = var.availability_zone<br>    cidr_public_subnet= var.cidr_public_subnet<br>    cidr_private_subnet = var.cidr_private_subnet<br>}</pre><p>The main.tf file under the networking folder contains this code:</p><pre>variable &quot;vpc_cidr&quot; {}<br>variable &quot;vpc_name&quot; {}<br>variable &quot;cidr_public_subnet&quot; {}<br>variable &quot;cidr_private_subnet&quot; {}<br>variable &quot;availability_zone&quot; {}<br><br># Output block is helpful when you are referencing vpc_id in another module definition<br>output &quot;dev_proj_1_vpc_id&quot; {<br>  value = aws_vpc.aws_vpc_us_east_1.id<br>}<br><br>output &quot;public_subnet_ids&quot; {<br>  value = [for s in aws_subnet.aws_public_subnet : s.id]  # list(string)<br>}<br><br>resource aws_vpc &quot;aws_vpc_us_east_1&quot; {<br>    cidr_block = var.vpc_cidr<br><br>    tags= {<br>        Name = var.vpc_name<br>    }<br>}<br><br>resource &quot;aws_subnet&quot; &quot;aws_public_subnet&quot; {<br>    count = length(var.cidr_public_subnet)<br>    vpc_id = aws_vpc.aws_vpc_us_east_1.id<br>    cidr_block= element(var.cidr_public_subnet,count.index)<br>    availability_zone= element(var.availability_zone, count.index)<br>    tags ={<br>        Name= &quot;dev_project_public_subnet-${count.index+1}&quot;<br>    }<br><br>}<br><br>resource &quot;aws_subnet&quot; &quot;aws_private_subnet&quot; {<br>    count= length (var.cidr_private_subnet)<br>    vpc_id= aws_vpc.aws_vpc_us_east_1.id<br>    cidr_block = element(var.cidr_private_subnet,count.index)<br>    availability_zone= element(var.availability_zone, count.index)<br><br>    tags= {<br>        Name= &quot;dev_project_private_subnet-${count. index+ 1}&quot;<br><br>    }<br><br>}<br><br># Internet Gateway for VPC<br>resource &quot;aws_internet_gateway&quot; &quot;vpc_us_east_1_igw&quot; {<br>    vpc_id = aws_vpc.aws_vpc_us_east_1.id<br><br>    tags = {<br>        Name= &quot;vpc_us_east_1_igw&quot;<br>    }<br>}<br><br># Public route table <br><br>resource &quot;aws_route_table&quot; &quot;dev_proj_public_RT&quot; {<br>    vpc_id = aws_vpc.aws_vpc_us_east_1.id<br>    route {<br>        cidr_block = &quot;0.0.0.0/0&quot;<br>        gateway_id = aws_internet_gateway.vpc_us_east_1_igw.id<br>    }<br>    tags = {<br>        Name = &quot;dev_proj_public_RT&quot;<br>    }<br>}<br><br># public Route table association<br><br>resource &quot;aws_route_table_association&quot; &quot;dev_proj_public_subnet_association&quot;{<br>    count = length (aws_subnet.aws_public_subnet)<br>    subnet_id = aws_subnet.aws_public_subnet[count.index].id<br>    route_table_id = aws_route_table.dev_proj_public_RT.id<br><br>}<br><br># Private route table<br><br>resource &quot;aws_route_table&quot; &quot;dev_proj_Private_RT&quot; {<br>    vpc_id =aws_vpc.aws_vpc_us_east_1.id<br>    #depends_on = [ aws_nat_gateway.nat_gatway ]<br>    tags={<br>        Name= &quot;dev_proj_private_RT&quot;<br>    }<br>}<br><br># Private RT association<br>resource &quot;aws_route_table_association&quot; &quot;dev_proj_private_subnet_association&quot;{<br>    count = length (aws_subnet.aws_private_subnet)<br>    subnet_id=aws_subnet.aws_private_subnet[count.index].id<br>    route_table_id = aws_route_table.dev_proj_Private_RT.id<br>}</pre><p><em>Output block helps referencing these values in another module definition. For example we will be referencing vpc_id alot in other modules.</em></p><h4>2. Security group module:</h4><p>The actual TF code to create security groups lie in another “main.tf” file under the security group folder.</p><p>Notice you are referencing vpc_id in this module using output block.</p><pre>module &quot;security_group&quot; {<br>    source = &quot;./security_group&quot;<br>    vpc_id= module.networking.dev_proj_1_vpc_id<br>    ec2_sg_name = &quot;Allow http, https, SSH&quot;<br>    ec2_jenkins_sg= &quot;Allow port 8080 for Jenkins&quot;<br>} </pre><p>main.tf file under Security group folder has actual security group creation with inbound and outbound rules.</p><pre>variable &quot;ec2_sg_name&quot; {}<br>variable &quot;ec2_jenkins_sg&quot; {}<br>variable &quot;vpc_id&quot; {}<br><br><br>output &quot;ec2_sg_ssh_http&quot; {<br>value = aws_security_group.ec2_sg.id<br><br>}<br><br>output &quot;ec2_sg_jenkins&quot; {<br>value = aws_security_group.sg_allow_jenkins.id<br>}<br><br><br>resource &quot;aws_security_group&quot; &quot;ec2_sg&quot; {<br>  name        = var.ec2_sg_name<br>  description = &quot;Allow SSH, HTTP and HTTPs from outside&quot;<br>  vpc_id      = var.vpc_id<br><br><br># allow SSH<br>ingress {<br>    description = &quot;Allow remote SSH from anywhere&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>  }<br># Allow http<br>ingress {<br>    description = &quot;Allow HHTP from anywhere&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>  }<br><br>  # allow https<br>  ingress {<br>    description = &quot;Allow HTTPs from anywhere&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 443<br>    to_port     = 443<br>    protocol    = &quot;tcp&quot;<br>  }<br><br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br><br>tags= {<br>    Name = &quot;dev_proj_ec2_sg_ssh_http&quot;<br>}<br>}<br><br>resource &quot;aws_security_group&quot; &quot;sg_allow_jenkins&quot;{<br>    vpc_id= var.vpc_id<br>    name= var.ec2_jenkins_sg<br>    description = &quot;Allow port 8080 for Jenkins&quot;<br><br>    ingress {<br>    description = &quot;Allow 8080 for Jenkins from anywhere&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 8080<br>    to_port     = 8080<br>    protocol    = &quot;tcp&quot;<br>  }<br><br>    egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br><br>  tags= {<br>    Name = &quot;Allow port 8080 to access Jenkins on EC2&quot;<br>  }<br>}</pre><h4>3. Jenkins module:</h4><p>In this module you are creating EC2 instance with Jenkins installed on it. You would need parameters like instance type, AMI, Security group, subnet_id, SSH key, and user data. Notice that you are passing all these values in this section to the jenkins folder with main.tf file.</p><pre>module &quot;jenkins&quot; {<br>    source = &quot;./jenkins&quot;<br>    instance_type= &quot;t2.medium&quot;<br>    ec2_ami= var.ec2_ami<br>    security_group=[module.security_group.ec2_sg_ssh_http, module.security_group.ec2_sg_jenkins]<br>    demokey                    = var.demokey<br>    subnet_id = module.networking.public_subnet_ids[0]<br>    enable_public_ip_address= true<br>    user_data_install_jenkins= templatefile (&quot;./jenkins_user_data/user_data.sh&quot;, {})<br>}</pre><p><em>Here we are referencing security groups from security group module, and subnet id from networking module.</em></p><p><em>User data is installed in another folder with name “jenkins_user_data”.</em></p><p>SSH key to login to the instance was created in UI already, I am just referencing it using data block. You will see it below how..</p><p>main.tf file under jenkins folder looks like this:</p><pre>variable &quot;instance_type&quot; {}<br>variable &quot;ec2_ami&quot; {}<br>variable &quot;subnet_id&quot; {}<br>variable &quot;user_data_install_jenkins&quot; {}<br>variable &quot;demokey&quot; {}<br>variable &quot;security_group&quot; {}<br>variable &quot;enable_public_ip_address&quot; {}<br><br><br>output &quot;ec2_jenkins_server&quot; {<br>  value = aws_instance.jenkins.id<br>}<br><br># data block is used when a resource exists already and you need to reference it<br>data &quot;aws_key_pair&quot; &quot;jenkins_key&quot; {<br>  key_name = var.demokey<br>}<br><br>resource &quot;aws_instance&quot; &quot;jenkins&quot;{<br>    ami = var.ec2_ami<br>    instance_type = var.instance_type<br>    tags ={<br>        Name = &quot;ec2_jenkins&quot;<br>    }<br>    subnet_id= var.subnet_id<br>    vpc_security_group_ids= var.security_group<br>    associate_public_ip_address = var.enable_public_ip_address<br>    user_data= var.user_data_install_jenkins<br>    key_name= data.aws_key_pair.jenkins_key.key_name<br><br><br>metadata_options {<br>    http_endpoint = &quot;enabled&quot;  # Enable the IMDSv2 endpoint<br>    http_tokens   = &quot;required&quot; # Require the use of IMDSv2 tokens<br>  }<br>}</pre><p>User data:</p><pre>#!/bin/bash<br>sudo yum update -y<br>sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo<br>sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key<br>sudo yum upgrade<br>sudo dnf install java-17-amazon-corretto -y<br>sudo yum install jenkins -y<br>sudo systemctl enable jenkins<br>sudo systemctl start jenkins<br>sudo systemctl status jenkins</pre><p>The list goes on and we will keep adding more modules and the source files with actual TF code to deploy those resources for us.</p><p>You can visit my repo for final code:</p><p><a href="https://github.com/hadia-noor16/github-actions">GitHub - hadia-noor16/github-actions: CICD with GitHub Actions</a></p><p>And here is how my root repo looks like: Notice every folder used as source has main.tf file with Terraform code to deploy AWS resource.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7rAqkAu8XpWeR5G3F78beg.png" /></figure><h4>Deploy with GitHub Actions</h4><p>Next step was to deploy it using GitHub Actions, for that you need to create “.github” folder under main repo, and then create “workflows” folder under .github folder. Finally the deploy.yml file with deployment process resides under sub folder like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/760/1*09Ppo_AEeXd9J9gfszLTmQ.png" /></figure><pre>name: Terraform CI/CD<br><br>on:<br>  push:<br>    branches:<br>      - main<br>  pull_request:<br>    branches:<br>       - main<br><br>jobs:<br>  terraform:<br>    name: &#39;Terraform initialize, validate, and deploy&#39;    #testing<br>    runs-on: ubuntu-latest<br><br>    env:<br>      AWS_REGION: &#39;us-east-1&#39;<br>      TF_VAR_availability_zone: &#39;us-east-1a&#39;<br>      TF_VERSION: &#39;1.7.5&#39;<br>      TF_VAR_demokey: demokey<br><br>    steps:<br>    - name: &#39;Checkout GitHub repository&#39;<br>      uses: actions/checkout@v2<br><br>    - name: &#39;Configure AWS credentials&#39;<br>      uses: aws-actions/configure-aws-credentials@v1<br>      with:<br>        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>        aws-region: ${{ env.AWS_REGION }}<br><br>    - name: &#39;Set up Terraform&#39;<br>      uses: hashicorp/setup-terraform@v1<br>      with:<br>        terraform_version: ${{ env.TF_VERSION }}<br><br>    - name: &#39;Terraform Init&#39;<br>      run: terraform init -input=false<br><br>    - name: &#39;Terraform Format&#39;<br>      run: terraform fmt<br><br>    - name: &#39;Terraform Validate&#39;<br>      run: terraform validate<br><br>    - name: &#39;Terraform Plan&#39;<br>      run: terraform plan -input=false -out=tfplan -var-file=terraform.tfvars<br><br>    - name: &#39;Terraform Apply&#39;<br>      if: github.ref == &#39;refs/heads/main&#39;<br>      run: terraform apply -auto-approve tfplan</pre><p>This file is telling all the code when pushed to main branch triggers the workflow that executes all the commands in this file.</p><p>The first job is to run Terraform initialize, validate, plan, and apply. Ofcourse you would need all the credentials to execute this file. For this pupose you need to create secrets in GitHub repo -&gt; Settings -&gt; Secrets and variables -&gt; Actions tab. Click New Repository secret and create AWS access key ID and secret access key to be used by GitHub actions to create AWS resources for you.</p><p>Pass necessary variables like AWS region, SSH key, TF version etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*85awo59Fktdl0nc5OCv11w.png" /></figure><p>After you push this code to main branch, it triggers the pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bu0nRRJFpxf5cYh2hQgpCw.png" /></figure><p>GitHub Actions workflow starts showing all the progress step by step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BpfeoVHcmtLeOGiFZKqF_Q.png" /></figure><p>There were some failures that I kept fixing on the go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h6vHfneoWn9NyzUGfhKPyA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M93oihXZN7XECsUyVh_PXw.png" /></figure><p>After the deployment is complete you can verify resources were created in AWS console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_udz7zbVDgbm76DrwEd8vQ.png" /><figcaption>VPC layout</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jpj7PcERrIYcGiFHlBuCJQ.png" /><figcaption>EC2 instance</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p2avmXrQaxex1jmICCEgvA.png" /><figcaption>Jenkins server served by R53 domain</figcaption></figure><p>Copy the command after connecting via EC2 instance connect to get the admin password, and paste it in “Administrator password”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bUJlWUYOKSGzFS7xH1EAbA.png" /></figure><p>And install the suggested plugins, create the user to get started with Jenkins</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*up1XYg4pbMRFM2F4TUvOMg.png" /></figure><h4>Cleanup.yml:</h4><p>Cleanup.yml file has a cron jon scheduled to run at midnight to destroy all the resources. It is pretty much same as deploy.yml file, with Terraform destroy command instead.</p><h4>Summary:</h4><p>You can go wild with Terraform modules and create as many modules as you can, later re-use them in other fun projects to save time. For now I will wind up here and see you soon with next project!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72e48f585f13" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automated deployment of CI/CD pipeline via Terraform]]></title>
            <link>https://medium.com/@hadianoor16/automated-deployment-of-ci-cd-pipeline-via-terraform-9d553d94b83b?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d553d94b83b</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Thu, 02 Oct 2025 02:43:09 GMT</pubDate>
            <atom:updated>2025-10-04T02:50:18.582Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey all,</p><p>For sometime I have been working on this project of deploying my portfolio website via Terraform. I have used CI/CD to detect any changes made in the website code (sourced GitHub) and continuously deploying those changes to S3 =&gt; CloudFront =&gt; R53 (the personal domain I bought).</p><p>So get ready and follow the steps…</p><p>I created just few things in console, for rest I used Terraform code to create and deploy. First of all I will cover AWS console area.</p><p>You have few things to do in AWS console.</p><h4><strong>1. Create GitHub connector:</strong></h4><p>Go to AWS Code Pipeline and on the left hand side expand Settings and see Connectors there. Click to create new connector. Select “GitHub” and choose name for your connection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*leAIL35F4wCeC96BakOTDg.png" /></figure><p>Click Connect to GitHub.</p><p>Now click on “Install an new app” and it will take you to your GitHub account, where you will authorize by using your password and MFA. <br>Since I had already made the connection before, I am seeing my repository name in the dropdown. Select that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RYe3dmaoqJYN-aohU2-_Ug.png" /></figure><p>Your connection will me made with an ARN that you will use in your Terraform code. This is how it looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F4NY3HNcv_5UDSoUbfHOug.png" /></figure><h4><strong>2. TLS certificate for my website:</strong></h4><p>Now I need a certificate for my website to be used with CloudFront for secure connection. Make sure you have bought your domain already under R53, as it takes sometime to register your domain (it’s about $14 annually)</p><p>Go to Amazon Certificate Manager, click “Request a public certificate”, click next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bRUH6MSoDr_8Sy_Zb_Cbnw.png" /></figure><p>Put your domain names in these fields, and keep DNS validation as default.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dSjWNjHAHNGJmOtGFJ4htQ.png" /></figure><p>Upon clicking next, it will ask you to create R53 records. Note, at this point the status of the certificate shows “Pending validation” as you didn’t create CNAME records yet in R53.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rvzZxP5hfuvatFWKSNCmfg.png" /></figure><p>Once you click “Create records in R53”, it takes you to R53 and the CNAME records will be created like this with the certificate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XKJolFY6gIKbs6EM9M3LFQ.png" /></figure><p>Back on Certificate Manager, the status now shows “Issued” for the requested certificate.</p><h4>3. Set Up Terraform Remote Backend with S3 and DynamoDB</h4><p><strong>Objectives:</strong></p><ul><li>Create an S3 bucket for Terraform state files.</li><li>Enable versioning on the S3 bucket.</li><li>Set up a DynamoDB table for state locking.</li><li>Configure Terraform backend settings in the Terraform configuration files.</li></ul><p>One of the critical aspects of using Terraform effectively is managing the <strong>Terraform state file</strong> — the single source of truth that records your infrastructure’s current status. It also helps with change detection i.e. When running terraform plan or terraform apply, Terraform checks this state file to understand what has changed.</p><blockquote><strong><em>Terraform offers two main ways to store the state file</em></strong><em>:</em></blockquote><ol><li><strong>Local Backend</strong>: Stores the state file on your local machine or a shared file system.</li><li><strong>Remote Backend</strong>: Stores the state file on a remote service, such as AWS S3, ensuring centralized access and better collaboration.</li></ol><p>The Local backend is simple, but it has few challenges like collaboration issues, risk of data loss, limited system memory, managing backups, and security concerns.</p><p>Remote backend is mostly the better choice with AWS S3 being preferred because of its Reliability, durability, version control, encryption of the objects, Access Control, and Recovery. Using S3 as a backend provides several benefits:</p><ul><li><strong>Shared State Management</strong>: It enables teams to share the Terraform state file, ensuring everyone operates with the same view of the infrastructure.</li><li><strong>State Locking with DynamoDB</strong>: Integrating S3 with DynamoDB allows for state locking, preventing simultaneous state modifications that could lead to conflicts or corruption.</li><li><strong>Versioning</strong>: S3’s versioning capabilities provide an extra layer of security by keeping a history of state changes, which can be invaluable for auditing and rollback.</li></ul><p>Next I will create an S3 bucket and DynamoDB table in UI and set up S3 as my Remote backend with DynamoDB for locking the state via Terraform.</p><p>Go to AWS console and search for S3 in the services bar. Make sure you are in <strong><em>us-east-1 region (N. Virginia). </em></strong>Go to S3 and click “Create bucket”. Choose a unique bucket name, keep everything as default but Enable versioning on the bucket. Create bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/0*BAxegsiezB_TSufo" /></figure><p>Now go to DynamoDB service. Create a table for Terraform state locking with a simple hash “<strong><em>LockID</em></strong>” key and one string attribute.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/0*GGL_qk1YDg74C3xv" /><figcaption>Continue with the rest of the default options and create a table.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*anggntE5qk8SdS-a" /></figure><p>Next step is to write magical Terraform code that will create all the AWS resources for you. But you would need to write all that code in a GitHub repository, since you want to use GitHub as a source where your website data would reside. So let’s create a repository next.</p><h4>Initialize GitHub repository</h4><p>Now create a public GitHub repository, choose a name, and select “add .gitignore” template. From the dropdown select Terraform. Create repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zFokTNRa5xiRYtVG04ujSw.png" /></figure><p>Install GitHub Desktop in your computer and open the repository there.</p><p>Create a test branch where you will write all your code and merge it in main. Click “Open in Visual Studio Code”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X4d_-uyCHO6MhtOXmcf7zA.png" /></figure><p>Start writing Terraform code to deploy couple of things for you..</p><ol><li>First you want to declare that the S3 bucket you just created along with the Dynamo Db table serves as a remote backend with state locking.</li><li>Next you need two S3 buckets; dev, and prod, where your website data goes (both buckets are empty right now).</li><li>CloudFront distribution to serve whatever is in prod bucket.</li><li>Artifacts bucket to store all the input/output of pipeline stages.</li><li>AWS Code Pipeline with Source, Build, and Deploy stages along with IAM roles for pipeline.</li><li>Manual approval stage before code could be deployed to prod S3 bucket.</li><li>SNS topic with subscription to the topic to Approve/Deny product deployment.</li><li>Route 53 to serve CloudFront contents.</li></ol><p>Create a folder (example CICD) in the repository, now create a sub-folder where your website data resides. In my case its sample page of my website as index.html and error.html pages. (website is a sub-folder in main folder)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/880/1*JN8_c9UiBVy0BWc4Y2VbKw.png" /></figure><p>Create some files under your main folder with names main.tf, variables.tf, output.tf, provider.tf, and terraform.tfvars.</p><p>I used <em>GitHub connector, approval_email, zone_id, and certificate_arn</em> in <strong>terraform.tfvars</strong> file. Its because this file won’t be pushed to GitHub and I want to keep them hidden. Rest of the variables are in variable.tf file, like bucket names, region, etc. (check code)</p><p>All your configuration goes in main.tf file.</p><blockquote>To access my AWS resources, I created a profile in my Users-&gt; hadia -&gt; .aws -&gt; credentials file, and stored my access keys in it.</blockquote><h4>1. Declaring S3 as Remote backend:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bjL9-p9hD8-TUDDQIavAvQ.png" /></figure><pre># Now initialize Terraform with the following command<br>terraform init</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0U-6zeT32H5DZBp1" /><figcaption>It successfully configured the backend S3</figcaption></figure><pre># Then I ran the following commands<br>terrafrom validate<br>terraform plan -out plan<br>terraform apply -auto-approv</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*F1UmqmT3WEAeLB38ZMY-gQ.png" /><figcaption>I checked my S3 bucket, to verify if the tfstate file in the bucket.</figcaption></figure><h4>2. Creating Dev and Prod bucket for Website hosting:</h4><p>In this code, I created two buckets with bucket policies. Set them accessible to public read, configured them as static website.</p><pre># The s3 bucket for the (dev) environment.<br>resource &quot;aws_s3_bucket&quot; &quot;hadia_dev_s3&quot; {<br>  bucket = var.dev_bucket<br>  force_destroy = true<br>}<br><br># The policy for the (dev) s3 buket to allow public read access.<br><br>resource &quot;aws_s3_bucket_policy&quot; &quot;dev_s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.dev_s3_public_access_block]  <br>  bucket = aws_s3_bucket.hadia_dev_s3.id<br><br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : &quot;arn:aws:s3:::${var.dev_bucket}/*&quot;<br>      }<br>    ]<br>  })<br>}<br><br># Configuring the (dev) s3 bucket as a static website.<br><br>resource &quot;aws_s3_bucket_website_configuration&quot; &quot;dev_s3_website_configuration&quot; {<br>  bucket = aws_s3_bucket.hadia_dev_s3.id<br><br>  index_document {<br>    suffix = &quot;index.html&quot;<br>  }<br><br>  error_document {<br>    key = &quot;error.html&quot;<br>  }<br>}<br>  <br># Configuring public access settings for the (dev) s3 bucket<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;dev_s3_public_access_block&quot;{<br>  bucket = aws_s3_bucket.hadia_dev_s3.id<br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}<br><br># s3 bucket for the (prod) environment  <br>  <br>resource &quot;aws_s3_bucket&quot; &quot;hadia_pro_s3&quot; {<br>  bucket = var.pro_bucket<br>  force_destroy = true<br>}<br><br># Policy for the (prod) s3 bucket to allow public read access<br><br>resource &quot;aws_s3_bucket_policy&quot; &quot;pro_s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.pro_s3_public_access_block]<br>  bucket = aws_s3_bucket.hadia_pro_s3.id<br><br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : &quot;arn:aws:s3:::${var.pro_bucket}/*&quot;<br>      }<br>    ]<br>  })<br>}<br><br># Configuring the (prod) s3 bucket as a static website<br><br>resource &quot;aws_s3_bucket_website_configuration&quot; &quot;pro_s3_website_configuration&quot; {<br>  bucket = aws_s3_bucket.hadia_pro_s3.id<br><br>  index_document {<br>    suffix = &quot;index.html&quot;<br>  }<br><br>  error_document {<br>    key = &quot;error.html&quot;<br>  }<br>}<br><br># Configuring public access settings for the (prod) s3 bucket<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;pro_s3_public_access_block&quot;{<br>  bucket = aws_s3_bucket.hadia_pro_s3.id<br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}</pre><h4>3. Creating CloudFront distribution:</h4><p>Creating CloudFront distribution with index.html as the default root object. Setting origin as prod bucket and redirecting every http to https request. I also used aliases for my website and tied ACM certificate with its ARN to each request.</p><pre># Creating CF distribution to serve prod bucket contents<br>resource &quot;aws_cloudfront_origin_access_control&quot; &quot;oac&quot; {<br>  name                              = &quot;${var.pro_bucket}-oac&quot;<br>  description                       = &quot;OAC for ${var.pro_bucket}&quot;<br>  origin_access_control_origin_type = &quot;s3&quot;<br>  signing_behavior                  = &quot;always&quot;<br>  signing_protocol                  = &quot;sigv4&quot;<br><br>}<br>resource &quot;aws_cloudfront_distribution&quot; &quot;s3_distribution&quot; {<br># This will define the origin which is the s3 bucket<br>origin {<br>    origin_id                = &quot;s3-${var.pro_bucket}&quot;<br>    domain_name = &quot;${aws_s3_bucket.hadia_pro_s3.bucket_regional_domain_name}&quot;<br>    origin_access_control_id = aws_cloudfront_origin_access_control.oac.id<br>    <br>}<br><br><br># Enable the distribution and configure basic settings<br>  enabled             = true<br>  is_ipv6_enabled     = true<br>  comment             = &quot;mydevopslife&quot;<br>  default_root_object = &quot;index.html&quot;<br><br>    default_cache_behavior {<br>    allowed_methods  = [&quot;DELETE&quot;, &quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;, &quot;PATCH&quot;, &quot;POST&quot;, &quot;PUT&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id = &quot;s3-${var.pro_bucket}&quot;<br>forwarded_values {<br>      query_string = false<br>cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }<br>    min_ttl                = 0<br>    default_ttl            = 3600<br>    max_ttl                = 86400<br>    compress             = true<br>    viewer_protocol_policy = &quot;redirect-to-https&quot;<br>  }<br><br># Pricing class for the distribution <br>price_class = &quot;PriceClass_200&quot;<br>restrictions {<br>    geo_restriction {<br>      restriction_type = &quot;whitelist&quot;<br>      locations        = [&quot;US&quot;]<br>    }<br>  }<br>tags = {<br>    Environment = &quot;production&quot;<br>  } <br><br>  aliases= [&quot;mydevopslife.com&quot;, &quot;www.mydevopslife.com&quot;]<br><br># SSL/TLS cerificate settings<br><br>viewer_certificate {<br>    acm_certificate_arn      = var.certificate_arn<br>    ssl_support_method       = &quot;sni-only&quot;<br>    minimum_protocol_version = &quot;TLSv1.2_2021&quot;<br>  }<br><br>}</pre><h4>4. Creating Artifact bucket:</h4><p>Now I will create artifact bucket to be used by all three stages; source, build, and deploy. Make sure when you write IAM role for code pipeline and Code Build, you give GET/PUT objects allowed into this artifact bucket.</p><p>Here goes my Artifact bucket:</p><pre># s3 srtifacts bucket for ci/cd pipeline<br>resource &quot;aws_s3_bucket&quot; &quot;artifacts&quot; {<br>  bucket = var.artifact_bucket<br>  force_destroy = true<br>}<br>resource &quot;aws_s3_bucket_ownership_controls&quot; &quot;owner&quot; {<br>  bucket = aws_s3_bucket.artifacts.id<br>  rule {<br>    object_ownership = &quot;BucketOwnerPreferred&quot;<br>  }<br>}<br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;acl&quot; {<br>  bucket = aws_s3_bucket.artifacts.id<br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}<br>resource &quot;aws_s3_bucket_policy&quot; &quot;s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.acl]<br>  bucket = aws_s3_bucket.artifacts.id<br> <br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : [&quot;${aws_s3_bucket.artifacts.arn}/*&quot;]<br>      }<br>    ]<br>  })<br>}</pre><h4>5. Service role for CodeBuild stage</h4><p>Service role for CodeBuild stage and Code Pipeline allows to use GitHub connection string (the very first step where we created GitHub connection), create log groups, put/get objects from artifact bucket.</p><pre># Service role for CodeBuild and assume role<br><br>data &quot;aws_iam_policy_document&quot; &quot;assume_role&quot; {<br>  statement {<br>    effect = &quot;Allow&quot;<br><br>    principals {<br>      type        = &quot;Service&quot;<br>      identifiers = [&quot;codebuild.amazonaws.com&quot;]<br>    }<br><br>    actions = [&quot;sts:AssumeRole&quot;]<br>  }<br>}<br><br>resource &quot;aws_iam_role&quot; &quot;example&quot; {<br>  name               = &quot;example&quot;<br>  assume_role_policy = data.aws_iam_policy_document.assume_role.json<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;example&quot; {<br>  statement {<br>    effect = &quot;Allow&quot;<br><br>    actions = [<br>      &quot;logs:CreateLogGroup&quot;,<br>      &quot;logs:CreateLogStream&quot;,<br>      &quot;logs:PutLogEvents&quot;,<br>    ]<br><br>    resources = [&quot;*&quot;]<br>  }<br><br><br>  statement {<br>    effect  = &quot;Allow&quot;<br>    actions = [&quot;s3:*&quot;]<br>    resources = [<br>      &quot;${aws_s3_bucket.artifacts.arn}/*&quot;<br>    ]<br>  }<br><br>  statement {<br>    effect = &quot;Allow&quot;<br>    actions = [<br>      &quot;codeconnections:GetConnectionToken&quot;,<br>      &quot;codeconnections:GetConnection&quot;<br>    ]<br>    resources = [var.github_connector]<br>  }<br>}<br><br>resource &quot;aws_iam_role_policy&quot; &quot;example&quot; {<br>  role   = aws_iam_role.example.name<br>  policy = data.aws_iam_policy_document.example.json<br>}</pre><h4>6. AWS Code Pipeline with Source, Build, and Deploy stages with IAM role for pipeline:</h4><p>Next step is to write code to create AWS code pipeline, where you define all the stages of pipeline.</p><pre># CI/CD Build phase with github repo as source, defining where build project artifacts go to in s3 bucket, and the specs of server to run this task<br><br>resource &quot;aws_codebuild_project&quot; &quot;project&quot; {<br>  name          = &quot;tf-codebuild-project&quot;<br>  description   = &quot;Build project for my portfolio&quot;<br>  service_role  = aws_iam_role.example.arn<br>  source {<br>    type      = &quot;GITHUB&quot;<br>    location  = &quot;https://github.com/hadia-noor16/CICD/website&quot;<br>    buildspec = &lt;&lt;-YAML<br>    version: 0.2<br>    phases:<br>      build:<br>        commands:<br>           - echo &quot;This is build phase&quot;<br> <br>      post_build:<br>        commands:<br>           - echo &quot;Deployed to S3&quot;<br>    artifacts:<br>      base-directory: website    # take files from website/ i.e. only contents from website folder is copied to dev bucket<br>      files:<br>        - &#39;**/*&#39; <br>               # include everything from website/ folder<br>YAML<br>}<br><br>  artifacts {<br>    type = &quot;S3&quot;<br>    location = aws_s3_bucket.artifacts.id<br>  }<br><br><br>  environment {<br>    compute_type                = &quot;BUILD_GENERAL1_SMALL&quot;<br>    image                       = &quot;aws/codebuild/amazonlinux2-x86_64-standard:4.0&quot;<br>    type                        = &quot;LINUX_CONTAINER&quot;<br>    image_pull_credentials_type = &quot;CODEBUILD&quot;<br> }<br>}<br><br><br># Code pipeline with source, build, and deploy stage (to s3 dev bucket)<br>resource &quot;aws_codepipeline&quot; &quot;pipeline&quot; {<br>  name     = &quot;tf-pipeline&quot;<br>  role_arn = aws_iam_role.codepipelinerole.arn<br>artifact_store {<br>    location = aws_s3_bucket.artifacts.id<br>    type     = &quot;S3&quot;<br>  }<br>  stage {<br>    name = &quot;Source&quot;<br>    action {<br>      name             = &quot;Source&quot;<br>      category         = &quot;Source&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;CodeStarSourceConnection&quot;<br>      version          = &quot;1&quot;<br>      output_artifacts = [&quot;source_output&quot;]<br>      configuration = {<br>        ConnectionArn = var.github_connector  # create variable and save the connector name in .tfvars file<br>        FullRepositoryId = &quot;hadia-noor16/CICD&quot;  # Update with your GitHub repo name<br>        BranchName = &quot;main&quot;  # Specify the branch<br>        <br>      }<br>    }<br>  }<br>  stage {<br>    name = &quot;Build&quot;<br>    action {<br>      name             = &quot;Build&quot;<br>      category         = &quot;Build&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;CodeBuild&quot;<br>      version          = &quot;1&quot;<br>      input_artifacts  = [&quot;source_output&quot;]<br>      output_artifacts = [&quot;build_output&quot;]<br>      configuration = {<br>        ProjectName = aws_codebuild_project.project.name<br>      }<br>    }<br>  }<br>  stage {<br>    name = &quot;Deploy&quot;<br>    action {<br>      name             = &quot;Deploy&quot;<br>      category         = &quot;Deploy&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;S3&quot;<br>      version          = &quot;1&quot;<br>      input_artifacts  = [&quot;build_output&quot;]<br>      configuration = {<br>        BucketName = var.dev_bucket  # my dev bucket<br>        Extract    = &quot;true&quot;<br>      }<br>    }<br>  }<br><br>  #  Manual approval stage before PROD<br>  stage {<br>    name = &quot;ApproveProd&quot;<br>    action {<br>      name      = &quot;ManualApproval&quot;<br>      category  = &quot;Approval&quot;<br>      owner     = &quot;AWS&quot;<br>      provider  = &quot;Manual&quot;<br>      version   = &quot;1&quot;<br>      configuration = {<br>        CustomData       = &quot;Approve deploy to S3 PROD bucket?&quot;<br>        NotificationArn  = aws_sns_topic.codepipeline_approvals.arn<br>    <br>      }<br>    }<br>  }<br><br>  # Deploy to PROD S3 (after approval)<br>  stage {<br>    name = &quot;DeployProd&quot;<br>    action {<br>      name            = &quot;DeployToProd&quot;<br>      category        = &quot;Deploy&quot;<br>      owner           = &quot;AWS&quot;<br>      provider        = &quot;S3&quot;<br>      version         = &quot;1&quot;<br>      input_artifacts = [&quot;build_output&quot;]   # same artifact from Build stage<br>      configuration = {<br>        BucketName = var.pro_bucket<br>        Extract    = &quot;true&quot;<br>      }<br>    }<br>  }<br>}</pre><p>Here is what code pipeline role looks like, notice that pipeline needs to have access to so many other resources like getting/putting contents in Dev and Prod S3 bucket, artifacts bucket, publishing to SNS topic, also use connection for GitHub. So its IAM policy looks like this:</p><pre># CodePipeline role<br><br>data &quot;aws_iam_policy_document&quot; &quot;assume_role_pipeline&quot; {<br>  statement {<br>    effect = &quot;Allow&quot;<br><br>    principals {<br>      type        = &quot;Service&quot;<br>      identifiers = [&quot;codepipeline.amazonaws.com&quot;]<br>    }<br><br>    actions = [&quot;sts:AssumeRole&quot;]<br>  }<br>}<br><br>resource &quot;aws_iam_role&quot;  &quot;codepipelinerole&quot; {<br>  name = &quot;codepipelinerole&quot;<br>  assume_role_policy = data.aws_iam_policy_document.assume_role_pipeline.json<br>}<br><br>data &quot;aws_iam_policy_document&quot; &quot;codepipeline_policy&quot; {<br>  statement {<br>    effect = &quot;Allow&quot;<br><br>    actions = [<br>      &quot;s3:GetObject&quot;,<br>      &quot;s3:GetObjectVersion&quot;,<br>      &quot;s3:GetBucketVersioning&quot;,<br>      &quot;s3:PutObjectAcl&quot;,<br>      &quot;s3:PutObject&quot;,<br>    ]<br><br>    resources = [<br>      aws_s3_bucket.artifacts.arn,<br>      &quot;${aws_s3_bucket.artifacts.arn}/*&quot;,<br>      aws_s3_bucket.hadia_dev_s3.arn,<br>      &quot;${aws_s3_bucket.hadia_dev_s3.arn}/*&quot;,<br>       aws_s3_bucket.hadia_pro_s3.arn,<br>       &quot;${aws_s3_bucket.hadia_pro_s3.arn}/*&quot;<br><br>    ]<br>  }<br><br>  statement {<br>    effect    = &quot;Allow&quot;<br>    actions   = [&quot;codestar-connections:UseConnection&quot;]<br>    resources = [var.github_connector]<br>  }<br><br>  statement {<br>    effect = &quot;Allow&quot;<br><br>    actions = [<br>      &quot;codebuild:BatchGetBuilds&quot;,<br>      &quot;codebuild:StartBuild&quot;,<br>    ]<br><br>    resources = [&quot;*&quot;]<br>  }<br>  statement {<br>    effect = &quot;Allow&quot;<br>    actions = [&quot;sns:Publish&quot;]<br>    resources = [aws_sns_topic.codepipeline_approvals.arn]<br>  }<br>}</pre><h4>7. Creating SNS topic and subscribing to it:</h4><p>SNS topic is created when an approval is needed from devs/manager to deploy website to S3 prod bucket. Its a safe practice. Here is how you create SNS topic and subscribe yourself to the topic.</p><pre>resource &quot;aws_sns_topic&quot; &quot;codepipeline_approvals&quot; {<br>  name = &quot;codepipeline-approvals&quot;<br>}<br><br># Email subscription for approval notifications (optional)<br>resource &quot;aws_sns_topic_subscription&quot; &quot;approver_email&quot; {<br>  count     = var.approval_email == null ? 0 : 1<br>  topic_arn = aws_sns_topic.codepipeline_approvals.arn<br>  protocol  = &quot;email&quot;<br>  endpoint  = var.approval_email<br>}</pre><h4>8. Creating Route 53 records to serve CloudFront:</h4><p>Now the final step is to serve CloudFront contents via the route 53 domain we created CNAME records for.</p><pre># Route 53 record to serve the cloudfornt distribution<br># A record (IPv4)<br>resource &quot;aws_route53_record&quot; &quot;a_alias&quot; {<br>  zone_id = var.zone_id<br>  name    = &quot;mydevopslife.com&quot;<br>  type    = &quot;A&quot;<br><br>  alias {<br>    name                   = aws_cloudfront_distribution.s3_distribution.domain_name<br>    zone_id =              aws_cloudfront_distribution.s3_distribution.hosted_zone_id<br>    evaluate_target_health = false<br>  }<br>}<br><br># AAAA record (IPv6) <br>resource &quot;aws_route53_record&quot; &quot;aaaa_alias&quot; {<br>  zone_id = var.zone_id<br>  name    = &quot;www.mydevopslife.com&quot;<br>  type    = &quot;AAAA&quot;<br><br>  alias {<br>    name                   = aws_cloudfront_distribution.s3_distribution.domain_name<br>    zone_id                = aws_cloudfront_distribution.s3_distribution.hosted_zone_id<br>    evaluate_target_health = false<br>  }<br>}</pre><p>At this point your code is ready, also create whatever variables you used in your configuration. You can add as many as you want.</p><p>Use terraform commands now:</p><pre>terraform init  <br>terrform validate<br>Terraform plan<br>Terraform apply</pre><p>Fix any errors you might see as a result of these commands.</p><p>You will see resources created in your AWS account as everything comes correct.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9IdDKxSdbeEuXKn_a_yBeQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5xlKGvtS1gyN-vrNb5UQIA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rpwQXrSp3mm4KHYIu8DpuA.png" /><figcaption>Empty Buckets are created with named dev and prod.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IS8CkZWotgx_in93rKyVGg.png" /><figcaption>CloudFront is in deploying state</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ctZRInxcoL9Ymtwao4OCKw.png" /><figcaption>Pipeline is created with all stages</figcaption></figure><p>Now to test the pipeline, I will make change to the index.html page under my website folder. It will trigger pipeline and copy the contents from website folder to the dev bucket. The pipeline will stay on approval stage unless I approve to deploy in prod bucket. The prod bucket will be served by CloudFront and my R53 records will show the website page.</p><p>Committing index.html file after making slight change in my website page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-Bg902WcF4cc46_Gy-YQCQ.png" /></figure><p>Click Push origin</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bKipCmshm47RZBBN07t_RA.png" /></figure><p>Preview pull request</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q34Zwk4rlINeHctEn7dmrQ.png" /></figure><p>Create Pull request</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OYt5NsR0O_qAOny31oFkNg.png" /></figure><p>Click merge pull request</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m8XGUuxyLjwz11KswKcuvg.png" /></figure><p>Pipeline is triggered again</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oj9Byf5hbwJvBGK6hs7fgw.png" /></figure><p>Dev bucket has website files now</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*miaVLyyYGIhjoYHfZgIKHQ.png" /></figure><p>Prod bucket is still empty, since I didn’t approve yet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sWsII7_pvmfzWyJLtDCt5A.png" /></figure><p>Approving now..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b5rSqyaEtyec3lpFW9DkPg.png" /></figure><p>You can also approve it via SNS topic that I created, but after you confirm subscribing to the topic, you will receive an email like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KJfW3dXJ-cuXdn-gavTBag.png" /></figure><p>Deployment to prod bucket is successful now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*miJvXWH7YhCtUzSN3EprGA.png" /></figure><p>Prod bucket:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5fLpoaLGVB0u-rNsjI2KiQ.png" /></figure><p>CloudFront:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jizAO1TMz0p2MapfYrPeJw.png" /></figure><p>My website shows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bpf7CGS8RkIi3S-IUUuvXw.png" /></figure><p>So, I was able to achieve continuous automated infrastructure deployment via Terraform.</p><p>In the end, don’t forget to run “terraform destroy” command, to avoid any bills.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y3yPZtLZAUsX2kujaHtrZQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d553d94b83b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying A Highly Available 2-Tier Architecture with Terraform]]></title>
            <link>https://medium.com/@hadianoor16/deploying-a-highly-available-2-tier-architecture-with-terraform-d18652ccc882?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d18652ccc882</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Thu, 31 Oct 2024 00:07:46 GMT</pubDate>
            <atom:updated>2024-10-31T00:07:46.022Z</atom:updated>
            <content:encoded><![CDATA[<p>Terraform is one of the popular Infrastructure as a Code (IaC) tools to automate the process of creating and managing resources. Together it can integrate with many Cloud providers like AWS, GCP, Azure to deploy efficient resources or services in cloud.</p><p>Terraform makes it easy to create, manage and build the resources, with version control and collaboration. In a Two-Tier setup, you have the user interface on one side and the database on the other with the two communicating directly. This is great for smaller simpler applications but as things get more complex a Three-Tier architecture is often better.</p><p>So, let&#39;s build this simple two-tier architecture via Terraform.</p><p><strong>Requirements:</strong></p><ol><li>Create a highly available two-tier AWS architecture containing the following:</li><li>Custom VPC with:</li></ol><ul><li>2 Public Subnets for the Web Server Tier</li><li>2 Private Subnets for the RDS Tier</li><li>Appropriate route tables</li></ul><p>3. Launch an EC2 Instance with your choice of webserver in each public web tier subnet (Apache, NGINX, etc.).</p><p>4. One RDS MySQL Instance (micro) in the private RDS subnets</p><p>5. Security Groups properly configured for needed resources (web servers, RDS)</p><p>6. Push your code to GitHub and include the link in your write up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/932/1*8wa8pSB9X0WlhpqEcwOngQ.png" /><figcaption>My network diagram!</figcaption></figure><h4><strong>Steps:</strong></h4><ol><li>Create a branch in Terraform repository, publish that branch to remote and open the branch in VS Code.</li><li>Create a new folder in terraform directory and create three files <strong><em>main.tf</em></strong>, <strong><em>variables.tf </em></strong>and <strong><em>terraform.tfvars</em></strong> within that folder.</li></ol><h4><strong><em>Variales.tf file:</em></strong></h4><p>Variable file contains all the attributes that can be changed in your configuration, but you don’t want to change your code every time. Simply change the values in variables.tf file and reference them in main file.</p><p>I also created variable for ACCESS_KEY and SECRET_ACCESS_KEY, but notice that I don’t have actual keys in variables.tf file</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*L_nADRCXh3H_tU_7bWv1ew.png" /></figure><p>This is because it&#39;s not safe as this file contains all the other variables that our code needs, hence you can’t keep it secret from anyone using your code. A safe way is to create a file with extension “.tfvars” and store the access key and secret access keys there and refer to them in the variables.tf file.</p><h4><strong><em>terraform.tfvars file:</em></strong></h4><p>In this file, you are defining the dynamic values for the variables declared in variables.tf file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/712/1*Uc1VzebbS1ehiacUZ6JTpA.png" /></figure><h4><strong><em>main.tf file</em></strong></h4><p>Start your main.tf file with provider block and provide your access key and secret access key as variables.</p><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;5.72.1&quot;<br>    }<br>  }<br>}<br><br>provider &quot;aws&quot; {<br>  region     = var.aws_region<br>  access_key = var.access_key<br>  secret_key = var.secret_key<br>}</pre><h4><strong>Creating VPC along with subnets</strong></h4><p>Now creating VPC along with four subnets, two public and two private subnets, spanning in two availability zones.</p><p>VPC Cidr: 10.20.0.0/16</p><p>Public subnets:</p><ul><li>subnet-1: 10.20.1.0/24</li><li>subnet-2: 10.20.2.0/24</li></ul><p>Private subnets:</p><ul><li>subnet-1: 10.20.101.0/24</li><li>subnet-2: 10.20.102.0/24</li></ul><p>This is the simple code below.</p><pre>#defining vpc<br>resource &quot;aws_vpc&quot; &quot;two-tier-vpc&quot; {<br>  cidr_block           = &quot;10.20.0.0/16&quot;<br>  enable_dns_hostnames = true<br>  enable_dns_support   = true<br><br>  tags = {<br>    Name = &quot;Two-tier VPC&quot;<br>  }<br><br>}<br><br>#Creating public subnet in AZ-A<br>resource &quot;aws_subnet&quot; &quot;public-1a&quot; {<br>  vpc_id                  = aws_vpc.two-tier-vpc.id<br>  cidr_block              = &quot;10.20.1.0/24&quot;<br>  availability_zone       = &quot;us-east-1a&quot;<br>  map_public_ip_on_launch = true<br><br>  tags = {<br>    Name = &quot;publicA&quot;<br>  }<br>}<br><br>#Creating public subnet in AZ-B<br>resource &quot;aws_subnet&quot; &quot;public-1b&quot; {<br>  vpc_id                  = aws_vpc.two-tier-vpc.id<br>  cidr_block              = &quot;10.20.2.0/24&quot;<br>  availability_zone       = &quot;us-east-1b&quot;<br>  map_public_ip_on_launch = true<br><br>  tags = {<br>    Name = &quot;publicB&quot;<br>  }<br>}<br><br><br>#Creating private subnet in AZ-A<br>resource &quot;aws_subnet&quot; &quot;private-1a&quot; {<br>  vpc_id            = aws_vpc.two-tier-vpc.id<br>  cidr_block        = &quot;10.20.101.0/24&quot;<br>  availability_zone = &quot;us-east-1a&quot;<br><br>  tags = {<br>    Name = &quot;PrivateA&quot;<br>  }<br>}<br><br>#Creating private subnet in AZ-B<br>resource &quot;aws_subnet&quot; &quot;private-1b&quot; {<br>  vpc_id            = aws_vpc.two-tier-vpc.id<br>  cidr_block        = &quot;10.20.102.0/24&quot;<br>  availability_zone = &quot;us-east-1b&quot;<br><br>  tags = {<br>    Name = &quot;PrivateB&quot;<br>  }<br>}</pre><p>Now attaching internet gateway to our custom VPC. Check Terraform registry<br><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/internet_gateway">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/internet_gateway</a></p><pre>#creates internet gateway<br>resource &quot;aws_internet_gateway&quot; &quot;igw&quot; {<br>  vpc_id = aws_vpc.two-tier-vpc.id<br><br>  tags = {<br>    Name = &quot;IGW&quot;<br>  }<br>}</pre><h4><strong>Creating Public route table and associating subnets:</strong></h4><p>First, I created public route table and associated subnets 1 and 2 to this route table. This is public route table as it defines the route to the internet through the Internet Gateway.</p><pre>#creates public RT<br>resource &quot;aws_route_table&quot; &quot;publicRT&quot; {<br>  vpc_id = aws_vpc.two-tier-vpc.id<br><br>  route {<br>    cidr_block = &quot;0.0.0.0/0&quot;<br>    gateway_id = aws_internet_gateway.igw.id<br>  }<br><br>  tags = {<br>    Name = &quot;Public-RT&quot;<br>  }<br>}<br><br># creating public subnet 1 association with public RT<br>resource &quot;aws_route_table_association&quot; &quot;rt-association-public1&quot; {<br>  subnet_id      = aws_subnet.public-1a.id<br>  route_table_id = aws_route_table.publicRT.id<br>}<br><br># creating public subnet 2 association with public RT<br>resource &quot;aws_route_table_association&quot; &quot;rt-association-public2&quot; {<br>  subnet_id      = aws_subnet.public-1b.id<br>  route_table_id = aws_route_table.publicRT.id<br>}</pre><h4><strong>Create Private route table and associating private subnets</strong></h4><p>To create a private route table, we still need a route to the internet for our private instances to download any packages if they need to.</p><p>This purpose is served by NAT Gateway. Nat gateways are (by default) placed in public subnets so that the instances in a private subnet can connect to services outside your VPC, but external services cannot initiate a connection with those instances.</p><p>You create a public NAT gateway in a public subnet and must associate an elastic IP address with the NAT gateway at creation. You route traffic from the NAT gateway to the internet gateway for the VPC.</p><p>So, our next step is to allocate elastic IP addresses for two NAT gateways. (As we have two public subnets, it&#39;s a good practice in terms of high availability). You can always keep one NAT gateway and route traffic from both private subnets through this NAT gateway. You can get help from Terraform registry.</p><p><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/nat_gateway">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/nat_gateway</a></p><pre># creates Elastic EIP for first NAT Gateway<br>resource &quot;aws_eip&quot; &quot;eip1&quot; {<br>  domain = &quot;vpc&quot;<br>}<br><br># creates Elastic EIP for second NAT Gateway<br>resource &quot;aws_eip&quot; &quot;eip2&quot; {<br>  domain = &quot;vpc&quot;<br>}<br><br><br># NAT Gateway Public Availability Zone: A<br>resource &quot;aws_nat_gateway&quot; &quot;natgw-A&quot; {<br>  subnet_id     = aws_subnet.public-1a.id<br>  allocation_id = aws_eip.eip1.id<br>  tags = {<br>    Name = &quot;NAT-GW-Public-A&quot;<br>  }<br>  depends_on = [aws_internet_gateway.igw]<br>}<br><br><br>#NAT Gateway Public Availability Zone: B<br>resource &quot;aws_nat_gateway&quot; &quot;natgw-B&quot; {<br>  subnet_id     = aws_subnet.public-1b.id<br>  allocation_id = aws_eip.eip2.id<br>  tags = {<br>    Name = &quot;NAT-GW-Public-B&quot;<br>  }<br>  depends_on = [aws_internet_gateway.igw]<br>}</pre><p>I created two EIPs, created NAT gateways and allocated elastic IP addresses to both.</p><p>Next step is to create Private route tables and associate private subnets to them. This is the code…</p><pre>#creates private RT for AZ-A<br>resource &quot;aws_route_table&quot; &quot;privateRT-A&quot; {<br>  vpc_id = aws_vpc.two-tier-vpc.id<br><br>  route {<br>    cidr_block = &quot;0.0.0.0/0&quot;<br>    gateway_id = aws_nat_gateway.natgw-A.id<br>  }<br>  tags = {<br>    Name = &quot;Private-RT-A&quot;<br>  }<br>}<br><br><br>#creates private RT for AZ-B<br>resource &quot;aws_route_table&quot; &quot;privateRT-B&quot; {<br>  vpc_id = aws_vpc.two-tier-vpc.id<br><br>  route {<br>    cidr_block = &quot;0.0.0.0/0&quot;<br>    gateway_id = aws_nat_gateway.natgw-B.id<br>  }<br><br>  tags = {<br>    Name = &quot;Private-RT-B&quot;<br>  }<br>}<br><br># creating private subnet association with Private RT A<br>resource &quot;aws_route_table_association&quot; &quot;rt-association-privatea&quot; {<br><br>  subnet_id      = aws_subnet.private-1a.id<br>  route_table_id = aws_route_table.privateRT-A.id<br>}<br><br># creating private subnet association with Private RT B<br>resource &quot;aws_route_table_association&quot; &quot;rt-association-privateb&quot; {<br>  subnet_id      = aws_subnet.private-1b.id<br>  route_table_id = aws_route_table.privateRT-B.id<br>}</pre><h4><strong>Create Autoscaling group</strong></h4><p>Now that all out network is laid out, the next step is to create autoscaling group for our instances to create two EC2 in public subnets that are hosting our application (I am creating Apache webservers) and one RDS instance that&#39;s in the private subnet.</p><p>Autoscaling group deploys instances based on the launch template we create/choose for our instances, so we will create security group for our instances and the launch template to follow.</p><pre>#Creating Security Group for webservers<br>resource &quot;aws_security_group&quot; &quot;EC2SG&quot; {<br>  name        = &quot;Two tier-EC2SG&quot;<br>  description = &quot;Security Group for EC2 webservers for two tier infrastructure&quot;<br>  vpc_id      = aws_vpc.two-tier-vpc.id<br>  ingress {<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>    #security_groups =aws_security_group.alb-sg.id<br>  }<br>  egress {<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>  }<br>  tags = {<br>    &quot;Name&quot; = &quot;Instance Security Group&quot;<br>  }<br>}<br># Creating launch template for Apache webservers<br>resource &quot;aws_launch_template&quot; &quot;twotier-lt&quot; {<br>  name_prefix            = &quot;twotier-LT&quot;<br>  image_id               = var.ami_id<br>  instance_type          = var.instance_type<br>  vpc_security_group_ids = [&quot;${aws_security_group.EC2SG.id}&quot;]<br>  user_data              = filebase64(&quot;user_data.sh&quot;)<br>}<br><br><br># Auto scaling group for apache webservers<br>resource &quot;aws_autoscaling_group&quot; &quot;twotier-ASG&quot; {<br>  vpc_zone_identifier       = [aws_subnet.public-1a.id, aws_subnet.public-1b.id]<br>  health_check_grace_period = 300<br>  health_check_type         = &quot;EC2&quot;<br>  desired_capacity          = 2<br>  max_size                  = 5<br>  min_size                  = 2<br><br>  launch_template {<br>    id      = aws_launch_template.twotier-lt.id<br>    version = &quot;$Latest&quot;<br>  }<br>}</pre><h4><strong>Creating RDS instance</strong></h4><p>In this step, we will create a security group for our database instance and create MySQL RDS.</p><p>Note: For latest engine, go to console and check. My db instance failed as I had some compatibility issues with the engine and instance size.</p><p>Terraform registry link: <a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance</a></p><pre>#create a security group for RDS Database Instance<br>resource &quot;aws_security_group&quot; &quot;rds_sg&quot; {<br>  name = &quot;rds_sg&quot;<br>  ingress {<br>    from_port   = 3306<br>    to_port     = 3306<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br>}<br><br>#create a RDS Database Instance<br>resource &quot;aws_db_instance&quot; &quot;two-tier-rds&quot; {<br>  engine                 = &quot;mysql&quot;<br>  identifier             = &quot;myrds&quot;<br>  allocated_storage      = 20<br>  engine_version         = &quot;8.0.35&quot;<br>  instance_class         = &quot;db.t3.micro&quot;<br>  username               = var.rdsuser<br>  password               = var.rdspassword<br>  parameter_group_name   = &quot;default.mysql8.0&quot;<br>  vpc_security_group_ids = [aws_security_group.rds_sg.id]<br>  skip_final_snapshot    = true<br>  publicly_accessible    = true<br>}</pre><h4><strong>Terraform commands:</strong></h4><p>First, we will need to initialize Terraform, so it downloads any providers required for our configuration.</p><p>Then we will validate Terraform if configuration is valid.</p><p>Next command is to see an execution plan, showing what actions Terraform will take to achieve the desired state.</p><p>And finally, you deploy all this plan.</p><p>Here are the commands…</p><pre>Terraform init<br>Terraform validate<br>Terraform plan -out plan<br>Terraform apply</pre><p><strong>Troubleshooting:</strong></p><p>When I ran “terraform plan” it showed me all the errors that I had made. I was missing public subnet Ids in Autoscaling group, and w=security group ID for RDS instance</p><p>This was the error, I was missing “.id” with subnets and security group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*951r5ODcjCjFrEOMEuezxA.png" /></figure><p>This is what I fixed in my main file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*flsftCyPox19Y2Swfz63iQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/681/1*pQi-P2TnK1QfVpRwyNb7IA.png" /></figure><p>When I ran “terraform plan” again, it ran fine with no errors, so I deployed my plan.</p><p><strong>Deployment successful:</strong></p><p>I was able to successfully deploy VPC, subnets, route tables, NAT gateways, EC2 and RDS instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_iLiL_AYIHrR_UnJ31_3zw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Umz5KAzsUEbdRAWvejaRQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cs8clPrnz458doVPpULeJA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NjzFQW_CIC6SvrmpKomu6w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AHtIa_2ktt-6CcYw0loHPw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XTM7XoTECoR5MRd5jVrupw.png" /><figcaption>RDS instance running</figcaption></figure><p><strong>Troubleshooting instance IP address:</strong></p><p>EC2 instances were running, but they had no public IP addresses assigned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bqrn-Dx5Hy2c4ICK-5dEZg.png" /></figure><p>I checked the subnet setting and it was missing “<em>Enable auto-assign public IPv4 address</em>”, as I had missed this in my public subnet settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B6CgeC8DwVZKA2p20-Uugw.png" /></figure><p>I added it to my main file for the public subnets, and re-ran terraform plan and apply.</p><pre>map_public_ip_on_launch = true</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/1*81MuiVvX2yDi1w8DT4-wOA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/873/1*FvaT03d17TZ2uWzg9siBbA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GTBJuFGTHTCzK9Yp8DZ3KA.png" /><figcaption>Plan shows the changes to deploy, and subnet setting is enabled now after I applied the plan.</figcaption></figure><p>But my instances still show no public IP address, because they have been deployed already, so I would have to terminate them so new instances will be deployed as ASG has to meet the desired instance capacity (i.e. 2), and they will have public IP addresses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IDTq5PeCIOVHZ9xL4rjmrQ.png" /><figcaption>Terminated instance with no public IP</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FFAfh8ZqgJbU14KkjStztw.png" /><figcaption>New instances created has IP address</figcaption></figure><p>I pasted the instance IP address onto the clipboard and verified that Apache is running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Va2sTrTZGE_084uEEM6U0g.png" /></figure><p>RDS connection was also successful when I tried on MySQL workbench.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wVp5F6-tFPDZEuEk_Wqfaw.png" /></figure><h4><strong>Creating Application Load balancer</strong></h4><p>Now that our deployment is successful, we will add Application Load balancer to our configuration. ALB balances the load between the instances that are registered to the Target group. I also included health check for the instances in my target group.</p><p>Registry link: <a href="https://registry.terraform.io/providers/PixarV/ritt/latest/docs/resources/lb_target_group">https://registry.terraform.io/providers/PixarV/ritt/latest/docs/resources/lb_target_group</a></p><pre>#creating target group<br>resource &quot;aws_lb_target_group&quot; &quot;my_tg&quot; {<br>  name        = &quot;target-group-twotier&quot;<br>  vpc_id      = aws_vpc.two-tier-vpc.id<br>  port        = 80<br>  protocol    = &quot;HTTP&quot;<br>  target_type = &quot;instance&quot;<br><br>  health_check {<br>    interval            = 30<br>    path                = &quot;/&quot;<br>    port                = 80<br>    healthy_threshold   = 5<br>    unhealthy_threshold = 2<br>    timeout             = 3<br>    protocol            = &quot;HTTP&quot;<br>    matcher             = &quot;200,202&quot;<br>  }<br>  lifecycle { <br>      create_before_destroy=true <br>  }<br>}</pre><p>You just defined the target group above, but to register the EC2 instances to the target group, we need to add target group ARN (Amazon Resource name) to the autoscaling group, so that any instances created via autoscaling group gets registered to this target group.</p><p>You will need to have a line of code in ASG like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*F6H0xQY61hl0z7jZbDyxoA.png" /></figure><p>Now creating an application load balancer with its security group.</p><p>(Remember that ALB is bound to receive all the traffic from the internet on Port 80 and you also defined public subnets for ALB as it is internet facing).</p><pre>#Creating Security Group for Application load balancer<br>resource &quot;aws_security_group&quot; &quot;alb-sg&quot; {<br>  name        = &quot;ALB-SG&quot;<br>  description = &quot;Security Group for ALB&quot;<br>  vpc_id      = aws_vpc.two-tier-vpc.id<br>  ingress {<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 80<br>    to_port     = 80<br>    protocol    = &quot;tcp&quot;<br>  }<br>  egress {<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>  }<br>  tags = {<br>    &quot;Name&quot; = &quot;ALB Security Group&quot;<br>  }<br>}<br><br># Create an ALB <br><br>resource &quot;aws_lb&quot; &quot;myalb&quot; {<br>  name               = &quot;my-alb&quot;<br>  internal           = false<br>  load_balancer_type = &quot;application&quot;<br>  security_groups    = [aws_security_group.alb-sg.id]<br>  subnets            = [aws_subnet.public-1a.id, aws_subnet.public-1b.id]<br><br>  tags = {<br>    Name = &quot;My-ALB-twotier&quot;<br>  }<br>}</pre><p>Now, what will ALB do when it receives traffic from the internet? This is defined by listener rules.</p><p>Here we are creating a simple rule that any HTTP traffic received by ALB should be forwarded to our Target group. (Target group has our instances registered).</p><pre>#Create listener rule for ALB<br>resource &quot;aws_lb_listener&quot; &quot;listenerrule-twotier&quot; {<br>  load_balancer_arn = aws_lb.myalb.arn<br>  port              = &quot;80&quot;<br>  protocol          = &quot;HTTP&quot;<br><br>  default_action {<br>    type             = &quot;forward&quot;<br>    target_group_arn = aws_lb_target_group.my_tg.arn<br>  }<br>}</pre><p>Now that ALB is ready and listener rules are set up, along with registered targets, we should be good to deploy this plan.</p><p>Once again, run terraform plan to see the changes made in plan and apply them once you verify.</p><p>I had this mistake again where I missed the subnet ID and Security group ID in ALB :)</p><p>The error is quite self-explanatory; “security group is object with 13 attributes” as I was not referring to the ID of security group. I fixed it again and ran the plan again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vetwmKo1VTfF5DSe65_Vyg.png" /></figure><p>ALB was deployed with listener and rules, to be forwarded to target group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tyfmW-vNUUI697BYmSBQ-w.png" /></figure><p>If you copy the DNS name for ALB, you will see the Apache page running because the ALB is forwarding this request to EC2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ly4tCoi7GpfPpsQHXg9xHg.png" /><figcaption>ALB showing apache page running</figcaption></figure><h4>Outputs.tf (optional)</h4><p>I also created outputs.tf file and was able to output DNS name of my load balancer and endpoint of my RDS database instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/803/1*vne3gcyr-_gccJb48kloGA.png" /></figure><p>After running terraform apply, it will output the values like this in cLI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*cHfICx-z8TUDqqsAdeaUjQ.png" /></figure><h4><strong>Adding Security to webservers:</strong></h4><p>We can add more security for our EC2 instances, as you can see our instances are reachable via internet as of now, we would just want to have them accessed via ALB and they should not accept any other traffic from outside, other than ALB.</p><p>We would need to add a line in EC2 instance security group which points to the ALB’s security group (means EC2 SG will only accept traffic from ALB SG) and comment out the cidr_blocks 0.0.0.0/0 to reject any traffic from the internet.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/887/1*xxJ3xkn_bwRINoccdkOuIg.png" /></figure><p>This is how the EC2 security group looks in the UI now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qBM16xjC9Z9j3kpCOU1Jxw.png" /></figure><p>As expected, my EC2 is not reachable from the internet now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hf2YXIWGtVWYD_xZ-aY_bg.png" /></figure><h4><strong>Push to GitHub:</strong></h4><p>I selected all the files I wanted to push to GitHub. I had already created a new branch before, so I just had to commit these new files (with a message) to remote and merge them into the main branch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/302/1*9L-VZxMpl6LrhsPXnD3mKg.png" /></figure><p>Here is my GitHub link: <a href="https://github.com/hadia-noor16/Terraform/tree/main/two-tier">https://github.com/hadia-noor16/Terraform/tree/main/two-tier</a></p><h4>Summary:</h4><p>This project is powerful because it significantly reduces the time required to create various resources in the UI while minimizing the risk of human error. This is especially beneficial when configuring multi-tier VPCs, which often involve considerable back-and-forth in the console.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d18652ccc882" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Modernizing A Hospital’s Website Infrastructure With DevOps Best Practices.]]></title>
            <link>https://medium.com/@hadianoor16/modernizing-a-hospitals-website-infrastructure-with-devops-best-practices-d20d0f68ec23?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d20d0f68ec23</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Tue, 29 Oct 2024 02:19:17 GMT</pubDate>
            <atom:updated>2024-10-29T02:19:17.515Z</atom:updated>
            <content:encoded><![CDATA[<h4>A guide for current or aspiring Cloud DevOps engineers on establishing an Hospital website infrastructure while leveraging DevOps best practices.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/0*J9dB3jvbXz2Zc83Z.png" /><figcaption>Image created by <a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------">Hadianoor</a></figcaption></figure><h4>Scenario:</h4><p><strong><em>HealthCare North</em></strong> is a regional hospital and they have recently encountered significant challenges when their sole cloud engineer resigned without notice, leaving no documentation for their cloud environment. The hospital’s static website is essential for patient communication and is currently hosted on an expensive traditional cloud server that incurs high costs and demands considerable administrative oversight. In light of these issues, <strong><em>HealthCare North</em></strong> seeks a more cost-effective and low-maintenance cloud solution that aligns with DevOps best practices.</p><h4>Project Overview:</h4><p>The primary goal is to transition <strong><em>HealthCare North’s </em></strong>website from its current costly and high-maintenance cloud server setup to a more economical and efficient AWS service, specifically leveraging a static S3 website. This move aims to reduce costs and administrative overhead. Additionally, the hospital wants to establish sound DevOps practices, including a CI/CD pipeline with proper testing mechanisms, to avoid untested changes going live.</p><h4>Task 1: Initialize Public GitHub Repository for Terraform Code</h4><p><strong>Objectives:</strong></p><ul><li><strong>Create a new GitHub repository.</strong></li><li><strong>Initialize the repository with a README for documentation.</strong></li><li><strong>Add a .gitignore file suitable for Terraform projects.</strong></li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/97cb1595aad7?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Kenneth Barnes Jr.</em></a></blockquote><p>First I created an organization within Github for the<strong> “<em>HealthCare North</em>” </strong>Company so that their Cloud Engineers will be able to take over after the Job is done.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/0*_HUhbpwnYRKr7mKd" /></figure><p>Then for this project I clicked on “Create a free organization” and filled out all the information down below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/791/0*tshy16YH1opubePg" /></figure><p>Once the organization was created I started working on the repository of the “<strong><em>HealthCare North website”</em></strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uAFuDzCMV1xXeGCF" /></figure><p>I also filled in the information for the repo while adding an “README” file for documentation and added a .gitignore specifically <strong><em>(Terraform)</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1013/0*yH6Xh3NjPdkhHFXw" /></figure><h4>Task 2: Establish Branching Strategy for Team Collaboration</h4><p><strong>Objectives</strong>:</p><ul><li>Define the branching model (e.g., Git Flow or GitHub Flow).</li><li>Document the branching strategy in the repository’s README or Wiki.</li><li>Create initial branches according to the strategy (e.g., develop, feature, release branches).</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Hadianoor</em></a></blockquote><p>Once the repository was created, I cloned the repository and selected “Open with GitHub Desktop”.</p><p><strong><em>NOTE: (you must install GitHub Desktop before.)</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y8unpWAvnTBTLHiN" /></figure><p>It will open the folder for you on your computer where you would like to clone it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h2OTxjo7HDSSJexJ" /><figcaption><strong>Give your repository a name and click “Clone”.</strong></figcaption></figure><p>Now create a new branch to make changes in that release or feature branch. I named it “branch strategy”. Click Create.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/0*KtOw132a-KzKkXXV" /></figure><p>Click Create.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/937/0*bXX-jtnyCwwWajpG" /><figcaption><strong>Publish this branch to GitHub.</strong></figcaption></figure><p>Now, open the repository in Visual Studio Code, to start working on your project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NjIZOcIZ8XBcoC0m" /></figure><p>I will first start with defining the branching strategy in the README file of my repository, so all of my team members can know the rules and there is less confusion.</p><p>Effective branch management is crucial for successful collaboration and efficient development with Git. There are four branching strategies; Git-Flow, GitHub-Flow, GitLab-Flow, and Trunk Based Development. Based on all the pros and cons, and ideal use cases, we opted for GitHub-Flow branching strategy based, here is why …</p><blockquote><strong><em>GitHub Flow:</em></strong></blockquote><p>GitHub Flow is a simplified workflow used in GitHub and similar platforms. Development occurs on feature branches, and changes are merged into the main branch (usually named main or master) via pull requests.</p><p>It encourages a fast-paced, review-centric development process, well-suited for smaller teams and frequent deployments. Since merging a feature branch implies it is production-ready,GitHub Flow may require careful attention to automated testing and continuous integration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UvaREb5NgzlMhx06" /></figure><p>Once you save the README file after writing your desired strategy, it will ask to commit the changes to the main branch (in remote).</p><p>Open GitHub Desktop, and see what has changed in the README file. Green lines means the lines that are newly added in the README file. While Red lines show lines to be replaced.</p><p>Always commit the changes to this branch with a message for everyone to get clear understanding of what changes you are making about.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0oaLIniIKIiIzllT" /></figure><p>Once you commit the changes to your feature branch, it asks that if we want to publish the changes to remote (i.e. main branch).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QIAi5uJ_w_fA9R4r" /><figcaption><strong>Click Publish branch.</strong></figcaption></figure><p>After you publish your feature branch to the main branch, it asks you to preview the pull request you would make to the remote. Create the Pull request if everything seems alright to you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2LhrqD1NNL68y7n8" /></figure><p>Once the pull request is created, it routes me to the remote repo, where I would create the PR and wait for it to be approved by the reviewers before I could merge it.</p><p>But I noticed that I have no rule set for requiring approval before merging into the main or any branch deletion protection rules, so I will set them now in the next step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*InPZpmSit1cKd5Oa" /></figure><p>Go to settings under your main repo, on the left hand side under code and automation &gt; then click Branches and click “<strong><em>Add a classic branch protection rule</em></strong>”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*n7H-Y25ykouMTkqO" /></figure><p>It shows that you selected to apply this rule to all the branches.</p><p>Select “<strong><em>Require a pull request before merging</em></strong>”.</p><p>I chose the users to create a PR first before merging into the main, as it’s a safe practice. You can also opt for other security settings, like having approvals from more than one team member or having PR to be approved by someone else other than the person who pushed it. These are the settings I enabled.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*juYJkshpGiDFv721" /><figcaption><strong>Enable these checks for branch protection rules.</strong></figcaption></figure><p>Keep other options default and scroll down to save the changes then the rule will be created.</p><h4>Task 3: Set Up Terraform Remote Backend with S3 and DynamoDB</h4><p><strong>Objectives:</strong></p><ul><li>Create an S3 bucket for Terraform state files.</li><li>Enable versioning on the S3 bucket.</li><li>Set up a DynamoDB table for state locking.</li><li>Configure Terraform backend settings in the Terraform configuration files.</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Hadianoor</em></a></blockquote><p>One of the critical aspects of using Terraform effectively is managing the <strong>Terraform state file</strong> — the single source of truth that records your infrastructure’s current status. It also helps with change detection i.e. When running terraform plan or terraform apply, Terraform checks this state file to understand what has changed.</p><blockquote><strong><em>Terraform offers two main ways to store the state file</em></strong><em>:</em></blockquote><ol><li><strong>Local Backend</strong>: Stores the state file on your local machine or a shared file system.</li><li><strong>Remote Backend</strong>: Stores the state file on a remote service, such as AWS S3, ensuring centralized access and better collaboration.</li></ol><p>The Local backend is simple, but it has few challenges like collaboration issues, risk of data loss, limited system memory, managing backups, and security concerns.</p><p>Remote backend is mostly the better choice with AWS S3 being preferred because of its Reliability,durability, version control, encryption of the objects, Access Control, and Recovery. Using S3 as a backend provides several benefits:</p><ul><li><strong>Shared State Management</strong>: It enables teams to share the Terraform state file, ensuring everyone operates with the same view of the infrastructure.</li><li><strong>State Locking with DynamoDB</strong>: Integrating S3 with DynamoDB allows for state locking, preventing simultaneous state modifications that could lead to conflicts or corruption.</li><li><strong>Versioning</strong>: S3’s versioning capabilities provide an extra layer of security by keeping a history of state changes, which can be invaluable for auditing and rollback.</li></ul><p>I will now go ahead and create S3 bucket and DynamoDB table in UI and set up S3 as my Remote backend with DynamoDB for locking the state via Terraform.</p><blockquote><strong><em>Creating the s3 bucket:</em></strong></blockquote><p>Go to AWS console and search for S3 in the services bar. Make sure you are in <strong><em>us-east-1 region (N. Virginia)</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/972/0*0DBoY9V0y4vf9esX" /><figcaption><strong>Go to S3 and click “Create bucket”. Choose a unique bucket name, keep everything as default but Enable versioning on the bucket.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FVfzzA0KmU5mD1LP" /><figcaption><strong>Click Create bucket.</strong></figcaption></figure><blockquote><strong><em>Creating the DynamoDB table:</em></strong></blockquote><p>Go to DynamoDB service. Create a table for Terraform state locking with a simple hash “<strong><em>LockID</em></strong>” key and one string attribute.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1002/0*Y7NTOU-WsW0NU-gG" /><figcaption><strong>Continue with the rest of the default options and create a table.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*idqmOaJLkh_upzUN" /></figure><p>Now comes the part where we are setting S3 and remote backend along with DynamoDB for state locking.</p><p>I will create a script and will execute it along with any Terraform configuration file that I will create resources from. The state file created as a result, will go to the S3 backend in the folder “<em>global/s3/terraform.tfstate”.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bYtT1nK1dtLhQMcx" /></figure><pre># Now initialize Terraform with the following command<br>terraform init</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5ab38T4edi_BUjaV" /><figcaption><strong>It successfully configured the backend S3</strong></figcaption></figure><pre># Then I ran the following commands<br>terrafrom validate<br>terraform plan -out plan<br>terraform apply -auto-approv</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JW1l6O49AECGM249" /><figcaption><strong>I checked my S3 bucket, to verify if the tfstate file in the bucket.</strong></figcaption></figure><p>Now go to GitHub desktop, and it will show the files that has been changed from main branch (remote), so you can select all those files and create a PR to push that into main.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hlb05gkJZGqQ1cxF" /><figcaption><strong>Select the files you want to push, and then commit with a message.</strong></figcaption></figure><p>It will then ask to Preview Pull request before you request to merge into main. Click that and it will open another window to let you create Pull request.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1OKrzVoSrORKHBTA" /></figure><p>It takes you to the remote repo, and you can add a reviewer to approve your PR.<br><strong><em>NOTE: (Notice that you cannot choose yourself to review or approve according to the branch protection rule that I set).</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bYf4Q2x49ZPQ8_X8" /><figcaption><strong>Click “Create pull request”</strong></figcaption></figure><p>Now once PR is opened, it is waiting to be reviewed by a team member and merging is blocked.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y7tYNnDOYdQQ1R09" /><figcaption><strong>Now wait until the “pull request” is approved</strong>.</figcaption></figure><p>Notice that I can “merge without waiting for requirements to be met (bypass protection rules)”, to avoid this, I can go to settings again and enable this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EAidJqaiU0TJ7OEt" /><figcaption><strong>After enabling this setting, I cannot bypass branch protection settings anymore.</strong></figcaption></figure><h4>Task 4 &amp; 5: Terraform Configuration for S3 Bucket (Development Environment) &amp; (Production Environment)</h4><p><strong>Objectives: <em>(Development Environment)</em></strong></p><ul><li>Write Terraform code to define an S3 bucket.</li><li>Configure the S3 bucket for website hosting.</li><li>Set appropriate permissions and policies on the S3 bucket.</li></ul><p><strong>Objectives: <em>(Production Environment)</em></strong></p><ul><li>Write Terraform code for the production S3 bucket.</li><li>Ensure the configuration matches the development environment.</li><li>Apply Terraform code to create the bucket.</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/97cb1595aad7?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Kenneth Barnes Jr.</em></a></blockquote><p><strong>Once the terraform remote backend was configured, I started the process of creating the “<em>Development</em>” s3 bucket and along with the deployment I leveraged the dev bucket to host the static website of the HealthCare North company. I also deploy the “<em>Production</em>” s3 bucket within the same terraform main.tf file and I made it so that it has the same configuration as the dev bucket so that they can mirror each other.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/212/0*_P7bHs2GlLXH-mrY" /><figcaption><strong>In the image I utilized the main.tf to hold the bulk of the info and the other files such as output.tf, provider.tf, and variable.tf and the module “website” has data/info that the main.tf calls for to create the Dev and Prod buckets and establish a static website in both buckets.</strong></figcaption></figure><p><a href="https://gist.githubusercontent.com/Kenneth-Barnes-Jr/2868ab76594022ddc5c237613d1599c7/raw/2bda34ae64621f48a95039b09015ee1f4f44bb05/gist-hcn-main.tf">https://gist.githubusercontent.com/Kenneth-Barnes-Jr/2868ab76594022ddc5c237613d1599c7/raw/2bda34ae64621f48a95039b09015ee1f4f44bb05/gist-hcn-main.tf</a></p><pre># Module to manage the static website files.<br><br>module &quot;website_files&quot; {<br>  source = &quot;./website&quot;<br>}<br><br># The s3 bucket for the (dev) environment.<br>resource &quot;aws_s3_bucket&quot; &quot;dev_s3&quot; {<br>  bucket = var.dev_bucket<br>}<br><br># The policy for the (dev) s3 buket to allow public read access.<br><br>resource &quot;aws_s3_bucket_policy&quot; &quot;dev_s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.dev_s3_public_access_block]  <br>  bucket = aws_s3_bucket.dev_s3.id<br><br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : &quot;arn:aws:s3:::${var.dev_bucket}/*&quot;<br>      }<br>    ]<br>  })<br>}<br><br># Configuring the (dev) s3 bucket as a static website.<br><br>resource &quot;aws_s3_bucket_website_configuration&quot; &quot;dev_s3_website_configuration&quot; {<br>  bucket = aws_s3_bucket.dev_s3.id<br><br>  index_document {<br>    suffix = &quot;index.html&quot;<br>  }<br><br>  error_document {<br>    key = &quot;error.html&quot;<br>  }<br>}<br><br># This block is uploading the files to the (dev) s3 bucket.<br><br>resource &quot;aws_s3_object&quot; &quot;dev_s3_files&quot; {<br>  bucket = aws_s3_bucket.dev_s3.id<br><br>  for_each = {<br>    &quot;index.html&quot; = {<br>      key          = &quot;index.html&quot;<br>      source_path  = &quot;website/index.html&quot;<br>      content_type = &quot;text/html&quot;<br>    }<br>    &quot;error.html&quot; = {<br>      key          = &quot;error.html&quot;<br>      source_path  = &quot;website/error.html&quot;<br>      content_type = &quot;text/html&quot;<br>    }<br>  }<br>  key          = each.key<br>  source       = each.value.source_path<br>  content_type = each.value.content_type<br>}<br>  <br># Configuring public access settings for the (dev) s3 bucket<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;dev_s3_public_access_block&quot;{<br>  bucket = aws_s3_bucket.dev_s3.id<br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}<br><br># s3 bucket for the (prod) environment  <br>  <br>resource &quot;aws_s3_bucket&quot; &quot;pro_s3&quot; {<br>  bucket = var.pro_bucket<br>}<br><br># Policy for the (prod) s3 bucket to allow public read access<br><br>resource &quot;aws_s3_bucket_policy&quot; &quot;pro_s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.pro_s3_public_access_block]<br>  bucket = aws_s3_bucket.pro_s3.id<br><br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : &quot;arn:aws:s3:::${var.pro_bucket}/*&quot;<br>      }<br>    ]<br>  })<br>}<br><br># Configuring the (prod) s3 bucket as a static website<br><br>resource &quot;aws_s3_bucket_website_configuration&quot; &quot;pro_s3_website_configuration&quot; {<br>  bucket = aws_s3_bucket.pro_s3.id<br><br>  index_document {<br>    suffix = &quot;index.html&quot;<br>  }<br><br>  error_document {<br>    key = &quot;error.html&quot;<br>  }<br>}<br><br># This block is uploading the files to the (prod) s3 bucket.<br><br>resource &quot;aws_s3_object&quot; &quot;pro_s3_files&quot; {<br>  bucket = aws_s3_bucket.pro_s3.id<br><br>  for_each = {<br>    &quot;index.html&quot; = {<br>      key          = &quot;index.html&quot;<br>      source_path  = &quot;website/index.html&quot;<br>      content_type = &quot;text/html&quot;<br>    }<br>    &quot;error.html&quot; = {<br>      key          = &quot;error.html&quot;<br>      source_path  = &quot;website/error.html&quot;<br>      content_type = &quot;text/html&quot;<br>    }<br>  }<br>  key          = each.key<br>  source       = each.value.source_path<br>  content_type = each.value.content_type<br>}<br><br># Configuring public access settings for the (prod) s3 bucket<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;pro_s3_public_access_block&quot;{<br>  bucket = aws_s3_bucket.pro_s3.id<br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}</pre><pre># This is the output for the URL of the (dev) static website.<br><br>output &quot;dev_website_url&quot; {<br>  description = &quot;URL of the dev website&quot;<br>  value       = aws_s3_bucket_website_configuration.dev_s3_website_configuration.website_endpoint<br>}<br><br># This is the output for the URL of the (prod) static website.<br><br>output &quot;prod_website_url&quot; {<br>  description = &quot;URL of the pro website&quot;<br>  value       = aws_s3_bucket_website_configuration.pro_s3_website_configuration.website_endpoint<br>}<br><br># The output for the path to the index.html file within the module.<br><br>output &quot;index_path&quot; {<br>  value = &quot;${path.module}/index.html&quot;<br>}<br><br># The output for the path to the error.html file within the module.<br><br>output &quot;error_path&quot; {<br>  value = &quot;${path.module}/error.html&quot;<br>}</pre><pre># Specifying required providers and the terraform versions. <br><br>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 4.0&quot; # A version constraint, that will use any version in the 4.x range. <br>    }<br>  }<br>}<br><br>provider &quot;aws&quot; {<br>  region  = var.aws_region<br>  profile = &quot;&lt;iam-user-name&gt;&quot;<br>}</pre><pre>variable &quot;dev_bucket&quot; {<br>  description = &quot;This is the name of the development bucket for HealthCare North&quot;<br>  type        = string<br>  default     = &quot;&lt;hcn-dev-bucket-name&gt;&quot;<br>}<br><br>variable &quot;pro_bucket&quot; {<br>  description = &quot;This is the name of the production bucket for HealthCare North&quot;<br>  type        = string<br>  default     = &quot;&lt;hcn-prod-bucket-name&gt;&quot;<br>}<br><br>variable &quot;aws_region&quot; {<br>  description = &quot;This is the main region where the resources for HealthCare North will be deployed&quot;<br>  type        = string<br>  default     = &quot;&lt;default-aws-region&gt;&quot;<br>}</pre><pre># I ran the following commands once I configured all the files I used aboved.<br><br>terraform init<br><br>terraform plan<br><br>terraform apply -auto-approve</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RhFKelrcfRh_xA1h" /><figcaption><strong>The dev and prod buckets are deployed within the us-east region</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*YuIfehWmZQyT5I08" /><figcaption><strong>I checked to see if static website hosting is enable and now it’s time to test if the static web page is configured correctly</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*w1X3mPU6QtnBdW-s" /><figcaption><strong>As you can see I’m able to access the static website of the bucket and you can verify by looking at the URL</strong></figcaption></figure><h3>Task 6: Terraform Configuration for CloudFront Distribution with Regional Restrictions.</h3><p><strong>Objectives:</strong></p><ul><li>Define a CloudFront distribution in Terraform.</li><li>Configure the distribution to serve the S3 website.</li><li>Set up regional restrictions on the CloudFront distribution to North America</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/97cb1595aad7?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Kenneth Barnes Jr.</em></a></blockquote><p>I added this CloudFront section to the main.tf file.</p><pre>resource &quot;aws_cloudfront_distribution&quot; &quot;s3_distribution&quot; {<br># This will define the origin which is the s3 bucket<br>origin {<br>    domain_name = &quot;${aws_s3_bucket.pro_s3.bucket_regional_domain_name}&quot;<br>    origin_id   = &quot;my_first_origin&quot;<br>}<br><br># Enable the distribution and configure basic settings<br>enabled             = true<br>  is_ipv6_enabled     = true<br>  comment             = &quot;CDN for HealthCare North&quot;<br>  default_root_object = &quot;index.html&quot;<br><br># Default cache behavior settings<br>default_cache_behavior {<br>    allowed_methods  = [&quot;DELETE&quot;, &quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;, &quot;PATCH&quot;, &quot;POST&quot;, &quot;PUT&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id = &quot;my_first_origin&quot;<br><br># The settings for forwarding requests<br>forwarded_values {<br>      query_string = false<br>cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }<br>viewer_protocol_policy = &quot;allow-all&quot;<br>    min_ttl                = 0<br>    default_ttl            = 3600<br>    max_ttl                = 86400<br>  }<br>  <br># Cache behavior with precedence 0<br>  ordered_cache_behavior {<br>    path_pattern     = &quot;/content/immutable/*&quot;<br>    allowed_methods  = [&quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;]<br>    target_origin_id = &quot;my_first_origin&quot;<br>forwarded_values {<br>      query_string = false<br>      headers      = [&quot;Origin&quot;]<br>cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }<br>min_ttl                = 0<br>    default_ttl            = 86400<br>    max_ttl                = 31536000<br>    compress               = true<br>    viewer_protocol_policy = &quot;allow-all&quot;<br>  }<br>  <br># Cache behavior with precedence 1<br>  ordered_cache_behavior {<br>    path_pattern     = &quot;/content/*&quot;<br>    allowed_methods  = [&quot;GET&quot;, &quot;HEAD&quot;, &quot;OPTIONS&quot;]<br>    cached_methods   = [&quot;GET&quot;, &quot;HEAD&quot;]<br>    target_origin_id  = &quot;my_first_origin&quot;<br>forwarded_values {<br>      query_string = false<br>cookies {<br>        forward = &quot;none&quot;<br>      }<br>    }<br>min_ttl                = 0<br>    default_ttl            = 3600<br>    max_ttl                = 86400<br>    compress             = true<br>    viewer_protocol_policy = &quot;redirect-to-https&quot;<br>  }<br><br># Pricing class for the distribution <br>price_class = &quot;PriceClass_200&quot;<br>restrictions {<br>    geo_restriction {<br>      restriction_type = &quot;whitelist&quot;<br>      locations        = [&quot;US&quot;]<br>    }<br>  }<br>tags = {<br>    Environment = &quot;production&quot;<br>  }<br><br># SSL/TLS cerificate settings<br><br>viewer_certificate {<br>    cloudfront_default_certificate = true<br>  }<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/0*WQvCaytVhCGsgQ0G" /><figcaption><strong>When I added the CloudFront section to the main.tf file and did a terraform plan command, it updated the infrastructure configuration of the HealthCare North company. In the image above is when I deployed the CloudFront resource it took about 4 mins. (NOTE: This resource takes some time to deploy as you can see in the image above.)</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6ZbBp41mQoB8ItCI" /><figcaption><strong>Once CloudFront is deployed, back in the console I verified that HealthCare North static web page is by clicking on “Distribution domain name”.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5ZUsKpVOMQsVubdn" /><figcaption><strong>Looking at the home page and the URL seems like it’s working properly.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J4fxtuUZ53AgXgm3" /><figcaption><strong>I also set the regional restrictions to North American.</strong></figcaption></figure><h3>Task 7: Set Up CI/CD Pipeline with GitHub Using Terraform</h3><p><strong>Objectives:</strong></p><ul><li>Define CI/CD pipeline structure in Terraform</li><li>Connect GitHub as the source repository.</li><li>Configure triggers for the pipeline.</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Hadianoor</em></a></blockquote><p>AWS Code Pipeline is a continuous integration and continuous delivery service for fast and reliable applications and infrastructure updates. You can use CodePipeline to fully model and automate your software release processes.</p><p>A pipeline defines your release process workflow and describes how a new code change progresses through your release process. A pipeline comprises a series of stages (e.g., build, test, and deploy), which act as logical divisions in your workflow. Each stage is made up of a sequence of actions, which are tasks such as building code or deploying to test environments. AWS CodePipeline provides you with a graphical user interface to create, configure, and manage your pipeline and its various stages and actions, allowing you to easily visualize and model your release process workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OteDdRnTXDumKyAz" /></figure><p>In this project, we will integrate AWS CodePipeline with GitHub repository as my source code resides in GitHub. Every time a change is made in source code, AWS CodePipeline is triggered, and it deploys the changes in the AWS S3 dev bucket that I am using to host my static website. Once those changes are manually approved by reviewers, (we will add another step in code pipeline later) that will deploy the final changes to the CloudFront for our customers.</p><p>Let’s jump into the prerequisites and get started with the steps to create our Code Pipeline.</p><blockquote><strong><em>Prerequisites</em></strong><em>:</em></blockquote><p>First you need a service role for Code Pipeline. A service role is an <strong>IAM role</strong> that a service assumes to perform actions on your behalf. The pipeline is always associated with the service role. You can follow this document to create one:</p><p><a href="https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-create-service-role-console.html?source=post_page-----b6e1f7e9391e--------------------------------">https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-create-service-role-console.html?source=post_page-----b6e1f7e9391e--------------------------------</a></p><p>This is how CodeBuild service role looks like for me</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Action&quot;: [<br>                &quot;s3:*&quot;,<br>                &quot;cloudfront:CreateInvalidation&quot;,<br>                &quot;codebuild:StartBuild&quot;,<br>                &quot;codebuild:BatchGetBuilds&quot;,<br>                &quot;logs:CreateLogGroup&quot;,<br>                &quot;logs:CreateLogStream&quot;,<br>                &quot;logs:PutLogEvents&quot;,<br>                &quot;ssm:CreateAssociation&quot;,<br>                &quot;ssm:CreateAssociationBatch&quot;,<br>                &quot;ssm:UpdateAssociation&quot;,<br>                &quot;ssm:UpdateAssociationStatus&quot;,<br>                &quot;codecommit:GetBranch&quot;,<br>                &quot;codecommit:GetCommit&quot;,<br>                &quot;codecommit:UploadArchive&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;<br>        }<br>    ]<br>}</pre><blockquote><strong><em>CodePipeline service role:</em></strong></blockquote><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Action&quot;: [<br>                &quot;iam:PassRole&quot;,<br>                &quot;codebuild:*&quot;,<br>                &quot;s3:*&quot;,<br>                &quot;codestar:*&quot;,<br>                &quot;codestar-connections:*&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;<br>        }<br>    ]<br>}</pre><p>In order to connect your GitHub repository to the pipeline, you need to create a connector that authenticates the GitHub repository. Here is how you can do that in the console.</p><p>In AWS services, go to CodePipeline. On the left hand side, looks for settings, expand it and click Connections.</p><p>Create a connection, give it a name. Choose GitHub as the provider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*B2mO7tbWbCOsHHtu" /><figcaption><strong>Click “Connect to GitHub”.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/0*v6nIB63dwY71PKAf" /><figcaption><strong>On the next page, click “Install a new app”.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/0*P_77EVk90ekOGnzE" /><figcaption><strong>Then it will take you to the GitHub repositories you want to connect to.</strong></figcaption></figure><p>When you click your repo, it will ask for password confirmation, and once it’s provided you will be taken to your repo page with all the permissions you want to give to your pipeline to interact to this repository/repositories.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ls-VMG6ccRdbS0z4" /><figcaption><strong>Click save.</strong></figcaption></figure><p>The connection is now made with an “arn” that you will be using in your TF code later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mLn2EB8V62gQzg2m" /></figure><p>Now we need to write Terraform code, so in order to do that I simply cloned my repository in my computer, its just showing “README” and .gitignore file as of now.</p><pre>git clone &lt;your repo url&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*QE44nxmmCxS8jqZw.png" /></figure><p><strong>Let’s get ready to create our pipeline.</strong></p><blockquote><strong><em>Create main.tf file:</em></strong></blockquote><p>First define the provider block with region you are working in.</p><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 5.56&quot;<br>    }<br>  }<br>}</pre><pre>provider &quot;aws&quot; {<br>    region = &quot;us-east-1&quot;<br>}</pre><p>Now if you remember that we created a bucket that we set as a remote in the beginning, this is where our state file goes in.</p><p>We need another bucket for CI/CD artifacts to go in. I am compiling the code altogether.</p><pre>terraform{<br>    backend &quot;s3&quot; {<br>        <br>        bucket = &quot;s3-remote-hcnorth&quot;<br>        encrypt = true<br>        key = &quot;.tfstate&quot;<br>        region = &quot;us-east-1&quot;<br>    }<br>}</pre><pre>resource &quot;aws_s3_bucket&quot; &quot;artifacts&quot; {<br>  bucket = &quot;hcn-artifacts-pod4&quot;<br>}<br>resource &quot;aws_s3_bucket_ownership_controls&quot; &quot;owner&quot; {<br>  bucket = aws_s3_bucket.artifacts.id<br>  rule {<br>    object_ownership = &quot;BucketOwnerPreferred&quot;<br>  }<br>}<br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;acl&quot; {<br>  bucket = aws_s3_bucket.artifacts.id<br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}</pre><pre>resource &quot;aws_s3_bucket_policy&quot; &quot;s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.acl]<br>  bucket = aws_s3_bucket.artifacts.id<br> <br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : &quot;arn:aws:s3:::hcn-artifacts-pod4/*&quot;<br>      }<br>    ]<br>  })<br>}</pre><p>I created another bucket with name “hcn-artifacts-pod4”, I gave public access to this bucket. This bucket is where all our CI/CD pipeline processes will go.</p><p>At this stage, run “Terraform init” and “Terraform validate” to check if we have no issue in our configuration file till this point.</p><p>I also did “terraform plan” and “Terrafom apply” and my artifacts bucket was created. But its empty as we have’t created the pipeline yet to store the artifact files in the bucket.</p><h4>Task 8: Integrate Build Phase Using AWS CodeBuild in CodePipeline</h4><p><strong>Objectives:</strong></p><ul><li>Define a build project in AWS CodeBuild.</li><li>Configure build specifications and environment.</li><li>Integrate CodeBuild project into the CI/CD pipeline.</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Hadianoor</em></a></blockquote><p>AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. You don’t need to provision, manage, and scale your own build servers. It automates the process of building and testing code every time a change is made, facilitating a more efficient and reliable development workflow.</p><pre>resource &quot;aws_codebuild_project&quot; &quot;project&quot; {<br>  name          = &quot;my-codebuild-project&quot;<br>  description   = &quot;Build project for my website&quot;<br>  service_role  = &quot;arn:aws:iam::060795916438:role/CodeBuildServiceRole&quot;<br>  source {<br>    type      = &quot;GITHUB&quot;<br>    location  = &quot;https://github.com/Client-HealthCare-North/Website&quot;<br>    buildspec = &quot;buildspec.yml&quot;  <br>  }<br>  artifacts {<br>    type = &quot;S3&quot;<br>    location = aws_s3_bucket.artifacts.id<br>  }<br>  environment {<br>    compute_type                = &quot;BUILD_GENERAL1_SMALL&quot;<br>    image                       = &quot;aws/codebuild/amazonlinux2-x86_64-standard:4.0&quot;<br>    type                        = &quot;LINUX_CONTAINER&quot;<br>    image_pull_credentials_type = &quot;CODEBUILD&quot;<br> }</pre><blockquote><strong><em>service_role</em></strong><em>: Provide service role arn for CodeBuild.</em></blockquote><p>CodeBuild is taking my code from GitHub repo and deploying to S3 bucket, but it needs a server to run this configuration right? This is where “environment” block comes in.<br>As you can see we defined a few parameters for our server like comoute type, image, type, etc in our env block above.</p><p>I used same config as defined in Terraform registry of “aws_codebuild_project”. (Very useful)</p><p><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project</a></p><p>Next I need to create a buildspec.yml file that I mentioned in source block, which means my repository should have this file uploaded.</p><p>Create buildspec.yml file in the VS Code (the repo you cloned earlier) and paste this code there.</p><pre>version: 0.2<br>phases:<br>  build:<br>    commands:<br>    - echo &quot;This is build phase&quot;<br> <br>  post_build:<br>    commands:<br>    - echo &quot;Deployed to S3&quot;<br>artifacts:<br>  files:<br>    - &#39;**/*&#39;</pre><p><strong>What is this buildspec.yml file doing?</strong></p><p>I have nothing fancy in my pre-build and post-build phase, as I am simply deploying static webpages to the dev bucket. So, I can use echo messages to print on the console just to show what stage we are in, and if they are successfully deployed.</p><ul><li>Defines a build process with a build phase and a post-build phase.</li><li>Outputs messages indicating the start of the build process and that deployment to S3 has occurred.</li><li>Collects all files generated during the build process to be stored as artifacts, ready for deployment (often handled by the subsequent steps in a CodePipeline).</li></ul><p>Now notice that you mentioned all the files of the current directory to be collected in build process, so you must have your website pages in your repository too. This is how my repository looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*i-loR9NEqlv6dWJ1.png" /></figure><p>That’s all about build project.. Now we told build project where to collect the files from in our repo, but where are we gonna say “where to deploy” ?<br>The next step shows to create a Code Pipeline that will include “Deploy” stage to achieve this task.</p><h4>Create Code Pipeline</h4><p>A structured and well-thought-out delivery process doesn’t just include a single build job. It’s an orchestration of many jobs, often idempotent jobs, that enable you to roll out deployment packages and infrastructure in a reliable and safe manner.</p><p>And that’s exactly what AWS CodePipeline does: it’s your CodeBuild orchestration tool. With CodePipeline, you can create pipelines that connect jobs into stages to create a multi-step rollout process that can be easily reproduced and understood.</p><p>The below piece of code will finally create a Code Pipeline for us.</p><pre>resource &quot;aws_codepipeline&quot; &quot;pipeline&quot; {<br>  name     = &quot;my-pipeline&quot;<br>  role_arn = &quot;arn:aws:iam::060795916438:role/CodePipelineServiceRole&quot;<br>artifact_store {<br>    location = aws_s3_bucket.artifacts.id<br>    type     = &quot;S3&quot;<br>  }<br>  stage {<br>    name = &quot;Source&quot;<br>    action {<br>      name             = &quot;Source&quot;<br>      category         = &quot;Source&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;CodeStarSourceConnection&quot;<br>      version          = &quot;1&quot;<br>      output_artifacts = [&quot;source_output&quot;]<br>      configuration = {<br>        ConnectionArn = var.github_connector  # create variable and save the connector name in .tfvars file<br>        FullRepositoryId = &quot;Client-HealthCare-North/Website&quot;  # Update with your GitHub repo name<br>        BranchName = &quot;main&quot;  # Specify the branch<br>      }<br>    }<br>  }<br>  stage {<br>    name = &quot;Build&quot;<br>    action {<br>      name             = &quot;Build&quot;<br>      category         = &quot;Build&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;CodeBuild&quot;<br>      version          = &quot;1&quot;<br>      input_artifacts  = [&quot;source_output&quot;]<br>      output_artifacts = [&quot;build_output&quot;]<br>      configuration = {<br>        ProjectName = aws_codebuild_project.project.name<br>      }<br>    }<br>  }<br>  stage {<br>    name = &quot;Deploy&quot;<br>    action {<br>      name             = &quot;Deploy&quot;<br>      category         = &quot;Deploy&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;S3&quot;<br>      version          = &quot;1&quot;<br>      input_artifacts  = [&quot;build_output&quot;]<br>      configuration = {<br>        BucketName = &quot;hcn-dev-bucket&quot;  # my dev bucket<br>        Extract    = &quot;true&quot;<br>      }<br>    }<br>  }<br>}</pre><blockquote><strong>role_arn</strong>: Provide service role for CodePipeline.</blockquote><blockquote><strong>artifact_store</strong> is storing all the CI/CD files created from this step in S3 bucket defined.</blockquote><blockquote><strong>FullRepositoryID</strong>: Your GitHub repo url</blockquote><blockquote><strong>Branch</strong>: Where you want to take the cource code from</blockquote><p>Now comes the most important part i.e.<strong> Source. </strong>Here your GitHub connection that you created earlier will come handy.</p><p>For ConnectionArn, notice that I defined a variable (so create a variables.tf file and define variable <strong>github_connector.</strong> I don’t want to keep the connector information in variables.tf file as it could be deployed to GitHub to public, so I am also creating terraform.tfvars file with then actual value of my GitHub connection arn that you created earlier), like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*pFPoLlOm30jmIbEK.png" /><figcaption><strong>(Alot of things can go on variables.tf file, like the service role arn for CodePipeline and CodeBuild, region, etc).</strong></figcaption></figure><p>The Deploy stage is fairly simple, it just mentions the bucket name your code will be deployed to.</p><p>Also, I kept my dev bucket empty to see the changes deployed.</p><p>Now we have everything ready, we should go ahead and run Terraform commands.</p><pre>Terraform plan<br>Terraform apply</pre><p>It will show the plan to create CodeBuild project and CodePipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*WyuVDikkMsJuuexj.png" /><figcaption><strong>I had used the wrong role before, fixed it later</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*5XVDtAVuIdoZ8Wgv.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*KfoZNUqvimL2sGJt.png" /></figure><p>The build and deploy stages are also successful, which is a good news.</p><p>We will head to S3 console to see if the dev bucket is populated with the source code our Git repo had. Yes it does.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*SaFr4Y7CNOf3oR0Y.png" /></figure><p>Now copy the bucket website endpoint in the browser to see if its showing the website page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*CDl38K7s1HToHgyu.png" /></figure><h4>Task 9: Testing the Pipeline by Making Small Changes to the HTML</h4><p><strong>Objectives:</strong></p><ul><li>Make a minor change to the index.html file.</li><li>Push the change through the CI/CD pipeline.</li><li>Verify the change in the development and production environments.</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Hadianoor</em></a></blockquote><p>We are checking if the Pipeline triggers when a change is made to index.html file in the GitHub repository.</p><p>In my VS Code, I will open index.html page in the editor, and make some changes. As the changes are detected, it will appear in the GitHub desktop to push those changes with commit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*TtcMYuy4vroAaeoP.png" /><figcaption><strong>Create Pull request and it will take you to remote repo where you can create PR. Wait for the approval to be able to merge.</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*U6wbwBmwx6n3eBhX.png" /></figure><p>When the index.html file is merged to the main branch, the pipeline will trigger. Check in the console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*A6qSl0njp7C_DzHs.png" /></figure><p>As the deploy stage is successful, refresh the webpage to see the changes you implemented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*9ISrTnz1kvbk1jW0.png" /></figure><p>These are the screenshots for my Remote backend bucket and artifact buckets and they are successfully storing the files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0hDmQ2dIi1wyOGZM.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4uI1DbthJmVujso6.png" /></figure><p>We are all good.</p><h4>Task 10: Manual Approval Process for Production Deployment</h4><p><strong>Objectives:</strong></p><ul><li>Configure a manual approval step in the CI/CD pipeline.</li><li>Set up notifications for approval requests.</li><li>Document the approval process for team members.</li></ul><blockquote><strong><em>contributor</em></strong><em> | </em><a href="https://medium.com/u/588e4115da9f?source=post_page---user_mention--b6e1f7e9391e--------------------------------"><em>Hadianoor</em></a></blockquote><p>For security reasons, we didn’t deploy the changes directly to Prod bucket as this is where are customers are being served from (together with CloudFront).</p><p>We will now include a “manual approval” stage to our pipeline which will indicate that changes will be deployed to Prod bucket, only if they are approved by the reviewers.</p><p>Go to CodePipeline. Edit the pipeline, scroll down and “Add Stage” after the Deploy stage like this. I named it “maunal approval”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*hvP9xkmx5177TySS.png" /></figure><p>Once the stage is created, click ‘Add Action Group’ and select ‘Manual Approval’ for Action Provider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*2Qqf_fjuscyvfHOv.png" /></figure><p>URL for review is optional but it’s good practice to review the Dev bucket once before we give go-ahead to deploy those changes in Prod. Hence, I used the Dev bucket URL. Save it.</p><p>Now Click “Add Action Group” again to specify where to deploy changes after approval. Notice the destination is Prod bucket now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*xHVG8acU9ntwwNTp.png" /></figure><p>Changing the index.html file again to trigger the pipeline</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*zC93R0On8IRpb5NP.png" /></figure><p>Index page shows changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*IghhGPkVzMhDfC0n.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*rANu2Zuv3h2hWWss.png" /><figcaption><strong>Create PR</strong></figcaption></figure><p>Once PR merged to main, Pipeline is triggered and showed updated webpage in dev bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Mk_QzoSPwh08MKS_.png" /></figure><p>While its still waiting for manual approval to be deployed to prod bucket.</p><p>Click Review and Approve.</p><p>(Note: You can create SNS topic and subscribe users to them so they can approve and receive notifications via email/sms).</p><p>I am just using the Pipeline console to review and approve the code myself.</p><p>Click Review and select “Approve” with a message. Click Submit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*NwtCKVv6KD0pWW5H.png" /></figure><p>Once it’s submitted it triggers the pipeline and deployment to prod bucket begins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*HDwDISPpD07nT4JH.png" /></figure><p>Now go to Prod bucket website endpoint and see that the changes are deployed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Hlhb4hdbpB42kccz.png" /><figcaption><strong>Success!</strong></figcaption></figure><p>One last thing to check, since our prod bucket is serving content to CF, we will check if the changes are reflected in CF domain name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*cazo2LMYLLhsTlMu.png" /><figcaption><strong>Success!</strong></figcaption></figure><h4>Summary:</h4><p>By modernizing <strong><em>HealthCare North’s </em></strong>website infrastructure and adopting sound DevOps practices, the hospital will not only reduce operational costs but also enhance the reliability and security of its critical patient communication channel. This transition also positions <strong><em>HealthCare North</em></strong> to better manage and scale its digital assets in the future.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d20d0f68ec23" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating CI/CD Pipeline via Terraform:]]></title>
            <link>https://medium.com/@hadianoor16/creating-ci-cd-pipeline-via-terraform-df1652d065d3?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/df1652d065d3</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Sat, 26 Oct 2024 23:21:50 GMT</pubDate>
            <atom:updated>2024-10-26T23:21:50.082Z</atom:updated>
            <content:encoded><![CDATA[<p>AWS CodePipeline is a continuous integration and continuous delivery service for fast and reliable applications and infrastructure updates. You can use CodePipeline to fully model and automate your software release processes.</p><p>A pipeline defines your release process workflow and describes how a new code change progresses through your release process. A pipeline comprises a series of stages (e.g., build, test, and deploy), which act as logical divisions in your workflow. Each stage is made up of a sequence of actions, which are tasks such as building code or deploying to test environments. AWS CodePipeline provides you with a graphical user interface to create, configure, and manage your pipeline and its various stages and actions, allowing you to easily visualize and model your release process workflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ESUbXOgLZA9vUs8S" /></figure><p>In this project, we will integrate AWS CodePipeline with GitHub repository as my source code resides in GitHub. Every time a change is made in source code, AWS CodePipeline is triggered, and it deploys the changes in the AWS S3 dev bucket that I am using to host my static website. Once those changes are manually approved by reviewers, (we will add another step in code pipeline later) that will deploy the final changes to the CloudFront for our customers.</p><p>Let’s jump into the prerequisites and get started with the steps to create our CodePipeline.</p><p><strong>Prerequisites:</strong></p><ol><li>First you need a service role for CodePipeline and CodeBuild. A service role is an <strong>IAM role</strong> that a service assumes to perform actions on your behalf. The pipeline is always associated with the service role. You can follow this document to create one:</li></ol><p><a href="https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-create-service-role-console.html">Create the CodePipeline service role (console)</a></p><p>This is how CodeBuild service role looks like for me</p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Action&quot;: [<br>                &quot;s3:*&quot;,<br>                &quot;cloudfront:CreateInvalidation&quot;,<br>                &quot;codebuild:StartBuild&quot;,<br>                &quot;codebuild:BatchGetBuilds&quot;,<br>                &quot;logs:CreateLogGroup&quot;,<br>                &quot;logs:CreateLogStream&quot;,<br>                &quot;logs:PutLogEvents&quot;,<br>                &quot;ssm:CreateAssociation&quot;,<br>                &quot;ssm:CreateAssociationBatch&quot;,<br>                &quot;ssm:UpdateAssociation&quot;,<br>                &quot;ssm:UpdateAssociationStatus&quot;,<br>                &quot;codecommit:GetBranch&quot;,<br>                &quot;codecommit:GetCommit&quot;,<br>                &quot;codecommit:UploadArchive&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;<br>        }<br>    ]<br>}</pre><p><strong>CodePipeline service role:</strong></p><pre>{<br>    &quot;Version&quot;: &quot;2012-10-17&quot;,<br>    &quot;Statement&quot;: [<br>        {<br>            &quot;Action&quot;: [<br>                &quot;iam:PassRole&quot;,<br>                &quot;codebuild:*&quot;,<br>                &quot;s3:*&quot;,<br>                &quot;codestar:*&quot;,<br>                &quot;codestar-connections:*&quot;<br>            ],<br>            &quot;Resource&quot;: &quot;*&quot;,<br>            &quot;Effect&quot;: &quot;Allow&quot;<br>        }<br>    ]<br>}</pre><p>2. In order to connect your GitHub repository to the pipeline, you need to create a connector that authenticates the GitHub repository. Here is how you can do that in the console.</p><p>In AWS services, go to CodePipeline. On the left hand side, looks for settings, expand it and click Connections.</p><p>Create a connection, give it a name. Choose GitHub as the provider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pQWtddeekmrLyZxO" /></figure><p>Click “Connect to GitHub”.</p><p>On the next page, click “Install a new app”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/0*FSGiS35MoxNn-tc8" /></figure><p>It will take you to the GitHub repositories you want to connect to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/0*xBKnqaDDNKBPtfiB" /></figure><p>When you click your repo, it will ask for password confirmation, and once it’s provided you will be taken to your repo page with all the permissions you want to give to your pipeline to interact to this repository/repositories.</p><p>Click save.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*p1j1iY8nGnmP0qjK" /></figure><p>The connection is now made with an ‘arn’ that you will be using in your TF code later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fHu5QN5Zn9c41iDr" /></figure><p>Now we need to write Terraform code, so in order to do that I simply cloned my repository in my computer, its just showing “README” and .gitignore file as of now.</p><pre>git clone &lt;your repo url&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*579aLII90PKvkPTHnsE8MQ.png" /></figure><p><strong>Let’s get ready to create our pipeline.</strong></p><h4>Create main.tf file:</h4><p>First define the provider block with region you are wroking in.</p><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 5.56&quot;<br>    }<br>  }<br>}<br><br>provider &quot;aws&quot; {<br>    region = &quot;us-east-1&quot;<br>}</pre><p>Now if you remember that we created a bucket that we set as a remote in the beginning, this is where our state file goes in.</p><p>We need another bucket for CI/CD artifacts to go in. I am compiling the code altogether.</p><pre><br><br>terraform{<br>    backend &quot;s3&quot; {<br>        <br>        bucket = &quot;s3-remote-hcnorth&quot;<br>        encrypt = true<br>        key = &quot;.tfstate&quot;<br>        region = &quot;us-east-1&quot;<br>    }<br>}<br><br>resource &quot;aws_s3_bucket&quot; &quot;artifacts&quot; {<br>  bucket = &quot;hcn-artifacts-pod4&quot;<br>}<br><br>resource &quot;aws_s3_bucket_ownership_controls&quot; &quot;owner&quot; {<br>  bucket = aws_s3_bucket.artifacts.id<br>  rule {<br>    object_ownership = &quot;BucketOwnerPreferred&quot;<br>  }<br>}<br><br>resource &quot;aws_s3_bucket_public_access_block&quot; &quot;acl&quot; {<br>  bucket = aws_s3_bucket.artifacts.id<br><br>  block_public_acls       = false<br>  block_public_policy     = false<br>  ignore_public_acls      = false<br>  restrict_public_buckets = false<br>}<br><br><br>resource &quot;aws_s3_bucket_policy&quot; &quot;s3_policy&quot; {<br>  depends_on = [aws_s3_bucket_public_access_block.acl]<br>  bucket = aws_s3_bucket.artifacts.id<br> <br>  policy = jsonencode({<br>    &quot;Version&quot; : &quot;2012-10-17&quot;,<br>    &quot;Statement&quot; : [<br>      {<br>        &quot;Effect&quot; : &quot;Allow&quot;,<br>        &quot;Principal&quot; : &quot;*&quot;,<br>        &quot;Action&quot; : &quot;s3:GetObject&quot;,<br>        &quot;Resource&quot; : &quot;arn:aws:s3:::hcn-artifacts-pod4/*&quot;<br>      }<br>    ]<br>  })<br>}</pre><p>I created another bucket with name “hcn-artifacts-pod4”, I gave public access to this bucket. This bucket is where all our CI/CD pipeline processes will go.</p><p>At this stage, run “Terraform init” and “Terraform validate” to check if we have no issue in our configuration file till this point.</p><p>I also did “terraform plan” and “Terrafom apply” and my artifacts bucket was created. But its empty as we have’t created the pipeline yet to store the artifact files in the bucket.</p><h4>Create CodeBuild Project:</h4><p>AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. You don’t need to provision, manage, and scale your own build servers. It automates the process of building and testing code every time a change is made, facilitating a more efficient and reliable development workflow.</p><pre>resource &quot;aws_codebuild_project&quot; &quot;project&quot; {<br>  name          = &quot;my-codebuild-project&quot;<br>  description   = &quot;Build project for my website&quot;<br>  service_role  = &quot;arn:aws:iam::060795916438:role/CodeBuildServiceRole&quot;<br>  source {<br>    type      = &quot;GITHUB&quot;<br>    location  = &quot;https://github.com/Client-HealthCare-North/Website&quot;<br>    buildspec = &quot;buildspec.yml&quot;  <br>  }<br>  artifacts {<br>    type = &quot;S3&quot;<br>    location = aws_s3_bucket.artifacts.id<br>  }<br>  environment {<br>    compute_type                = &quot;BUILD_GENERAL1_SMALL&quot;<br>    image                       = &quot;aws/codebuild/amazonlinux2-x86_64-standard:4.0&quot;<br>    type                        = &quot;LINUX_CONTAINER&quot;<br>    image_pull_credentials_type = &quot;CODEBUILD&quot;<br> }<br><br>}</pre><blockquote>service_role: Provide service role arn for CodeBuild.</blockquote><p>CodeBuild is taking my code from GitHub repo and deploying to S3 bucket, but it needs a server to run this configuration right? This is where “environment” block comes in. <br>As you can see we defined a few parameters for our server like comoute type, image, type, etc in our env block above.</p><p>I used same config as defined in Terraform registry of “aws_codebuild_project”. Very useful</p><p><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project">Terraform Registry</a></p><p>Next I need to create a buildspec.yml file that I mentioned in source block, which means my repository should have this file uploaded.</p><p>Create buildspec.yml file in the VS Code (the repo you cloned earlier) and paste this code there.</p><pre>version: 0.2<br><br>phases:<br>  build:<br>    commands:<br>    - echo &quot;This is build phase&quot;<br> <br>  post_build:<br>    commands:<br>    - echo &quot;Deployed to S3&quot;<br>artifacts:<br>  files:<br>    - &#39;**/*&#39;</pre><p><strong>What is this buildspec.yml file doing?</strong></p><p>I hav enothing fancy in my pre-build and post-build phase, as I am simply deploying static webpages to the dev bucket. So, I can use echo messages to print on the console just to show what stage we are in, and if they are successfully deployed.</p><ul><li>Defines a build process with a build phase and a post-build phase.</li><li>Outputs messages indicating the start of the build process and that deployment to S3 has occurred.</li><li>Collects all files generated during the build process to be stored as artifacts, ready for deployment (often handled by the subsequent steps in a CodePipeline).</li></ul><p>Now notice that you mentioned all the files of the current directory to be collected in build process, so you must have your website pages in your repository too. This is how my repository looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bc84rxKTd-E38ZOhvJ2wxA.png" /></figure><p>That’s all about build project.. Now we told build project where to collect the files from in our repo, but where are we gonna say “where to deploy” ? <br>The next step shows to create a Code Pipeline that will include “Deploy” stage to achieve this task.</p><h4>Create Code Pipeline</h4><p>A structured and well-thought-out delivery process doesn’t just include a single build job. It’s an orchestration of many jobs, often idempotent jobs, that enable you to roll out deployment packages and infrastructure in a reliable and safe manner.</p><p>And that’s exactly what AWS CodePipeline does: it’s your CodeBuild orchestration tool. With CodePipeline, you can create pipelines that connect jobs into stages to create a multi-step rollout process that can be easily reproduced and understood.</p><p>The below piece of code will finally create a Code Pipeline for us.</p><pre>resource &quot;aws_codepipeline&quot; &quot;pipeline&quot; {<br>  name     = &quot;my-pipeline&quot;<br>  role_arn = &quot;arn:aws:iam::060795916438:role/CodePipelineServiceRole&quot;<br><br>  artifact_store {<br>    location = aws_s3_bucket.artifacts.id<br>    type     = &quot;S3&quot;<br>  }<br><br>  stage {<br>    name = &quot;Source&quot;<br>    action {<br>      name             = &quot;Source&quot;<br>      category         = &quot;Source&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;CodeStarSourceConnection&quot;<br>      version          = &quot;1&quot;<br>      output_artifacts = [&quot;source_output&quot;]<br><br>      configuration = {<br>        ConnectionArn = var.github_connector  # create variable and save the connector name in .tfvars file<br>        FullRepositoryId = &quot;Client-HealthCare-North/Website&quot;  # Update with your GitHub repo name<br>        BranchName = &quot;main&quot;  # Specify the branch<br>      }<br>    }<br>  }<br><br>  stage {<br>    name = &quot;Build&quot;<br>    action {<br>      name             = &quot;Build&quot;<br>      category         = &quot;Build&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;CodeBuild&quot;<br>      version          = &quot;1&quot;<br>      input_artifacts  = [&quot;source_output&quot;]<br>      output_artifacts = [&quot;build_output&quot;]<br><br>      configuration = {<br>        ProjectName = aws_codebuild_project.project.name<br>      }<br>    }<br>  }<br><br>  stage {<br>    name = &quot;Deploy&quot;<br>    action {<br>      name             = &quot;Deploy&quot;<br>      category         = &quot;Deploy&quot;<br>      owner            = &quot;AWS&quot;<br>      provider         = &quot;S3&quot;<br>      version          = &quot;1&quot;<br>      input_artifacts  = [&quot;build_output&quot;]<br><br>      configuration = {<br>        BucketName = &quot;hcn-dev-bucket&quot;  # my dev bucket<br>        Extract    = &quot;true&quot;<br>      }<br>    }<br>  }<br>}</pre><blockquote><strong>role_arn</strong>: Provide service role for CodePipeline.</blockquote><blockquote><strong>artifact_store</strong> is storing all the CI/CD files created from this step in S3 bucket defined.</blockquote><blockquote><strong>FullRepositoryID</strong>: Your GitHub repo url</blockquote><blockquote><strong>Branch</strong>: Where you want to take the cource code from</blockquote><p>Now comes the most important part i.e.<strong> Source. </strong>Here your GitHub connection that you created earlier will come handy.</p><p>For ConnectionArn, notice that I defined a variable (so create a variables.tf file and define variable <strong>github_connector.</strong> I don’t want to keep the connector information in variables.tf file as it could be deployed to GitHub to public, so I am also creating terraform.tfvars file with then actual value of my GitHub connection arn that you created earlier), like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CA420YABS4RvmnqR9zqVQQ.png" /></figure><p>(Alot of things can go on variables.tf file, like the service role arn for CodePipeline and CodeBuild, region, etc).</p><p>The Deploy stage is fairly simple, it just mentions the bucket name your code will be deployed to.</p><p>Also, I kept my dev bucket empty to see the changes deployed.</p><p>Now we have everything ready, we should go ahead and run Terraform commands.</p><pre>Terraform plan<br>Terraform apply</pre><p>It will show the plan to create CodeBuild project and CodePipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LzfeI1tbq_IsalDJotq1JA.png" /><figcaption>I had used the wrong role before, fixed it later</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wraMyhB1ZCUHR1hlpnycJQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/1*P5bqj5BwQ8gWU7Mdop3xPQ.png" /></figure><p>The build and deploy stages are also successful, which is a good news.</p><p>We will head to S3 console to see if the dev bucket is populated with the source code our Git repo had. Yes it does.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QFbUYHlhlbZMcCVD7frBiA.png" /></figure><p>Now copy the bucket website endpoint in the browser to see if its showing the website page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ibwa7948y8GaszFzT45QwA.png" /></figure><h4>Updating Index.html page</h4><p>We are chekcing if the Pipeline triggers when a chnage is made to index.html file in the GitHub repository.</p><p>In my VS Code, I will open index.html page in the editor, and make some changes. As the changes are detected, it will appear in the GitHub desktop to push those changes with commit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c75Sxia0Zkds3bR8rMTVXw.png" /></figure><p>Create Pull request and it will take you to remote repo where you can create PR. Wait for the approval to be able to merge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k9QcdQ1CfvNtNM3xUYLqBw.png" /></figure><p>When the index.html file is merged to the main branch, the pipeline will trigger. Check in the console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/747/1*P5bqj5BwQ8gWU7Mdop3xPQ.png" /></figure><p>As the deploy stage is successful, refresh the webpage to see the changes you implemented.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lZ3nUUHPlSO_fAOHqqu16g.png" /></figure><p>These are the screenshots for my Remote backend bucket and artifact buckets and they are successfully storing the files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9eX6-MVoTSeGg4XfNkIJpw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pISJaIEeEaBjXYl3yWg-Sg.png" /></figure><p>We are all good.</p><h4>Deploy to Prod bucket:</h4><p>For security reasons, we didn’t deploy the changes directly to Prod bucket as this is where are customers are being served from (toegther with CloudFront).</p><p>We will now include a “manual approval” stage to our pipeline which will indicate that changes will be deployed to Prod bucket, only if they are approved by the reviwers.</p><p>Go to CodePipeline. Edit the pipeline, scroll down and “Add Stage” after the Deploy stage like this. I named it “maunal approval”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HGAsiy3AIGA9YbFkR7_ZUQ.png" /></figure><p>Once the stage is created, click ‘Add Action Group’ and select ‘Manual Approval’ for Action Provider.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vvKWscPybTOeINbvRBI45Q.png" /></figure><p>URL for review is optional but it’s good practice to review the Dev bucket once before we give go-ahead to deploy those changes in Prod. Hence, I used the Dev bucket URL. Save it.</p><p>Now Click “Add Action Group” again to specify where to deploy changes after approval. Notice the destination is Prod bucket now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I6eHjuDUzuRTaoemE5p75Q.png" /></figure><p>Changing the index.html file again to trigger the pipeline</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7WoRFuxOIy7vYBgob3M9uQ.png" /></figure><p>Index page shows changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1Bwz95DPTOgqM8jj3H7YWg.png" /></figure><p>Create PR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*40eMMRioIEvNsgo3qbtC5g.png" /></figure><p>Once its merged to main, Pipeline triggered and showed updated webpage in dev bucket.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pwaqypzVburXOmXlbIHupw.png" /></figure><p>While its still waiting for manual approval to be deployed to prod bucket.</p><p>Click Review and Approve.</p><p>(Note: You can create SNS topic and subscribe users to them so they can approve and receive notifications via email/sms).</p><p>I am just using the Pipeline console to review and approve the code myself.</p><p>Click Review and select “Approve” with a message. Click Submit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/757/1*9halLBvdMK_oMzGLJwimxA.png" /></figure><p>Once it’s submitted it triggers the pipeline and deployment to prod bucket begins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*D2aRhRYt9RSlhFM6vRB1Ew.png" /></figure><p>Now go to Prod bucket website endpoint and see that the changes are deployed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oZNbHhDIz2W-LDItkSn5MA.png" /></figure><p>Success!</p><p>One last thing to check, since our prod bucket is serving content to CF, we will check if the changes are relected in CF domain name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ej5SQlGpd6iAPCFA8UMbkw.png" /></figure><p>Success!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df1652d065d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Docker Swarm — Deploying Wordpress and MySQL]]></title>
            <link>https://medium.com/@hadianoor16/docker-swarm-deploying-wordpress-and-mysql-bd87a146997e?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd87a146997e</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Sat, 26 Oct 2024 19:12:21 GMT</pubDate>
            <atom:updated>2024-10-26T19:12:21.852Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Docker Swarm — Deploying Wordpress and MySQL</strong></h3><p>Level Up Publications is an online publication that posts blogs about the hottest updates and trends in IT. Level Up Publications is in the process of transitioning its web infrastructure to a containerized environment using Docker Swarm due to the rapid growth of its reader base. In a business landscape that is increasingly digital, efficient and flexible web infrastructure is not just a competitive advantage, but a necessity. The need to quickly adapt to changing customer demands, scale up or down based on traffic, and ensure uninterrupted service availability means businesses need a solution that can manage these requirements effectively and affordably.</p><p>When a business adopts Docker Swarm, it significantly simplifies the process of managing and scaling applications. Businesses can easily scale up to meet peak times and scale down during off-peak times, ensuring efficient use of resources and cost savings. It also improves the resiliency of the system. If a container fails, Docker Swarm can automatically replace it, minimizing downtime.</p><p>Moreover, Docker Swarm helps with maintaining the security and integrity of applications. By isolating applications into separate containers, one application’s vulnerability does not affect others. Also, sensitive information like passwords or API keys can be securely managed using Docker secrets.</p><p>For a company like Level Up Publications, the adoption of Docker Swarm provides the agility to respond to rapidly changing news cycles. It can also efficiently serve a growing readership, ensuring that the publication’s website remains available and responsive, even as traffic fluctuates. Moreover, by improving the security of the system, Docker Swarm helps the publication protect its valuable content and data, as well as maintain the trust of its readership.</p><p><strong>Foundational</strong></p><p>Using the CLI:</p><ol><li>Deploy a 3 node Docker Swarm cluster.</li><li>Create 2 networks. One for frontend and another for backend.</li><li>Create MySQL service</li></ol><ul><li>associated with the backend network</li><li>Only 1 replica</li><li>Make the database service resilient and its data persistent across container restarts, use Docker Volumes.</li></ul><p>4. Create the Wordpress service</p><ul><li>Associated with the frontend network</li><li>Should use port 80</li><li>Accessible from the browser</li><li>3 replicas</li><li>Associated with the MySQL service</li></ul><p>Steps to follow</p><h4><strong>1. Deploy 3 node swarm cluster</strong></h4><p>Please refer to my previous guide on creating 3 node swarm cluster here <a href="https://medium.com/@hadianoor16/creating-three-nodes-swarm-cluster-e3283118c404">https://medium.com/@hadianoor16/creating-three-nodes-swarm-cluster-e3283118c404</a></p><h4>2. Create 2 networks. One for frontend and another for backend</h4><p>Container networking refers to the ability for containers to connect to and communicate with each other, or to non-Docker workloads.</p><p>Docker offers four distinct network types:</p><ul><li><strong>Bridge</strong> networks create a software-based bridge between your host and the container.</li><li><strong>Host</strong> networks allow containers to share your host’s network stack without any isolation.</li><li><strong>Overlay</strong> networks are distributed networks that span multiple Docker hosts.</li><li><strong>None</strong> networks are closed networks that do not allow any inbound or outbound traffic.</li></ul><p>When you initialize a swarm or join a Docker host to an existing swarm, two new networks are created on that Docker host.</p><p><em>Use </em><strong><em>‘Docker network ls’</em></strong><em> command to see those networks.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*L9VZzZAJFtAIvQR4.png" /></figure><ul><li>An <strong>overlay </strong>network called ingress, which handles the control and data traffic related to swarm services. When you create a swarm service and do not connect it to a user-defined overlay network, it connects to the ingress network by default.</li><li>A <strong>bridge</strong> network called docker_gwbridge, which connects the individual Docker daemon to the other daemons participating in the swarm.</li></ul><p>Hence you can see that containers have networking enabled by default, so that they can make outgoing connections. Also Overlay network type defines well how it’s suitable for Swarm.</p><p>Run the commands below to create both networks with overlay type.</p><pre>docker network create — driver overlay frontend</pre><pre>docker network create — driver overlay backend</pre><pre>docker network ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*YqbfKXw4oroiEZX0V1qFkA.png" /></figure><h4>Create MySQL service</h4><p>Creating MySQl service with the backend network, only 1 replica while using Docker volume to persist data across containers.</p><p>We need to create Docker volume first:</p><pre>docker volume create mysql_vol</pre><p>Verify the volume is created:</p><pre>docker volume ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*1SoADNAxG1rxeyCu9dtuiw.png" /></figure><p>Now run the MySql service:</p><pre>docker service create --name mysql2 --network backend --mount type=volume,source=mysql_vol,target=/var/lib/mysql --replicas 1 -e MYSQL_ROOT_PASSWORD=mypasswd  mysql</pre><p>The -v mysql_vol:/var/lib/mysql part of the command mounts the mysql_vol directory from the underlying host system as /var/lib/mysql inside the container, where MySQL by default will write its data files.</p><blockquote>HINT: To find what directory to mount the volume to, you should see the official image page at docker hub and see where the volume is mounted.</blockquote><p><a href="https://hub.docker.com/_/mysql">https://hub.docker.com/_/mysql</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YDxazS931cLXRuU9byeTZw.png" /></figure><pre>docker service ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/677/1*2ufT8R94js1Qk_c5wZbPRQ.png" /></figure><p>Run below command to see the tasks assigned to the service along with node the task is running.</p><pre>docker service ps mysql2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oX-qKrXXNsKkVt6twcqvkw.png" /></figure><p>Notice that <strong>mysql2 </strong>service created a task with ID “i5nqd659mxd5”, which is running on a container assigned to Node2.</p><p>We can either find the container ID by listing the containers on Node2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Fr3pY9QKop9vhzgssZREiw.png" /></figure><p>Or, if you don&#39;t have access to Node2, then you can find the container ID from task ID, Use below command:</p><pre>docker inspect -f &quot;{{.Status.ContainerStatus.ContainerID}}&quot; &lt;task_id&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/827/1*yc4yS6dCx83SNmBQiaXd6g.png" /></figure><p>The output of the above command is just:</p><pre>6882c47182976d8feeccbd51bfb444d50fda37ad062a8710db622814b8bd8fbb</pre><p>This is the container ID in the format of the 64-character SHA-256 ID.</p><p>But I’m familiar with the 12-character length Container IDs that docker ps shows.</p><p>The explanation is that the 12-character Container ID is just a truncated version of the full length 64-character ID — it takes the first 12 characters.</p><p>So, in my case, <strong><em>6882c47182976d8feeccbd51bfb444d50fda37ad062a8710db622814b8bd8fbb</em></strong> will be <strong><em>6882c4718297</em></strong> in the 12-character format.</p><p>Use docker service inspect command to verify volume is mounted to the service.</p><pre>docker service inspect mysql2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KUs87PEkKrNZ_BTwWNlpoA.png" /></figure><h4>Create the WordPress service</h4><p>Pull the image first</p><pre>docker pull wordpress:latest</pre><p>create service now</p><pre>sudo docker service create — name wordpress — network frontend — replicas 3 -p 80:80 — mount type=volume,source=wordpress-data,target=/var/www/html<br> wordpress:latest</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6Ymj8JctnueK5IWcduHEZA.png" /></figure><p>docker service ls</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*k0kJCVWunYSaN35A3xVZaw.png" /></figure><p>wordpress service is created with all 3 replicas successful.</p><pre>docker service ps wordpress</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*epLOYd6WkbKOIE2Pg_kbGQ.png" /></figure><p>Each node is running a replica</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V4t42Nwu65p0I7JIoyN2RA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r-NAGbME-aeL6nfw13eCZg.png" /></figure><p>mysql2 is service name</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xyR8lHTDjSCTGERQ_36MKw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd87a146997e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Terraform ASG Deployment]]></title>
            <link>https://medium.com/@hadianoor16/terraform-asg-deployment-2e54b704297f?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e54b704297f</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Mon, 14 Oct 2024 07:06:48 GMT</pubDate>
            <atom:updated>2024-10-14T07:10:11.575Z</atom:updated>
            <content:encoded><![CDATA[<h4>A step-by-step guide of using Terraform to deploy your servers via Autoscaling group with AWS S3 as remote backend</h4><p><strong>Background:</strong></p><p>An e-commerce company needs to handle a surge in traffic during the holiday season. The company wants to ensure that their website remains available and responsive to customers even during high traffic periods.</p><p>The project can be used to launch an Auto Scaling group that spans two subnets in the default VPC, ensuring high availability and fault tolerance. The Auto Scaling group will automatically scale up or down based on traffic, ensuring that the website remains responsive to customers at all times.</p><p>To ensure that the instances in the Auto Scaling group are secure, a security group is created that allows traffic from the internet and associates it with the instances. Additionally, a script is included in the user data to launch an Apache web server, ensuring that the website is available to customers.</p><p>To ensure that the Auto Scaling group has the appropriate capacity, it is set to have a minimum of two instances and a maximum of five. This ensures that the website can handle a surge in traffic during the holiday season, while also minimizing costs during periods of low traffic.</p><p>To verify that everything is working correctly, the public IP addresses of the two instances are checked. One of the instances is manually terminated to verify that another one spins up to meet the minimum requirement of two instances.</p><p>Finally, to ensure that the infrastructure is reliable and can be easily managed, an S3 bucket is created and set as the remote backend for Terraform. This ensures that the infrastructure is versioned and can be easily rolled back if necessary. These are the steps to follow:</p><ol><li>Launch an Auto Scaling group that spans 2 subnets in your default VPC.</li><li>Create a security group that allows traffic from the internet and associate it with the Auto Scaling group instances.</li><li>Include a script in your user data to launch an Apache webserver. The Auto Scaling group should have a min of 2 and max of 5.</li><li>To verify everything is working check the public IP addresses of the two instances. Manually terminate one of the instances to verify that another one spins up to meet the minimum requirement of 2 instances.</li><li>Create an S3 bucket and set it as your remote backend.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/0*xnGvKpyYS8_oVC1c.png" /></figure><p><strong>Prerequisites:</strong></p><ol><li>Terraform installed on your machine.</li><li>Use ‘aws configure’ and use your access keys on CLI to interact with AWS services.</li></ol><p>Let&#39;s start accomplishing these tasks one by one.</p><p><strong>Steps:</strong></p><ol><li>In the repository, create a new branch in GitHub desktop with the name “Autoscaling”. (I have explained it in detail in my first terraform project how to create a new repo, clone it and create a new branch, please find it here: <a href="https://medium.com/@hadianoor16/deploy-a-jenkins-server-with-terraform-b6a28272572f">https://medium.com/@hadianoor16/deploy-a-jenkins-server-with-terraform-b6a28272572f</a>)</li><li>Publish that branch to remote and open in VS code.</li></ol><p>3. Create a new folder under Terraform directory with the name ‘autoscaling’. Create main.tf file under this folder.</p><p>I copied most of the code from terraform registry, from this link:</p><p><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*H6kMDWfOwhKuJ4hHt6UaUw.png" /></figure><h4>Configuring main.tf file:</h4><ol><li>Starting our main file with the provider block:</li></ol><pre>terraform {<br>  required_providers {<br>    aws = {<br>      source  = &quot;hashicorp/aws&quot;<br>      version = &quot;~&gt; 5.56&quot;<br>    }<br>  }<br>}<br><br>provider &quot;aws&quot; {<br>  region = var.aws_region<br>}</pre><p><strong><em>Note</em></strong><em>: The value for AWS region is defined in variables.tf file. I explained below how we can create variables.tf file and reference them in main file.</em></p><h4><strong>2. Create Security Group for EC2 instances:</strong></h4><p>Now, we need to create a security group for the EC2 instances that will be deployed via Autoscaling group.</p><p>Go to UI. In EC2, find security groups and create a security group with any name and description.</p><p>Allow HTTP and HTTPs from anywhere for our Apache servers. Make sure to select default VPC from the dropdown. Click create.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*94iSpMNbuk98jl_BvZ35TQ.png" /></figure><h4><strong>3. Adding Launch template resource:</strong></h4><p>This is how my final resource block for Launch template looks like:</p><pre>resource &quot;aws_launch_template&quot; &quot;tf-lt&quot; {<br>  name_prefix            = &quot;terraforn-LT&quot;<br>  image_id               = var.ami_id<br>  instance_type          = var.instance_type<br>  vpc_security_group_ids = [&quot;sg-039b33bfa674b46df&quot;]<br>  user_data              = filebase64(&quot;user_data.sh&quot;)<br>}</pre><p>NOTE: Launch template has lots of other attributes that can be included. I just used these parameters for my project.</p><blockquote><strong>name_prefix</strong>: Name for your Launch template. (I named it wrong with space, and it threw an error about naming convention when I initialized it. I am going to discuss that in the Troubleshooting section).</blockquote><blockquote><strong>image_id</strong>: Copy AMI ID from AMI catalog for Amazon Linux 2. Value is defined in variables.tf file.</blockquote><blockquote><strong>instance_type</strong>: I used t3.micro instance. Value is defined in variables.tf file.</blockquote><blockquote><strong>vpc_security_group_ids</strong>: Paste the SG id you just created in the UI.</blockquote><blockquote><strong>user_data</strong>: Create a “user_data.sh” file in the same Autoscaling folder, and reference it here.</blockquote><h4>Troubleshooting:</h4><p>I had a lot of errors when I initialized the Terraform configuration file. Listing them below:</p><ol><li><strong><em>name_prefix: </em></strong>I used the wrong format (with space in the name) and it&#39;s not expected.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3lbZroWyqpNMWqifA2cjjg.png" /><figcaption>error I got with name_prefix</figcaption></figure><p><strong><em>Resolution</em></strong>: I fixed it by replacing the space with “-”.</p><p>2. <strong><em>vpc_security_group_ids:</em></strong></p><p>I used the wrong attribute type for the security group in Launch template. It threw an error like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jGXzJbriEpA5ZHSsUxeEuA.png" /></figure><p>I googled “how to define security group in Launch Template in Terraform”.</p><p>I found my answer in Terraform registry, in aws_autoscaling_group resource.</p><p><a href="https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template">https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1WxCqus9Sh1-s2w8SSqcrA.png" /></figure><p>Another error in defining SG was to use it without quoted strings, if you do so, you will see error like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*TzrJVmNf_rZ4eOcnYFFw5g.png" /></figure><p>This is how I resolved the error:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/668/1*HA-g3LqDGNCL8-AE4oAHeg.png" /></figure><p>3. <strong><em>User_data: </em></strong>With user data, I had decoding issue when I first initialized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*plY6ODiAXVRjgiLbGkxGjg.png" /></figure><p>If you notice the error clearly says “Malformed: Invalid BASE64 encoding of user data”.</p><p>So, I looked it up in the same Terraform registry page and found that the user data file must be in filebase64 format.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*9CBLVCq1M2i2cDUf-L6MZA.png" /></figure><h4><strong>4. Create user_data.sh:</strong></h4><p>Create user data script to install Apache webservers through the Launch template. The script is quite simple.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*TsrrqmFYUMudRQdZlzefog.png" /></figure><h4><strong>5. Create Variables.tf file:</strong></h4><p>Variable file contains all the attributes that can be changed in your configuration, but you don’t want to change your code every time. Simply change the values in variables.tf file and reference them in main file.</p><p>These are some of the variables that I used, you can always use more for efficient code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*SCq4Njkc-4FCNbJQQTceMg.png" /></figure><h4>6. Create Auto Scaling Group:</h4><p>Now that Launch template is defined, we need to create an ASG to use this LT and deploy our EC2 instances.</p><p>This is how the ASG resource block looks like:</p><pre>resource &quot;aws_autoscaling_group&quot; &quot;TF-ASG&quot; {<br>  vpc_zone_identifier       = [&quot;subnet-076333c7e353f00d7, subnet-0addb86de1f7595b1&quot;]<br>  health_check_grace_period = 300<br>  health_check_type         = &quot;EC2&quot;<br>  desired_capacity          = 2<br>  max_size                  = 5<br>  min_size                  = 2<br><br>  launch_template {<br>    id      = aws_launch_template.tf-lt.id<br>    version = &quot;$Latest&quot;<br>  }<br>}</pre><blockquote><strong>vpc_zone_identifier:</strong> It defines what subnets your EC2 should be deployed in. I listed two subnets only.</blockquote><blockquote><strong>health_check_grace_period</strong>: The health check grace period specifies the minimum amount of time (in seconds) to keep a new instance in service before terminating it if it’s found to be unhealthy.</blockquote><blockquote><strong><em>Launch_template</em></strong>: It references the Launch template that this ASG will use.</blockquote><p>Other parameters are self-explanatory like minimum, desired and maximum capacity.</p><p>Now that everything is declared and ready, we will run Terraform commands to create these resources in AWS.</p><h4>Terraform commands:</h4><ol><li>First, we will initialize the working directory.</li></ol><pre>terraform init</pre><p>This command initializes a working directory containing Terraform configuration files, downloads, and installs the necessary providers.</p><p>2. The other important command to run is:</p><pre>terraform validate</pre><p>It validates the syntax and configuration of the Terraform files in the directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-JW2Qx3VcvosCMOm5-KD9Q.png" /><figcaption>Terraform init and validate shows success</figcaption></figure><p>3. The next command that can be used is:</p><pre>terraform fmt</pre><p>This command is used to rewrite Terraform configuration files to a canonical format and style. This command applies a subset of the Terraform language style conventions, along with other minor adjustments for readability.</p><p>4. Now we will run terraform plan command to see an execution plan, showing what actions Terraform will take to achieve the desired state.</p><pre>terraform plan -out plan.txt</pre><p>Note: I am saving the output of this command to plan.txt file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RkxxLjs-ZnEVyZRoXTWXEw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L4BxXYr1-RwgiRWRXf4syQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4qEceOSzzoB_ZDVdVIBBJw.png" /></figure><p>Now we will go through this plan if everything looks Ok. Next command is to finally deploy this plan.</p><p>5. The command to be used is,</p><pre>terraform apply</pre><p>It will ask for approval if we want to create this infrastructure, type Yes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bGVz8wL-ggiGEfPl2a9vwg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hBtc0Oo2Ff7uRm3BW8lc6Q.png" /></figure><p>After you type Yes, it will show what resources are being created with their progress.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1017/1*AqERxbMEaWMPIGpTcTcUOg.png" /></figure><h4>Verify</h4><p>Go to AWS console to check if all the resources are being created.</p><p>Under EC2 section, see that the two EC2 instances are initializing, one in each AZ.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*05PcpTdtUp71sHkG0vNiyQ.png" /></figure><p>Copy one of EC2 instance IP and paste in clipboard, you will see default Apache page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L4wymtg-oMykDOv08eL0wg.png" /></figure><p>Now check that the Launch template and Auto scaling group are created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HAvvl1V7KXcsDHU38swJbQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gj44_xbUoMPvpVzCYGigVw.png" /></figure><p>Now try terminating one of the instances and ASG should try replacing the unhealthy instance with the healthy one.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IfMQ6iZGspb3RDJGRnShvA.png" /><figcaption>Stop or terminate the running ins</figcaption></figure><p>As you stop the instance, it’s considered unhealthy and ASG would terminate this instance and create a new one matching the desired capacity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iN8pYE8oaUCGR5BIb5AosQ.png" /><figcaption>Activity tab shows the instance is terminated and new instance was created in response.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G8AQjmmGzpp0DmVa95gOJw.png" /><figcaption>Instance management tab shows unhealthy instance is terminating and new instance in pending state.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GYwnoCtSBHCL2Sj3gijYEA.png" /><figcaption>Instances tab shows two instances running in both AZ</figcaption></figure><h4>Create an S3 bucket:</h4><p>The last step is to create an S3 bucket and set it as our remote backend.</p><p><strong>What is the purpose of creating AWS S3 remote backend?</strong></p><p>One of the critical aspects of using Terraform effectively is managing the <strong>Terraform state file</strong> — the single source of truth that records your infrastructure’s current status. It also helps with change detection i.e. When running terraform plan or terraform apply, Terraform checks this state file to understand what has changed.</p><p>Terraform offers two main ways to store the state file:</p><ol><li><strong>Local Backend</strong>: Stores the state file on your local machine or a shared file system.</li><li><strong>Remote Backend</strong>: Stores the state file on a remote service, such as AWS S3, ensuring centralized access and better collaboration.</li></ol><p>The Local backend is simple, but it has few challenges like collaboration issues, risk of data loss, limited system memory, managing backups, and security concerns.</p><p>Remote backend is mostly the better choice with AWS S3 being preferred because of its Reliability, version control, encryption of the objects, Access Control, and Recovery.</p><p>You first need to have a bucket created then you can set it as remote backend. Bucket can be created either in the same configuration file or through AWS console. I created it via AWS console and then added backend S3 module in my main.tf file.</p><p>Choose a unique bucket name. Keep all the settings as default, just enable Versioning. It will help keep all the versions of state file and protect them from accidental deletion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t9snZpFyP3qkTv_C5GMcsQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/987/1*ZJyLSVo4_khdq2WSLMqysw.png" /></figure><p>After you are done creating the bucket, go to main.tf file and add the following code preferably in the beginning of the main.tf file to include the remote backend settings.</p><pre>backend &quot;s3&quot; {<br>    bucket         = &quot;terraform-asg-hnoor&quot; <br>    key            = &quot;global/s3/terraform.tfstate&quot;<br>    region         = &quot;us-east-1&quot;<br>  }</pre><blockquote><strong>bucket</strong>: your bucket name where to store state files.</blockquote><blockquote><strong>key</strong>: Path to the state file within the bucket.</blockquote><blockquote><strong>region</strong>: AWS region where the bucket is located.</blockquote><p>This is how my final configuration file looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*8AqbJEr6x1R3AkBrrCFekQ.png" /></figure><p>Now again we will run all the TF commands, and after they are all successful, go to AWS console -&gt; S3.</p><p>Open the bucket you had created and notice that there are folders created in it, same as you defined in the “Key” attribute of backend S3 module.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aEyFVfCozIdp8TrD5JBZvw.png" /></figure><h4>Push to GitHub:</h4><p>Go to GitHub Desktop and see all the files that have been changed and ready to be pushed.</p><p>Select all files like main.tf, variables.tf, and user_data.sh, commit with a message push to remote.</p><p>On remote, create PR to merge them into the main and delete the ‘autoscaling’ branch safely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MPofyQX5WI-sw19tHUqJxA.png" /></figure><p>When you are done, don’t forget to remove all the resources to avoid charges on your account.</p><pre>terraform destroy</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aX2211MrTY6B2mAGWevmVg.png" /></figure><h4>Summary:</h4><p>This project was helpful in gaining some of the concepts about defining ASG and setting S3 as remote backend. There was a lot of troubleshooting involved, and it took me time to figure out a few things with a lot of Google checks. There could be other security measures added to this infrastructure, for example creating a Load balancer and configuring the Autoscaling group to accept traffic only from the Load balancer, I am closing it for now, but I will try to include those too :) stay tuned!</p><p>Link to my Github and project code: <a href="https://github.com/hadia-noor16/Terraform/tree/main/autoscaling">https://github.com/hadia-noor16/Terraform/tree/main/autoscaling</a></p><p>If you like my articles, feel free to connect on <a href="https://www.linkedin.com/in/hadianoor/">Linkedin</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e54b704297f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploy a Jenkins server with Terraform]]></title>
            <link>https://medium.com/@hadianoor16/deploy-a-jenkins-server-with-terraform-b6a28272572f?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b6a28272572f</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Mon, 07 Oct 2024 05:09:01 GMT</pubDate>
            <atom:updated>2024-10-09T23:57:52.071Z</atom:updated>
            <content:encoded><![CDATA[<p>A detailed guide of using Terraform to deploy your infrastructure.</p><h4>Introduction:</h4><p>Terraform is a popular infrastructure-as-code tool that allows you to automate the provisioning and management of infrastructure resources. It uses configuration files written in the HashiCorp Configuration Language (HCL) to define the desired state of your infrastructure, and it uses various commands to apply those configurations and manage your infrastructure resources.</p><h4>What is Hashicorp Terraform?</h4><p>Terraform is an open-source infrastructure as code (IaC) software tool which can be used to provision the infrastructure of any cloud platform. The scripts which have been used to provision infrastructure can be human-readable configuration files that can be versioned, reused, and shared. You can use a wide range of resources in the cloud by using terraform like compute, storage, networking, and application services, across a variety of cloud providers and on-premises environments.</p><p>The language used in the terraform is declarative type where you can just mention the required state no need of mentioning how to achieve that desired state. With the help of the state file terraform will create, modify, or destroy resources.</p><h4>What is Infrastructure as a Code (IaC)?</h4><p>Infrastructure as Code (IaC) is a method of managing and provisioning IT infrastructure using code, rather than manual configuration. It allows teams to automate the setup and management of their infrastructure, making it more efficient and consistent. This is particularly useful in the DevOps environment, where teams are constantly updating and deploying software. Another popular service for ‘Infrastructure as Code’ is AWS CloudFormation.</p><h4>What are Use Cases of Terraform?</h4><p>The following are some of the use cases of terraform.</p><ol><li><strong>Provisioning Cloud Resources: </strong>Several types of cloud resources can be provisioned by using terraform like AWS, GCP, and others. The resources that can be managed are compute, storage, networking, and application services.</li><li><strong>Multi-Cloud Management: </strong>You can manage the infrastructure of different cloud platforms at a time which will help you to maintain the multi-cloud or hybrid cloud environments.</li><li><strong>Infrastructure Versioning and Collaboration: </strong>You can store the scripts which have been written to provision the infrastructure in the version control system like git form where other teams can collaborate on infrastructure changes, track revisions, and roll back to previous states if needed.</li><li><strong>Automation and Continuous Integration/Continuous Deployment (CI/CD): </strong>You can also integrate the terraform into your <a href="https://www.geeksforgeeks.org/what-is-ci-cd/">CI/CD pipelines</a> wherever the build is triggered if there are any changes the infrastructure will upgrade automatically.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KNn1mgdBSS5GmFIZ.png" /></figure><p>Now keeping all these advantages in mind, your team would like to start using Jenkins as their CI/CD tool to create pipelines for DevOps projects. They need you to create the Jenkins server using Terraform so that it can be used in other environments and so that changes to the environment are better tracked. We will start with the following steps, and I will also cover how you can create your repository in GitHub and push this code for future use.</p><p><strong>Prerequisites:</strong></p><p>You are required to have access to your AWS account to interact with AWS services. I used my access keys on CLI to interact with AWS services. It&#39;s not recommended to put your keys on a system that’s being used by multiple users, the best way is to set up environment variables for your credentials. I ran AWS Configure command and provided my Access_Keys and SECRET_ACCESS_KEY, region, and output field in VS Code and I verified it by running ‘<em>aws s3 ls</em>’ that showed me all the buckets in my account.</p><p><strong>Steps:</strong></p><ol><li>Deploy EC2 Instances in your Default VPC.</li><li>Bootstrap the EC2 instance with a script that will install and start Jenkins. Review the official Jenkins Documentation for more information:<a href="https://www.jenkins.io/doc/book/installing/linux/"> </a><a href="https://www.jenkins.io/doc/tutorials/tutorial-for-installing-jenkins-on-AWS/">https://www.jenkins.io/doc/tutorials/tutorial-for-installing-jenkins-on-AWS/</a></li><li>Create and assign a Security Group to the Jenkins Security Group that allows traffic on port 22 from your IP and allows traffic from port 8080.</li><li>Create an S3 bucket for your Jenkins Artifacts that is not open to the public.</li><li>Verify that you can reach your Jenkins install via port 8080 in your browser. Be sure to include a screenshot of the Jenkins login screen in your documentation.</li><li>Push your code to GitHub and include the link in your write up.</li><li>Add a variables.tf file and make sure nothing is hardcoded in your main.tf.</li><li>Create separate file for your providers.tf if you have not already.</li></ol><p>We will first create a repository where all our code resides.</p><h4>How to create a new repository on remote</h4><ol><li>First create a public repository in Github with the <strong>Add a README</strong> file and choose Terraform from .<strong>gitignore </strong>file dropdown. The .gitignore file can keep track of secret files like state files that are sensitive to share with the public.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/951/1*yUYPnW4lKtYrHj7Xhu7YQw.png" /></figure><p><strong>Why to add .gitignore file</strong></p><p>For several reasons, not all files and directories should be part of the git repository. The following are some of the files included in the .gitignore file in a generic Terraform project.</p><ol><li><strong>.terraform.tfstate</strong> — Terraform state files should never be pushed to the git repositories. Note that when using the remote backend for Terraform, the state files will not be available on the local system. A couple of reasons are:</li></ol><ul><li><em>Security </em>— State files may store sensitive details like keys, tokens, passwords, etc.</li><li><em>Collaboration </em>— When working within teams, managing the state file locally by each developer poses a high risk of state files needing to be more consistently overwritten.</li></ul><p><strong>2. Binaries</strong> — The provider plugins downloaded locally or on a Terraform host (in .terraform directory) should not be part of the Git repository. The binaries thus downloaded are large in memory. Pushing and pulling the binaries from a remote git repo is inefficient for using network bandwidth.</p><p><strong>3. Crash.log</strong> — Crash log files are not always required, especially when a crash occurs due to the local environment.</p><p><strong>4. *.tfplan</strong> — We use the terraform plan command to save and use the output during the apply phase. This information is not required to be stored on a remote git repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rw0GF_xbLtlkPVPRme-eBw.png" /></figure><p>Now go to Github desktop and clone the repository on your computer, you just created.</p><p>Click Add -&gt; Clone repository</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/821/1*Dcwj11U1W-opui0kDDB3-g.png" /></figure><p>Go to last tab “URL” and paste the repo url from remote.</p><p>On the Local path, you can choose where you want to create the local folder for this repository in your computer. I have created a folder “All my repos” in my computer, every repository that I create on remote, goes under it.</p><p>Click “Clone”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wan4BS9LciS8xOIHl27mfA.png" /></figure><p>Cloning will be successful in some time and you will be switched to your new repo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kanBApugzmeeeDr2b7_q5g.png" /></figure><p>Now, it’s good practice to create a separate branch in your repository. Common use case is ‘you want to build a new feature based on the master branch, without doing the actual work in master’. By using a separate branch for your feature, you can do all your development and testing in a separate place, leaving your stable master branch alone. This would allow other developers to continue working directly in master or to allow them to create their own feature branches.</p><p>Once you have completed the feature and it has been tested, you can bring this branch back into master by using <strong>git merge.</strong></p><p>So, let’s create a new branch under this repo. Click ‘New branch’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-9dwrySlJBuaA9Nqs2FH_Q.png" /></figure><p>Name your branch and click “Create branch” and publish it to remote.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QegJ7gK5MbMDY-AtbSJpYw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tWyjs7jXPG-O8aiQOS8pXw.png" /></figure><p>Your remote repo will now show two branches, main and Jenkins. Just stay on the main branch. Every change that you make in Jenkins branch locally, will be tracked by Github and in the end you will be able to merge all changes into the main. so Main branch is safe from any accidental changes or deletions.</p><p>Click “Open in VS Code” and you will be taken right into the folder you just created your repo in your computer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iwpXK9km8J0DD2YsEpjGDQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mAG8VxMF75eUvR-AGA9Ofg.png" /></figure><p>You can always navigate into the folder by simply opening VS Code and click File to open folder.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/947/1*2R3l-5czuXkwYXwPGEILEA.png" /></figure><p>For every project that I do, I create a separate folder for it under my Terraform repository like this…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IGsfPnWdmPOKln4RWS35Nw.png" /></figure><h4>Deploying EC2 via Terraform</h4><p>Now you will need to create a couple of new configurations files under this folder.</p><p><strong>What are Terraform configuration files?</strong></p><p>Terraform configuration files are used for writing your Terraform code. They have a .tf extension and use a declarative language called HashiCorp Configuration Language (HCL) to describe the different components that are used to automate your infrastructure.</p><p>To begin writing a Terraform configuration while adhering to the best practices, we create the <strong>files</strong> below in the project’s root directory.</p><p>Terraform file types include:</p><ol><li><strong>main.tf</strong> — containing the resource blocks that define the resources to be created in the target cloud platform.</li><li><strong>variables.tf</strong> — containing the variable declarations used in the resource blocks.</li><li><strong>provider.tf</strong> — containing the terraform block, s3 backend definition, provider configurations, and aliases.</li><li><strong>output.tf</strong> — containing the output that needs to be generated on successful completion of “apply” operation.</li><li><a href="https://spacelift.io/blog/terraform-tfvars"><strong>*.tfvars</strong></a><strong> </strong>— containing the environment-specific default values of variables.</li></ol><p><strong>main.tf</strong></p><p>The <strong>main.tf</strong> file is the starting point where you will implement the logic of infrastructure as code. This file will include Terraform resources, but it can also contain data sources and locals. For example, it would contain the AMI ID of the EC2 instance you want to launch, along with the security group, user data and SSH key pair to attach.</p><ol><li>Create main.tf file in the Jenkins folder in VS Code.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qoeKiXpSBzlhoTJRQYM4tQ.png" /></figure><p><strong>Add Provider</strong></p><p>I started my main file with the provider I am working with. Terraform can be integrated with so many providers, check the list here.</p><p><a href="https://registry.terraform.io/browse/providers">https://registry.terraform.io/browse/providers</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8qEkaz29jJJEjbWlRt7LlQ.png" /></figure><p>Click on AWS and on the Right side you will be able to see how you can integrate AWS with Terraform in your code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o9KCP9G6XL1EeBqysHj8Wg.png" /></figure><p>In you main.tf file, you just need to include</p><pre>provider “aws” {<br>region = “us-east-1”<br>}</pre><p>Save this file and run “<strong>terraform init</strong>” in the terminal. You see Terraform is smart enough to know this is AWS provider and what source and version it requires to run this program. Terraform will download all latest versions for you (though you did not define it).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*t_lmrvkeIzfgmsOHLcLK2A.png" /></figure><p><strong>Add resources</strong></p><p>Now time to add EC2 configuration in main.tf file.</p><p>You can either go to Documentation, type EC2 in the filter to get all the results with examples of how you would use Terraform to launch EC2 instance for you.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kLxRjbXNSMwshRWl3VtXNA.png" /></figure><p>OR….</p><p>you can simply Google with keywords like “EC2 instance terraform” and you will see lots of examples there.</p><p><strong>Create Security group:</strong></p><p>First create a security group to attach to your EC2 instance.</p><p>You can search for example in the registry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-QNGy0GMN-cYbnfVWVkzXg.png" /></figure><pre># Creating securoty group to open SSH port from your IP, port 8080 to open for all traffic and allowing all outbound traffic<br>resource &quot;aws_security_group&quot; &quot;tf_jenkins_sg&quot; {<br>  name        = &quot;tf_jenkins_sg&quot;<br>  description = &quot;Allow https inbound traffic and SSH from my IP&quot;<br>  vpc_id      = &quot;vpc-0af945834a669fd8d&quot;<br>  ingress {<br>    from_port   = 22<br>    to_port     = 22<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;your-IP-address&quot;]<br>  }<br><br>  ingress {<br>    from_port   = 8080<br>    to_port     = 8080<br>    protocol    = &quot;tcp&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br><br>  egress {<br>    from_port   = 0<br>    to_port     = 0<br>    protocol    = &quot;-1&quot;<br>    cidr_blocks = [&quot;0.0.0.0/0&quot;]<br>  }<br>  tags = {<br>    Name = &quot;tf_jenkins_sg&quot;<br>  }<br>}</pre><blockquote><strong>resource</strong> : aws_security_group (standard) while <em>tf_jenkins_sg</em> is the name this SG will be recognized in my main.tf file.</blockquote><blockquote><strong>VPC_ID</strong>: Provide VPC id to your SG. I am launching my instance in Default VPC, hence creating my SG in default VPC as well.</blockquote><blockquote>Ingress means Inbound rules. Egress means outbound rules. All inbound and outbound rules are self-explanatory. I am allowing SSH from my IP only, and port 8080 (that is used by Jenkins) from everywhere. I am allowing all traffic that’s going out from my instance.</blockquote><blockquote><strong>Tags</strong>: is used to tag your resources.</blockquote><h4><strong>Create EC2 instance</strong></h4><p>In resource block, resource type is “aws_instance” this is standardized by Terraform while “jenkins-ec2” is the name for this resource in configuration file.</p><pre># Creating an EC2 instance with Amazon Linux AMI ID, SG, key pair to associate and user data to install in EC2<br>resource &quot;aws_instance&quot; &quot;jenkins-ec2&quot; {<br>  ami           = &quot;ami-0fff1b9a61dec8a5f&quot;<br>  instance_type = &quot;t3.micro&quot;<br>  vpc_security_group_ids =[&quot;tf_jenkins_sg&quot;]<br>  <br>  key_name  = aws_key_pair.key_pair.key_name<br>  user_data = file(&quot;user_data.sh&quot;)<br>  tags = {<br>    Name = &quot;jenkins-ec2&quot;<br>  }<br>}</pre><blockquote><strong>ami</strong>: Copy AMI ID for Amazon Linux 2023, from AMI catalog in UI.</blockquote><blockquote><strong>instance_type</strong>: I chose t3.micro</blockquote><blockquote><strong>vpc_security_group_ids</strong>: The SG you created, just reference it here.</blockquote><p><strong>key_name</strong>:</p><p>You need to create a key for EC2 instance to be able to SSH into it. I created a file in the same folder with .tf extension. This code will create a SSH key pair and save it into the same folder when I run the terraform file (main.tf).</p><p>Reference: <a href="https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key">https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/868/1*SXT44-mg-9PYuflGi748Uw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W69mPxfxmBflUhrL4ks76A.png" /><figcaption>The tls private key is used to create private key.</figcaption></figure><p><strong>User_data:</strong></p><p>Save the script in same folder with .sh as an extension. This script contains user data for your instance to install Jenkins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tA28VgvHgE33TByI9qQFEQ.png" /></figure><p>You reference this in your main.tf file with “file” function.</p><h4>Create an S3 bucket for your Jenkins Artifacts</h4><p>Now another resource in the same file will be S3 bucket. By default, any bucket created has public access blocked, so I will be only creating bucket in us-east-1 region.</p><pre># Creating an S3 bucket for Jenkins artifacts<br>resource &quot;aws_s3_bucket&quot; &quot;jenkins-artifact&quot; {<br>  bucket = &quot;jenkins-artifact-hnoor&quot;<br><br>  tags = {<br>    Name = &quot;Jenkins-artifact&quot;<br>  }<br>}</pre><p>This is how your code looks like in the end.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-gka8oBzrUjTfzG5ABLHCw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*giE_I6Coe72JG4WP76nfuQ.png" /></figure><p>Don’t worry, I will also include a link to my GitHub repo for this code in the end.</p><p>Now that your files are all ready, save it and get ready to do some Terraform magic…</p><h4>Basic Terraform commands</h4><h4>1. Initialization</h4><p>Initialize a Working Directory</p><pre>terraform init</pre><p>Initializes a working directory containing Terraform configuration files. This command downloads and installs the necessary providers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cUO-ctvl4-LDtIKVNbIf3g.png" /></figure><p><strong>2. Validation</strong></p><p>Validates Configuration Files</p><pre>terraform validate</pre><p>Validates the syntax and configuration of the Terraform files in the directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D5S3wxivSm7Y_iksWZqjFQ.png" /></figure><p>The error you are seeing in the screenshot is because I didn’t use AMI ID in quotation. I fixed it and Terraform validation passed successfully.</p><p><strong>3. Plan</strong></p><p>Generates an Execution Plan</p><pre>terraform plan</pre><p>Generates an execution plan, showing what actions Terraform will take to achieve the desired state.</p><p>You can also save plan in some file in the same folder by running</p><pre>terraform plan -out plan.txt</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MJ4qGHAsGcXk8MS2gYs7-Q.png" /></figure><p><strong>4. Apply</strong></p><p>Apply Changes to Infrastructure</p><pre>terraform apply</pre><p>Applies the changes required to reach the desired state of the configuration. This command prompts for confirmation before proceeding. It prompts you to say yes or no if we want to proceed with the configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DukWYeeYja0ymbMQBJCxXQ.png" /></figure><p>As you run terraform apply, this will create a state file. This state file is very important. Next time when you make any changes to your configuration file, terraform will check this state file against your current configuration file and warn you about the changes you are going to make with this new configuration.</p><p><strong>Troubleshooting:</strong></p><p>I didn’t get everything right in the first attempt :) I had several issues with my original main.tf file.</p><p>First was the missed quotation for AMI Id. Validation failed due to this error.</p><p>The other issue was I couldn’t see Jenkins running as I copy the instance public IP with :8080 appended in the end. The page failed to load.</p><p>(All the other resources were just created fine, the security group, key pair, and bucket).</p><p>I ssh’ed into the EC2 with the key Terraform created, and ran</p><pre>sudo systemctl status Jenkins</pre><p>and found that the service was not running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uaawCEjj5kk9p4qSazp2mQ.png" /></figure><p>The user data had some issues, later I found out that the “Jenkins” that I am using in installing and enabling commands, has uppercase J in it, since Linux is case sensitive, it was not installing Jenkins. I changed it to lowercase j and saved the file, ran all terraform commands again.</p><p>The best thing is, you don’t need to destroy everything and redeploy it again. Terraform saves the state of your configuration in ‘state file’ and it compares new configuration with the state file, only deploying recent changes when you run terraform apply, and keeping already deployed resources untouched.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*l00-1mYDATXX_tEEhVn1dw.png" /></figure><p><strong>Jenkins Installed</strong></p><p>Once the file ran, EC2 instance was updated with new user data. I copied the EC2 instance ID with :8080 and was able to see default Jenkins page in the browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a5R2rmh9iVNyu_e2ElJrig.png" /></figure><p>This also shows the path to find the admin password for Jenkins to proceed with the setting up process.</p><p>You can get this password from your EC2 instance while you ssh into it.</p><pre>cd /var/lib/Jenkins/secrets</pre><p>and then run below command to see the file contents.</p><pre>cat initialAdminPassword</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*sYFMPxWa5kbApDeQdejlAg.png" /></figure><p>I completed the setup, and this is how it looks like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O9YU-3onIT5jLj1oIlV4Ew.png" /></figure><p><strong>Push your code to GitHub</strong></p><p>Go to Github Desktop, and you will see all the new files you might want to push. Select them all and click “Commit” with a message.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QaN1x8EQL-BEj31WvL2iaw.png" /></figure><p>On the remote, you will see Compare and Pull.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FClzN89VxtTnqOABkfXQPQ.png" /></figure><p>Click that and create Pull Request (PR) to merge these files into the main.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mYA2W58W925LPHph_mNHrQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8jdG9iVRFXpQ2SAf73kIeA.png" /></figure><p>After you click “Merge PR” your files show up in your remote repo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hy5xxnPL-Ah-Cy8Ngz6YrQ.png" /></figure><p>I didn’t push my ssh key that was created to my public repository. The State file was not in the list of files to be pushed to remote since we have added .gitignore file to our repository, it avoids all sensitive files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/553/1*NnpovfNQNMzGQSKqJqNhhg.png" /></figure><h4>Adding Providers.tf file:</h4><p><strong>What is Terraform provider:</strong></p><p>A provider in Terraform is a plugin that enables communication with external APIs and services. It lets Terraform interact with various cloud platforms, such as AWS, infrastructure providers, and other third-party services. Providers are specified within the Terraform configuration code, allowing users to define the specific services and resources they need to manage and provision.</p><p>In our project, we used AWS provider, and we used it in the beginning of main file, we can also create a separate file for all our providers we are using in Terraform infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/807/1*dpH4T-Qg-6SPMZJceEqWSg.png" /></figure><p>The Terraform AWS provider is a plugin that enables seamless integration between Terraform and the Amazon Web Services (AWS) cloud platform. It allows users to define, provision, and manage AWS resources using Terraform’s declarative configuration language.</p><p>(Note: Before using this provider, you must configure it with the appropriate credentials to authenticate and authorize access to your AWS account).</p><p>There are many ways of defining providers. In the example below, we specified the version of the provider we want to use in our configuration.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/832/1*2-Eg2Rt3ibUBpKIyk6bWSQ.png" /></figure><p>If we don’t specify a Terraform block with version constraint for the provider, Terraform will automatically download the latest available version of the AWS provider.</p><p>So, I am now adding a providers.tf file to my Terraform configuration files.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/432/1*b9P0PioeH1uEzFd4Ten2DQ.png" /><figcaption>Final look of my provider.tf file</figcaption></figure><p>I also defined another provider i.e. <strong>tls. </strong>If you remember, we used this provider to create private key (.pem) file for our EC2 instance to be able to SSH into it. Now we are explicitly defining in our providers.tf file with a specific version we are looking for. You can add as many providers as you want to your file.</p><p>Based on the provider above, we can configure AWS resources. For this example, we just configured an AWS EC2:</p><pre>resource &quot;aws_instance&quot; &quot;jenkins-ec2&quot; {<br> ami =         &quot;your-ami-id&quot;<br>instance_type= &quot;Your instance type&quot;<br>}</pre><p>You can configure as many resources as you want.</p><h4>Adding Variables.tf file:</h4><p>Imagine working in multiple environments, such as testing, staging, and production. We would prefer to deploy a larger instance in production and a smaller instance in the test environment to save money. Meaning that the code remains the same except for the instance type, which varies depending on the environment.</p><p>We already know that variables can be used to solve this problem. To keep things running, we’ll create a variable for the instance type and set its default value to “t3.micro”.</p><pre>variable “instance_type” {<br>description = “instance size”<br>type        = string<br>default     = “t3.micro”<br>}</pre><p>You don’t need to change your main.tf file every time, as it poses risks of human errors. You just need to change the variable according to different environments.</p><p>The variable can be referred to by using the var keyword followed by “.” and then the &lt;variable_name&gt; in main.tf file, like below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/762/1*v0BSHn9gkDRUwk15nw6oXw.png" /></figure><p>In my project, I updated my code to use the variables. I defined variables for aws_region, ami_id, instance_type, and vpc_id.</p><p>You can make use of more variables for building scalable, maintainable, and adaptable infrastructure configurations, ultimately contributing to efficient infrastructure management and deployment practices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*Rj6kWH54PRaAXZm01CyjRA.png" /><figcaption>Final look of my variables.tf file</figcaption></figure><p>I ran same set of commands again for Terraform:</p><pre>terraform init</pre><pre>terraform validate</pre><pre>terraform fmt # to check for any syntax error, and have a nice readable code</pre><pre>terraform plan -out plan # saving plan in a file</pre><pre>terraform apply</pre><p>After my instance was running, I copied its IP address and pasted it in the browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5zTpLHXd2ASraoiFQ0s4mg.png" /></figure><p>My instance is running the same as before, but I followed good practice of using Variables instead of hard coding everything in my main configuration file.</p><p>Now that you have added some new files to your Terraform world, get ready to push your new code to the remote repository.</p><p>Head over to the GitHub Desktop, and it will highlight the files that’s being changed or added.</p><p>Note that it shows nicely what lines are added and updated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ra_G1fgUVFSm97VBghCyFg.png" /><figcaption>VPC_ID is now being replaced by var.vpc_id</figcaption></figure><p>I am seeing all my new files here, (just unselect <em>key-pair.pem</em> file) and commit to your repo with a nice message for your future self.</p><p>Commit to your local branch and merge it with the main branch in the remote repository. (Follow the same steps of creating PR and merge).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ScbrqhlIpqCuB1VZIIrS2w.png" /></figure><p>After merging it with the main branch, you are welcome to delete your local branch, as it’s very safe to do so.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hQwgUILI9QfEylsz1OE7nA.png" /></figure><p>Your final folder looks something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j_D6AIURSxapdhgV88HvOg.png" /></figure><p>You can find all the files in this link to my GitHub account.</p><p><a href="https://github.com/hadia-noor16/Terraform/tree/main/Jenkins">Terraform/Jenkins at main · hadia-noor16/Terraform</a></p><h4>Terraform Destroy</h4><p>After you are done with your project, you can simply use this command to get rid of everything that was created as part of your configuration. You don’t have to delete them one by one.</p><pre>terraform destroy</pre><p>Terraform destroy command will destroy all the resources created by Terraform in the current working directory. It is a useful command for tearing down your infrastructure when you no longer need it.</p><h4>Summary:</h4><p>Terraform represents a significant leap forward in infrastructure management, offering benefits from automation and version control to multi-cloud deployment and cost management. Its approach of infrastructure as code not only streamlines the provisioning and management processes but also aligns them with modern DevOps practices, ensuring faster, safer, and more efficient operational workflows.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6a28272572f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating three nodes Swarm cluster]]></title>
            <link>https://medium.com/@hadianoor16/creating-three-nodes-swarm-cluster-e3283118c404?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/e3283118c404</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Thu, 26 Sep 2024 04:48:01 GMT</pubDate>
            <atom:updated>2024-09-30T05:49:54.277Z</atom:updated>
            <content:encoded><![CDATA[<p>In this document, I explained how we can create a three node Swarm cluster. This article is the subset of one of my projects <a href="https://medium.com/@hadianoor16/docker-swarm-a3877045ffae">https://medium.com/@hadianoor16/docker-swarm-a3877045ffae</a></p><p>Since that was a long project, I am keeping just Docker Swarm creation part in this article for ease.</p><p>Here are the steps to follow…</p><h4>Steps:</h4><h4>1. Launching EC2 instances with Docker installed:</h4><p>Login to EC2 console, click Launch instances. Use below values in these fields:</p><blockquote><em>Number of instances = 3</em></blockquote><blockquote><em>OS image as Ubuntu Server 22.04LT</em></blockquote><blockquote><em>Instance type as t2.micro,</em></blockquote><blockquote><em>Choose a key pair to be able to SSH.</em></blockquote><blockquote><em>For security group, keep SSH from ‘My IP’ and HTTP and HTTPs traffic from anywhere, we will keep modifying security group rules according to the project requirements.</em></blockquote><blockquote><em>Expand Advanced details, and in user data section, paste below commands to install Docker in all hosts.</em></blockquote><pre>#!/bin/bash<br>curl -fsSL https://get.docker.com -o get-docker.sh<br>sh get-docker.sh</pre><p>Click Launch instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/767/0*J6ux4wPaGvRgSp4g.png" /></figure><p>Notice that your instances are created with same name i.e Swarm-Node 1 (in my case), you can edit them as you click on the pencil to the right of all these names. I changed them to Node 1, 2 and 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*lQIpoSuw6Nk-TFzS.png" /></figure><h4>SSH into Node 1:</h4><p>We will now SSH into Node 1</p><ol><li>Open Visual Studio Code with Remote-SSH as an added extension installed.</li><li>On the bottom left you will see green icon with<strong> &gt;&lt;</strong> symbol, it says “Open a Remote Window”. Click that then you will see several options appearing at the top.</li></ol><p>Select ‘<strong>Connect to Host’</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*C1b77bZiMKPvVb72.png" /></figure><p>3. Select <strong>Configure SSH Hosts</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/0*NZEB87knJKr3pUci.png" /></figure><p>4. Select the option that includes the <strong>.ssh</strong> folder in the path. It’s the config file that we have to edit next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/0*dPm-p1Gb3unqrbp1.png" /></figure><p>5. Open config file and add with the following information:</p><pre>Host Docker<br>   #Docker EC2, ip to be changed everytime instance changes<br>    HostName 3.95.1.144<br>    User ubuntu<br>    IdentityFile C:\Users\hadia\Downloads\Linux-Ubuntu.pem</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*IZyQqw80GeMZWV8O.png" /></figure><p>Where…</p><blockquote>Host Docker is just the connection name you want to choose.</blockquote><blockquote>HostName is the public IP address of your Node 1.</blockquote><blockquote>User will always be ubuntu since it’s an ubuntu instance you are connecting to</blockquote><blockquote>IdentityFile is the location of your key pair pem file.</blockquote><p>Save the file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*r4Fi9EboJjCwgL_A.png" /></figure><p>6. Now click on the green icon on the left again and this time when you click it you will see your connection listed. Click it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*NA85PIsGlAOieWsg.png" /></figure><p>It will open a new window for VS code and will ask the type of instance you want to connect to. Since we have created ubuntu instance, we will select Linux. Next it will ask to Continue, press Continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*s2JTEuGWdHkB2jPB.png" /></figure><p>The connection will show progressing, and you will see the green icon with the name of your connection (that you gave in the ssh file) means you’re connected to your EC2 instance successfully via SSH.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*t6RyxkcqSuWQaRb1.png" /></figure><p>7. Now select Terminal from the top -&gt; New Terminal and you will see the EC2 instance id in the prompt (Note this Private IP of the instance).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*fo9_Rf7vfz23KRhq.png" /></figure><p>ssh successfully into node 1</p><p>Success!</p><h4>Change the Hostname:</h4><p>Your current environment will act as your Node 1 instance. Change<br>the hostname so you can easily recognize this. Run the following<br>command as root.</p><pre>sudo hostnamectl set-hostname node1<br>reboot</pre><p>After it rebooted, the hostname changed to Node 1.</p><h4>Modifying the Security Group:</h4><p>We want our nodes to be talking to each other without the use of Public IP address. This is the whole purpose of swarm; nodes are connected to each other and Load balancer distributes traffic between the services seamlessly.</p><p>We will add the Security group rule to allow all TCP traffic within the VPC.</p><p>To get the CIDR range of VPC, go to Networking tab, and open VPC ID in new tab. Copy the CIDR range and go to security group (in Security tab)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*SPXGly3GII7EnCbY.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*6bKoqGo4Z4NkkZGg.png" /></figure><p>Now for the same Node 1 selected, go to Security tab and open SG in new tab. Click Edit Inbound rules and Add rules.</p><p>Select All TCP from VPC CIDR.</p><p>Click save rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*Rzq0HpVQcUrFWaHF.png" /></figure><p>Now back to VS code.</p><h4>Configure SSH file for Node 1</h4><p>We will now configure our ssh configuration file to add Node 2 and Node 3 Private IP addresses so that we can ssh from Node 1 to both Node 2 and Node 3.</p><p>While on Node 1, run…</p><pre>cd .ssh<br>ll</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/0*o6M7A2I7TwkQxA6s.png" /></figure><p>There is no ‘config’ file so we need to create one.</p><pre>vim config</pre><p>Paste the following code in config file</p><pre>Host node2<br>HostName &lt;private IP addr od node 2&gt;<br>User ubuntu<br>IdentityFile /root/.ssh/Linux-Ubuntu.pem<br><br>Host node3<br>HostName &lt;private IP addr od node 3&gt;<br>User ubuntu<br>IdentityFile /root/.ssh/Linux-Ubuntu.pem</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*aFD3-K8mzu5LUMF-.png" /></figure><p>Grab private IP addresses from Node 2 and Node 3</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/660/0*FUVCM-BjAj07KJri.png" /></figure><p>After you’re done editing, press “Esc” and Ctrl+C to exit.</p><p>It will ask you to save your changes, press “:wq!” and it will take you back to CLI.</p><p><strong>Copy the key pair file contents in root folder</strong></p><p>Notice that the Key pair file path you gave in the file is not the location your key is residing, right? Like for me, it’s in my computer C:/Users/hadia/Downloads.</p><p>So, what we need to do next is to copy the contents of key file (open in editor) and paste it in the new file created in the same ssh folder you’re in.</p><p>Create a new file with the same name by using:</p><pre>vim Linux-Ubuntu.pem</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/0*olizNSJR6TYITMKT.png" /></figure><p>Copy the key contents in the ssh folder of Node 1 Ec2</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/0*YDeOK9I_9rpc99vD.png" /></figure><p>Now list the contents of ssh directory and you will see your key pair.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/0*ZXatxklqQ-AkpqTL.png" /></figure><p>Run the command below to avoid all the permission related errors for this key file.</p><pre>chmod 400 &quot;Linux-Ubuntu.pem&quot;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*ShJax9mMl3BPI2-7.png" /></figure><p>you can find it here as well, select EC2 instance, and connect</p><p>Now we will attempt to ssh into node 2.</p><pre>ssh node2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/766/0*ZoZ4x5jTYwD3FU4r.png" /></figure><p>We are getting permission error regarding the file path for the key pair.</p><p>There is a useful command called “readlink” that helps you determine the exact path that these links point to. It’s an essential command for managing symbolic links and understanding their relationships within the file system.</p><p><strong>Syntax:</strong></p><pre>readlink [OPTION] &lt;filename&gt;</pre><p>There could be many options to use, but what we are interested in is “-f”</p><pre>readlink -f Linux-Ubuntu.pem</pre><p>This command will display the resolved path by recursively following all symbolic links up to the final component, even if the final component does not exist.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/487/0*kkLu8ndosxwVt4Di.png" /></figure><p>So, we will copy the full key pair path and edit our ssh file one more time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/0*bpByokumnfHcILvs.png" /></figure><p>File path updated</p><p>Save and exit.</p><p>Now ssh into Node 2 using the command below.</p><pre>ssh node2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/0*D0rdrAPFUqv8-S5q.png" /></figure><p>Notice the hostname has changed to the IP back, it’s because this is Node 2<br>(match the private IP address).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/746/0*_8COa5XqswxQw9wy.png" /></figure><p>Now we also need to change the hostname for this Node to be able to recognize easily</p><p>Use the same command.</p><pre>sudo hostnamectl set-hostname node2<br>reboot</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/613/0*YoFpdHki_W4L_NNh.png" /></figure><p><strong>Troubleshoot</strong></p><p>If you see any failures to reboot, this is what I did.</p><p>Use command <strong><em>sudo su</em></strong> to go as root user, then reboot, and you will be back to node 1 (initial node)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/792/0*o8XvnqVzo_F2ttrw.png" /></figure><p>Now if you ssh again to node2, its hostname will be changed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/0*kA677jhERSUX8kiM.png" /></figure><p>You are now in node2.</p><p>Simply ‘Exit’ to come back to Node 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/371/0*z17w3rsFTSUI_uqk.png" /></figure><p>Now, ssh into node3</p><pre>ssh node3</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/0*GRJnfuXCy94fNEHA.png" /></figure><p>Same as before, we will change the hostname for node3.</p><pre>sudo hostnamectl set-hostname node3<br>sudo su<br>reboot</pre><p>Now all three instances/nodes are ready.</p><p>I would highly suggest splitting the terminal and opening them altogether while working with swarm so it’s easy to get back to other nodes and see what’s happening there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*OMX8dTWPAgQpHbEX.png" /></figure><p>Opened three terminals</p><p>Once the terminals open (notice that it will be all node 1), run ‘ssh node2’ and ‘ssh node3’ in both terminals respectively to see all different nodes.</p><p>All three nodes visible:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*0YoQ4wctm2aGgPJJ.png" /></figure><h4>Verifying that docker is in Active state:</h4><p>we will run docker version command to check if docker is installed.</p><pre>docker version</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/622/0*7oIjAIH1rALfDwDb.png" /></figure><p>Docker version is showing, means its installed. But there is permission warning:</p><pre>permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get “http://%2Fvar%2Frun%2Fdocker.sock/v1.47/version&quot;: dial unix /var/run/docker.sock: connect: permission denied</pre><p>Run below command as this will avoid getting a <strong>docker: permission denied while trying to connect to the Docker daemon socket</strong>…error when trying to execute docker commands.</p><pre>sudo chmod 666 /var/run/docker.sock</pre><p><strong>Warning:</strong></p><p>With the above command, you are giving full permissions to owner/group/others to the docker socket.</p><p>The Docker daemon binds to a Unix socket, not a TCP port. By default, it’s the root user that owns the Unix socket, and other users can only access it using sudo. The Docker daemon always runs as the root user.</p><p><strong>Alternative:</strong></p><p>If you don’t want to preface the docker command with sudo,or give full permissions to all the users, create a Unix group called docker and add users to it (whoever you need to give access). When the Docker daemon starts, it creates a Unix socket accessible by members of the docker group. On some Linux distributions, the system automatically creates this group when installing Docker Engine using a package manager. In that case, there is no need for you to manually create the group. All you have to do is add the user to this group to be able to get root-user privileges and run Docker commands without permission issues.</p><p>To create the docker group and add user:</p><ol><li>Create a docker group.</li></ol><pre>sudo groupadd docker</pre><p>2. Add your user to the docker group.</p><pre>sudo usermod -aG docker $USER</pre><p>3. Log out and log back in so that your group membership is re-evaluated.</p><blockquote><em>If you’re running Linux in a virtual machine, it may be necessary to restart the virtual machine for changes to take effect.</em></blockquote><pre>newgrp docker</pre><p>Now run any docker command without <strong>sudo </strong>to verify.</p><p>Check Docker is installed on both nodes by running docker version command.</p><h3>Creating the Swarm using one master node and two worker nodes</h3><p>We will first verify if the swarm is Active in Docker.</p><p>We don’t need to install swarm (unlike Docker compose), it comes within docker, you just need to activate it</p><p>Run below command to check swarm status:</p><pre>docker info</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/0*pMMYC6ohMMRrM8Ea.png" /></figure><h4>Configure the advertise address</h4><p>Manager nodes use an advertise address to allow other nodes in the swarm access to the Swarmkit API and overlay networking. The other nodes on the swarm must be able to access the manager node on its advertise address.</p><p>If you don’t specify an advertise address, Docker checks if the system has a single IP address. If so, Docker uses the IP address with the listening port 2377 by default. If the system has multiple IP addresses, you must specify the correct --advertise-addr to enable inter-manager communication and overlay networking:</p><pre>docker swarm init --advertise-addr &lt;MANAGER-IP&gt;</pre><blockquote>where Manager-IP is the public IP of Node 1 as shown below.</blockquote><pre>docker swarm init — advertise-addr 172.31.46.184</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*Wieom3otp30kKtJG.png" /></figure><p>The output says that the current node (Node 1) is the manager with a message that if we want to add a worker to this swarm, we must run the highlighted command. (in the screenshot above)</p><p>So, I will copy this docker swarm join command with the token given in node 2 and node3.</p><p>Before we do that, verify on node 1 that this is indeed the leader or manager node. We run:</p><pre>docker node ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*V4OTUdJO1mUlqlKl.png" /></figure><p>Verified that this is the master node with active status.</p><p>Node 2 and 3 joined swarm as worker node from the command below:</p><pre>docker swarm join - token SWMTKN-1–436x0hbevs35xna7w13uhozo7c8sz2jvcr8ytcopjz262gesrx-76t8c74jhha72381123p4yov7 172.31.46.184:2377</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*zfkMH7NjdU5ylFRI.png" /></figure><h4>Verify master and worker nodes:</h4><p>Run the command below to check the status of all nodes.</p><pre>docker node ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*zwPNYa1hoboTl9KW.png" /></figure><p>Node 1 has status as Leader, Node 2 and 3 have no manager status so they are worker nodes.</p><p><strong>Conclusion:</strong></p><p>Once you reach this step where Docker swarm is created with manager and worker nodes, you can proceed further with any of your projects.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e3283118c404" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Continuous Development and Deployment with Docker and Jenkins]]></title>
            <link>https://medium.com/@hadianoor16/continuous-development-and-deployment-with-docker-and-jenkins-2e001a5ab544?source=rss-588e4115da9f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e001a5ab544</guid>
            <dc:creator><![CDATA[Hadianoor]]></dc:creator>
            <pubDate>Sun, 22 Sep 2024 03:56:03 GMT</pubDate>
            <atom:updated>2024-09-22T16:14:33.802Z</atom:updated>
            <content:encoded><![CDATA[<p>Level Up Solutions, a leading provider of technology solutions, is constantly exploring ways to optimize its software development and deployment procedures. The company is keen to adopt tools and methodologies that can streamline their operations, improve productivity, and enhance the quality of their applications. To address these needs, the proposed solution involves leveraging Docker and Jenkins.</p><p><strong>Objective:</strong></p><p>The main objective of this project at Level Up Solutions is to use Docker for creating and managing Jenkins containers, emphasizing the implementation of persistent volumes. This strategy aims to build a robust, scalable Continuous Integration and Continuous Deployment (CI/CD) pipeline, facilitating frequent and reliable deployment of software updates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/0*hjWRk21WrsyJgVR2.png" /></figure><h4>Business Goals and Strategic Outcomes:</h4><p>1. <strong>Enhanced Productivity</strong>: Docker’s containerization approach can reduce the time taken for setting up and configuring Jenkins, thereby improving the productivity of Level Up Solutions’ engineers. The team can focus more on core development tasks and less on environment-related issues.</p><p><strong>2.</strong> <strong>Consistent and Reliable Operations</strong>: Docker containers maintain consistency across multiple development and deployment environments. This will enhance the reliability of applications developed by Level Up Solutions by eliminating discrepancies between different environments.</p><p><strong>3.</strong> <strong>Scalability and Efficiency:</strong> Docker’s capacity to spin up new Jenkins instances as per demand will streamline Level Up Solutions’ CI/CD pipeline, thereby improving its efficiency.</p><p><strong>4. Persistent Data:</strong> By utilizing Docker volumes for Jenkins, Level Up Solutions can ensure the persistence of crucial data such as build history, job configurations, and more, even if the Jenkins container is updated or removed. This implementation will improve the resilience and stability of the company’s CI/CD pipeline.</p><p><strong>5</strong>. <strong>Cost-Effective:</strong> The ability of Docker to run multiple containers on the same host allows Level Up Solutions to optimize resource usage, resulting in a reduction in infrastructure costs.</p><p><strong>6.</strong> <strong>Streamlined Version Control and Updates:</strong> With Docker, containers can be version-controlled and systematically updated. This will make rolling back to previous Jenkins versions easier at Level Up Solutions, if necessary.</p><p>By integrating Docker and Jenkins, Level Up Solutions can establish a resilient, scalable, and efficient solution for its CI/CD requirements. This approach will not only optimize the company’s software development and deployment procedures but also provide significant advantages in terms of cost savings, consistency, and reliability.</p><h4>Using the CLI:</h4><p><strong>Steps to follow:</strong></p><ol><li>Pull the Jenkins LTS image from Docker Hub.</li><li>Create a Docker volume for Jenkins to persist its data.</li><li>Run the container with the volume and correct port mapped to the container.</li><li>Retrieve the Admin password.</li><li>Login and create a new user.</li><li>Launch a new Jenkins container using the same volume and verify that the data has persisted by logging in as the newly created user.</li></ol><p><strong>Prerequisites:</strong></p><ol><li>Running EC2 instance with Docker installed.</li><li>EC2 Security group has port 8080 opened from your IP address.</li><li>SSH into your running EC2 instance and run the command below to avoid any permission related warnings.</li></ol><pre>sudo chmod 666 /var/run/docker.sock</pre><p><strong>Pull Jenkins image:</strong></p><p>Pull Jenkins&#39; latest image from Docker Hub with latest tag.</p><p><a href="https://hub.docker.com/r/jenkins/jenkins">https://hub.docker.com/r/jenkins/jenkins</a></p><pre>docker pull jenkins/jenkins:lts-jdk17</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*NRUV6ODu5RZ-rGMi5eC4uA.png" /></figure><p>Run command ‘<em>docker images</em>’ to verify.</p><p><strong>Create a Docker volume:</strong></p><p>Now we need to create a Docker volume for Jenkins to persist its data.</p><pre>docker volume create jenkins_vol</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*PRXIP94IFe_kR3WrSlklFw.png" /></figure><p><strong>Run the container with mounted volume</strong></p><p>Run the container with the volume and correct port mapped to the container.</p><pre>docker run -d --name jenkins -p 8080:8080 -v jenkins_vol:/var/jenkins_home jenkins/jenkins:lts-jdk17</pre><p>List the container to see its running.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q8G0fRh97NtpcFU6qUoXRA.png" /></figure><p><strong>Verify the volume is mounted</strong>:</p><p>If you inspect the container ID, you will see the new volume is mounted.</p><pre>docker inspect 463</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/846/1*fKChuZMzzV96D6SkOMu1iQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/657/1*dzrXgEkNuhKDLes_TVv93g.png" /><figcaption>Verified that the volume is indeed mounted to the container.</figcaption></figure><p><strong>Verify Jenkins is running</strong></p><p>Copy the EC2 public IP address on browser to see the default page from Jenkins.</p><p>Remember that you mapped port ‘8080’ of your container to port ‘8080’ of the host (in this case EC2), so you must have opened port 8080 in EC2 security group and append ‘:8080’ with public IP address as Jenkins page is served at this port</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sva_d8eYNcwsvat44rFOdw.png" /></figure><p>Here you go..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*41yktABFPlTGGXf47kRhRw.png" /></figure><p>I need to find the admin password that was created with Jenkins container.</p><p><strong>Bash into the container</strong></p><p>Note: If you run the container without the ‘-d’ flag, you will be able to see the admin password in the console like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*uDdA7fEus_7tuFi6KHx1Jw.png" /></figure><p>If you ran the container with the <strong>-d</strong> flag, you need to bash into the container to retrieve the admin password.</p><pre>docker exec -it 463 bash</pre><blockquote>where 463 are the first three digits of container ID</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YPHdqAse8AgGRjeYwFefOA.png" /></figure><p>Now list all the contents of home using ‘ls’ command.</p><p>This is how I cd’ed into each folder one by one to check the contents. There is no harm to explore :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4zqZLuMEBw87MNki5n5bhA.png" /></figure><p>so, the full path to the admin password is:</p><pre>cd /var/jenkins_home/secrets</pre><p>and when you list the contents of secrets folder, you will be able to see “InitialAdminPassword”.</p><p>Use ‘cat’ command to see the contents of this “InitialAdminPassword”. Copy the password.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*8aiqLz4F2b83d959p8RGGA.png" /></figure><p><strong>Create a new user in Jenkins:</strong></p><p>Paste the admin password you noted before and press continue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xNmqv4JW2m9I-ubU9vl-2g.png" /></figure><p>On the next page I chose “Install Suggested Plugins” It will take time to install.</p><p>We will be presented with this page to create a new user. Input all the information and then click ‘Save and Continue’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q91K_NtMiiNMW1ECOQbbiQ.png" /></figure><p>It auto populated the instance configuration information for me.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*08Eqevqbl7IbRrSlFAag3w.png" /></figure><p>Just click ‘Save and Finish’.</p><p>It’s ready to be used. Click ‘Start using Jenkins’.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*bYs27b0wRO4n4N-3dI03iw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r-hqkYYW31SnQVSwf0dRSg.png" /></figure><p>You will see its dashboard.</p><p>Now what if you want to run a new container with the same configuration example as the user that you created?</p><p>It&#39;s possible because you mounted a volume in the beginning to this container, you can mount this volume to any new container and all the information for user and configuration should persist.</p><p><strong>Create a new container with the same volume:</strong></p><p>First, I stopped and removed the old Jenkins container to avoid any confusion.</p><pre>docker stop &lt;container ID&gt;<br>docker rm &lt;container ID&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/1*fItFgsWrTTJ-PXbLMduFwQ.png" /><figcaption>No container running</figcaption></figure><p>Now spinning up a new container with same mount volume.</p><pre>docker run -d --name jenkins_new -p 8080:8080 -v jenkins_vol:/var/jenkins_home jenkins/jenkins:lts-jdk17</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j66uFMBXhH9LNqFSDo-CKw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BqSUwUOg5cVlZ6XOFlV7ew.png" /></figure><p>Now if you copy the public IP address of EC2 with :8080 appended, you’ll see the login page directly instead of the configuration setup page.</p><p>It&#39;s because the volume you mounted from the previous container has all the configuration setup for Jenkins.</p><p>Use the username and password from the last container you created, and it should login</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6xH2ja5PtodqnW37X9Hdqw.png" /></figure><p>You are signed in successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ngIPa-A-ik6Uml3n3W6f8w.png" /></figure><h4>Docker compose to run Jenkins:</h4><p>You now need to do all the above steps but with docker compose file.</p><ol><li>First remove the volume you created, to avoid any confusion.</li></ol><pre>docker volume rm jenkins_vol</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*92C6ypyaU0Bq7FuGRaASIw.png" /></figure><p>2. Create a directory, so all your work stays within the directory.</p><pre>mkdir jenkins</pre><p>List to see the directory.</p><pre>ls</pre><p>3. Now open the folder from the left side of VS code and choose the directory you created. Press OK.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MgTiFPEFrplFIMw4DkCacg.png" /></figure><p>4. Create a new file under this directory with name ‘docker-compose.yml’</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/422/1*tdy4uJrWjQ8J2qCIyhOkqQ.png" /></figure><p>Open this file and it will open at the top, with our terminal at the bottom.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e1yaObaCScoEcdZ3TMWj-Q.png" /></figure><p>Start composing your docker file.</p><pre>version: &#39;3.7&#39;<br>services:<br>  jenkins:<br>    image: jenkins/jenkins:lts-jdk17<br>    ports:<br>      - 8080:8080<br>    volumes:<br>      - jenkins_vol:/var/jenkins_home<br>    <br>volumes:<br>  jenkins_vol:</pre><blockquote>Version is deprecated, you can remove this line from code</blockquote><blockquote>Services defines what services will be running as we run this file and jenkins is the name I gave to my service. So, it’s under the services section.</blockquote><blockquote>Port exposes the port of this container to the host</blockquote><blockquote>Volumes creates a volume and mount it to the directory</blockquote><blockquote>In the end, whatever resources are created as part of the docker compose file, you need to declare it globally. In this case only the volume is created so I am defining it at the bottom.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*GMQ-WZbZhGUfJiBn1hGKJg.png" /></figure><p>Save the file.</p><p>5. Use the command below to run this file in the terminal.</p><pre>docker compose up</pre><p>It will start running all the necessary configurations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QXdAnIltgRin1VMBTOOElg.png" /></figure><p>6. Copy the admin password from the console.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*HqlTe4ezm5cdjrOHli4cvg.png" /></figure><p>7. Now copy the public IP address of EC2 in browser with 8080 at the end and see the Jenkins default page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UkPKwJ7va1K2VMXbm86Ibw.png" /></figure><p>It will first ask for Admin password just like before.</p><p>It went through the same setup of installing plugins and creating user and is now ready to be used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GRuS-2_xNfxeL6vjimWkLw.png" /></figure><p>8. Now if you want to remove the container, simply run:</p><pre>docker compose down</pre><p>Notice that when you run this command, it removes the container and the network (if created with this compose file). It won’t remove any volumes or any images that are downloaded.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4cFrlQDl-gbqzE-dOs-tfA.png" /></figure><p>So, you can re-use the mount volume to create a new container using the docker- compose file.</p><p><strong>Summary:</strong></p><p>Hence, you saw how we can optimize our time by making use of persistent volumes. I hope this tutorial demonstrates the increased efficiency and control made possible by docker compose.</p><p>If you like this article and want to connect, please find me on <a href="https://www.linkedin.com/in/hadianoor/">Linkedin</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e001a5ab544" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>