<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[DevOps Manuals and Technical Notes - Medium]]></title>
        <description><![CDATA[Here I’m publishing DevOps-related technical manuals and notes. I’m writing articles aimed at junior DevOps engineers or those aspiring to become DevOps engineers. - Medium]]></description>
        <link>https://medium.com/devops-technical-notes-and-manuals?source=rss----07c04f098b7e---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>DevOps Manuals and Technical Notes - Medium</title>
            <link>https://medium.com/devops-technical-notes-and-manuals?source=rss----07c04f098b7e---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 03:58:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/devops-technical-notes-and-manuals" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[DevOps Example Project with Source Code. AWS, Terraform, Jenkins, Docker, GitHub Actions, Nginx.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/devops-example-project-for-your-resume-db45a77d6607?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/db45a77d6607</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Fri, 31 Jan 2025 11:05:13 GMT</pubDate>
            <atom:updated>2025-08-23T09:58:27.086Z</atom:updated>
            <content:encoded><![CDATA[<h3>DevOps Example Project with Source Code. AWS, Terraform, Jenkins, Docker, GitHub Actions, Nginx, and Kubernetes. [2025]</h3><p>One of the best ways to show off your DevOps skills is to create an example project and include it in your portfolio.</p><p>Executing this project, you will use GitHub Actions, Docker, Sonar Cloud, Snyk, AWS services, Terraform, and Kubernetes.</p><p>The project idea comes from a real-life exercise I received as homework during one of my job interview processes.</p><blockquote>Note that most probably during the project implementation, you will face an issues that you will need to solve by yourself. Use the official documentation, Google, Stack Overflow, or ChatGPT to search for the solution. It’s a part of the learning process and imitates real-life DevOps challenges. The ability to find the necessary information is one of the most important skills for a DevOps engineer.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KqkRh0ys3KOHZ7mgGGE5Sw.png" /></figure><p>This is my second DevOps example project.<br>To access the first one, read here: <a href="https://medium.com/devops-technical-notes-and-manuals/devops-example-project-for-your-resume-198e34d874b4">The First DevOps Example Project</a>.</p><p>The projects are completely independent of one another.</p><h4>The goal of the exercise is to create a Jenkinsfile that will do the following:</h4><ul><li>Build a Docker Image from Python application on EC2-Instance-1 (Build Node) and push it to Docker Hub.</li><li>Run the Docker Image on EC2-Instance-2 (Deploy Node) as a Kubernetes POD and expose the POD to port 443.</li></ul><h4>Prerequisites.</h4><p>Basic understanding in:</p><ul><li>CI/CD: <a href="https://medium.com/devops-technical-notes-and-manuals/what-is-ci-cd-and-why-we-need-it-guide-for-devops-engineer-d81dcf1abd1f">read here</a></li><li><a href="https://git-scm.com/">git</a></li><li><a href="https://github.com/features/actions">GitHub Actions</a></li><li><a href="https://docs.aws.amazon.com/ec2/?icmpid=docs_homepage_featuredsvcs">AWS EC2</a></li><li><a href="https://nginx.org/en/">Nginx</a></li><li><a href="https://docs.docker.com/">Docker</a></li><li><a href="https://kubernetes.io/docs/home/">Kubernetes</a></li><li><a href="https://www.jenkins.io/doc/">Jenkins</a></li><li><a href="https://www.terraform.io/">Terraform</a></li><li><a href="https://sonarcloud.io/">Sonar Cloud</a></li><li><a href="https://snyk.io/">Snyk</a></li></ul><p><strong>Free accounts:</strong></p><ul><li>GitHub account:<br><a href="https://github.com/">https://github.com/</a></li><li>AWS account:<br><a href="https://aws.amazon.com/free/">https://aws.amazon.com/free/</a></li><li>Docker Hub account:<br><a href="https://hub.docker.com/">https://hub.docker.com/</a></li><li>Sonar Cloud account:<br><a href="https://www.sonarsource.com/products/sonarcloud/signup/">https://www.sonarsource.com/products/sonarcloud/signup/</a></li><li>Snyk account:<br><a href="https://app.snyk.io/login/">https://app.snyk.io/login/</a></li></ul><p><strong>Optionally:</strong></p><ul><li>Route 53 domain.<br><a href="https://aws.amazon.com/route53/">https://aws.amazon.com/route53/</a></li></ul><p><strong>Install on your local PC:</strong></p><ul><li><a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git">git</a></li><li><a href="https://docs.docker.com/engine/install/">Docker</a></li><li><a href="https://www.python.org/downloads/">Python</a></li><li><a href="https://code.visualstudio.com/">Visual Studio Code</a> or any other source-code editor you prefer</li></ul><h4>Project goals and definitions:</h4><ul><li>Create a simple Python application.</li><li>Create a Jenkins declarative pipeline that builds Docker Image from Python application on server-01, then runs the created Docker Image as a POD on server-02.</li><li>Expose the application to the world on port 443.</li><li>Avoid using ChatGPT. Use the official documentation instead. Trust me, it’s the only way to learn.<br>Besides, ChatGPT often makes mistakes, <a href="https://medium.com/@andreybyhalenko/can-you-really-write-your-devops-code-using-chatgpt-a1d7b638e378">read about it here</a>.</li></ul><h4>Project implementation steps:</h4><ol><li>Create a GitHub repository on <a href="https://github.com/">github.com</a> and clone it to your pc.</li><li>Develop a simple Python application and test it locally.</li><li>Create a Dockerfile for the Python app, build a Docker Image for it, run the Docker Container locally to verify it works.</li><li>Create Kubernetes yaml files for the Python app.</li><li>Push your code changes to GitHub.</li><li>Create an EC2 Instance for the Jenkins server, install Docker, and run Jenkins as a Docker container.</li><li>Create an EC2 Instance (server-01) that will build the Python app’s Docker Image and push it to Docker Hub. Install Docker on it.</li><li>Create an EC2 Instance (server-02) that will run the Python app container on POD. Install <a href="https://minikube.sigs.k8s.io/docs/start/?arch=%2Fwindows%2Fx86-64%2Fstable%2F.exe+download">Minikube</a> on it.</li><li>Create a Docker Hub repository.</li><li>Set up Jenkins.</li><li>Create a Jenkinsfile.</li><li>Add the Sonar Cloud and Snyk tests to the GitHub repo using GitHub Actions.</li><li>Connect the AWS Route 53 domain name to the server-02 EC2 Instance. This is an optional step, as a domain name costs money. If you don’t have any domain and don’t want to purchase one, you can skip this step, but you will need to change the Jenkinsfile and exclude the Domain Name certificate creation.</li></ol><h4>Notes:</h4><ul><li>You can automate everything in this exercise using Jenkins, including the EC2 Instances launch. Jenkins can interact with AWS API directly, <a href="https://plugins.jenkins.io/pipeline-aws/">read about it here</a>. However, one of the goals of this exercise is Terraform practice.</li><li>I choose to use NodePort Service and Nginx to expose the app.</li><li>Build Node and Deploy Node will run on Ubuntu 24.04 EC2 Instances.</li><li>The Jenkins server will run on Ubuntu 24.04 EC2 Instance as a Docker Container.</li><li>I’m using a public Docker repository for this project because of the free Docker account limitation. However, this approach is acceptable for learning or POC purposes only. You should always create private repositories in the real world.</li><li>I’m using the same SSH key for all three EC2 Instances. I also use the same key as a Jenkins credential. This approach is against the best practices and security standards. In the real world, I would create an SSH key for each Instance and a separate SSH key for Jenkins. Read about the <a href="https://repost.aws/knowledge-center/ec2-ssh-best-practices">EC2 SSH best practices here</a>.</li><li>My GitHub repository is also public, which is ok for training. In the real world you should use private repositories only.</li><li>I’m using the “latest” tag for my Docker Images. You should avoid using the “latest” tag in your working environments.</li><li>Everything I define in this project (such as a Docker Image name, tag, versions, etc.) can be dynamic (environment variables). I make it static for training purposes because the goal is to show how Jenkins and Terraform work.</li></ul><h4>Step 1: Create a GitHub repository and clone it to your pc.</h4><p>If you want to see my final code for this exercise, it’s here in this repository:<br><a href="https://github.com/Andrey-B-lab/exercise-python-application">https://github.com/Andrey-B-lab/exercise-python-application</a></p><p>Instead of forking mine, you should create an empty repository and start from the beginning. The only way to learn is to do it by yourself.</p><p>So create an empty repository, <strong>clone it</strong> to your local PC, and write the code yourself.</p><p><a href="https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-new-repository">How to create a new GitHub repository.</a></p><h4>Step 2: Develop a simple Python application and test it locally.</h4><blockquote><em>You should always create a virtual environment when developing new applications.</em></blockquote><blockquote><em>Read about it here: </em><a href="https://docs.python.org/3/library/venv.html"><em>https://docs.python.org/3/library/venv.html</em></a><em>.</em></blockquote><p>Write a very simple Python service that shows the “It’s a Flask application running on POD!” output in the browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/1*8q7Kd-qt4npVTEMnj-kpTw.png" /></figure><p>Call the service app.py(or any other name you prefer).</p><p>It’s very simple, and it’s only returns “It’s a Flask application running on POD!”.</p><pre>from flask import Flask<br>import os<br><br>app = Flask(__name__)<br><br>@app.route(&quot;/&quot;)<br>def hello():<br>    return &quot;It&#39;s a Flask application running on POD!&quot;<br><br>if __name__ == &quot;__main__&quot;:<br>    app.run(host=&#39;0.0.0.0&#39;, port=5001, debug=False)</pre><p>Test it locally.</p><p>Verify that you are in the project directory.<br>Execute python3 app.py.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZZf3Jdhvzq0DYz7v-ZCdjQ.png" /></figure><p>Now open <a href="http://127.0.0.1:5001/">http://127.0.0.1:5001</a> on your browser, and you should see the app output.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/1*oeFevqHZSK2n7boV1GOKNg.png" /></figure><p>It works locally. Now create a Docker Image for the application.</p><h4>Step 3: Create a Dockerfile for the Python app, build a Docker Image, run and test it locally.</h4><p>If you don’t have Docker installed on your PC, follow the<a href="https://docs.docker.com/engine/install/"> instructions here</a>.</p><p>Instead of creating the Dockerfile manually, you can use the docker init command.</p><p>The docker init utility will walk you through creating the following files with sensible defaults for your project:</p><ul><li>.dockerignore</li><li>Dockerfile</li><li>compose.yaml</li><li>README.Docker.md</li></ul><p>Just type docker init in the terminal in the same folder where app.py is, and follow the instructions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NS_0jWGKqmoHq4lOHaBdhQ.png" /></figure><p>Once you have the Dockerfile in the app.py directory, create a Docker Image and run the Container.</p><p>Type and run docker compose up --build.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iEYQDJ0wYszd34MlaQIBSA.png" /></figure><p>Now proceed to <a href="http://127.0.0.1:5001/">http://127.0.0.1:5001</a> on your browser, and you should see the app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*dU08uokEqyRsBSt8PFeDCA.png" /></figure><p>Good job, you developed an app, created the Dockerfile, and verified both are working.</p><p>Move on to the next step.</p><h4>Step 4: Create Kubernetes yaml files for the Python app.</h4><p>In order to run the Docker container on Kubernetes, you need a <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployment</a> to run an application workload and <a href="https://kubernetes.io/docs/concepts/services-networking/service/">Service</a> to expose the application.</p><p>I created separate folder for all Kubernetes files inside the GitHub repository root folder.</p><p>Here is thedeployment.yml:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: exercise-python-application<br>  labels:<br>    app: exercise-python-application<br>spec:<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      app: exercise-python-application<br>  template:<br>    metadata:<br>      labels:<br>        app: exercise-python-application<br>    spec:<br>      containers:<br>      - name: python-flask-app<br>        image: andreybyhalenko/exercise-python-application:latest<br>        resources:<br>          requests:<br>            memory: &quot;64Mi&quot;<br>            cpu: &quot;250m&quot;<br>          limits:<br>            memory: &quot;128Mi&quot;<br>            cpu: &quot;500m&quot;<br>        ports:<br>        - containerPort: 5000</pre><p>Here is theservice.yml:</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: exercise-python-service<br>spec:<br>  selector:<br>    app: exercise-python-application<br>  ports:<br>    - protocol: TCP<br>      port: 80<br>      targetPort: 5000<br>      nodePort: 30443<br>  type: NodePort</pre><p>Note that service type is <a href="https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport">NodePort</a>.</p><h4>Step 5: Push your code changes to GitHub.</h4><p>If you are working with Visual Studio Code, you can push the changes using GUI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GNQ22OFyc1CAQgm6uTTnng.png" /></figure><p>If you are working inside the terminal, <a href="https://docs.github.com/en/get-started/using-git/pushing-commits-to-a-remote-repository">read about pushing commits to a remote repository</a>.</p><h4>Step 6: Create an EC2 Instance for the Jenkins server, install Docker on it, and run Jenkins as a Docker container.</h4><p>Create a Terraform files that will launch an EC2 Instance, install Docker on it, and run Jenkins as a Docker container.</p><p>I decided to split my Terraform configurations into multiple files:</p><ul><li><strong>main.tf </strong>file contains the constant information of my Terraform configuration.</li></ul><blockquote>I’m planning to run only one EC2 Instance, so theoretically I can include all the content from the jenkins-ec2.tf file, but I decided to split it for two reasons. First, it’s a best practice. Second, I’ll do it for future use. What if I will need to add another Instance? So I keep only provider and a region in main.tf.</blockquote><ul><li><strong>jenkins-ec2.tf</strong> file contains the AWS Instance resource (EC2 Instance).</li></ul><blockquote>This file will create an EC2 Ubuntu 22.04 Instance, install Docker, start Docker, and run a Jenkins Image on it.</blockquote><ul><li><strong>networking.tf</strong> file contains the AWS Security Group definition.</li></ul><blockquote>I’m allowing all traffic from my laptop IP in order to access Jenkins and EC2 Instance as well if needed.</blockquote><ul><li><strong>outputs.tf</strong> file contains outputs from the resources created in jenkins-ec2.tf.</li></ul><blockquote>I need to know the EC2 Instance public IP in order to open Jenkins in my browser, so I output it.</blockquote><ul><li><strong>variables.tf</strong> file contains declarations of variables used in jenkins-ec2.tf and networking.tf.</li></ul><blockquote>I will define my IP and key pair name as variables because I will upload my code to Git and I don’t want to expose those parameters.</blockquote><ul><li><strong>terraform.tfvars</strong> file contains the values for my variables.</li></ul><blockquote>I will add this file to .gitignore on my Git repository, so it will not be uploaded to Git.</blockquote><h4>Here are the Terraform files:</h4><p><strong>main.tf</strong></p><pre>provider &quot;aws&quot; {        # provider definition<br>  region = &quot;us-east-1&quot;  # the region I run the EC2 instance<br>}<br><br>// You need this if you use `data.aws_vpc.default.id<br>data &quot;aws_vpc&quot; &quot;default&quot; {<br>  default = true<br>}</pre><p><strong>jenkins-ec2.tf</strong></p><pre>resource &quot;aws_instance&quot; &quot;jenkins_server&quot; {<br>  ami           = &quot;ami-0e86e20dae9224db8&quot; # check for the correct ami for the region<br>  instance_type = &quot;t2.micro&quot;<br><br>  vpc_security_group_ids = [aws_security_group.my_ip_ssh.id]<br>  key_name               = var.key_pair_name  # taken from variables.tf<br><br>  root_block_device {<br>    volume_size = 10  # 10GB is a recommended minimum if running Jenkins as a Docker container<br>    volume_type = &quot;gp3&quot;<br>  }<br><br>  tags = {<br>    Name = &quot;JenkinsServer&quot;<br>  }<br><br>user_data = &lt;&lt;-EOF<br>            #!/bin/bash<br>            # Update the package index<br>            sudo apt-get update -y<br><br>            # Install prerequisite packages<br>            sudo apt-get install -y \<br>                apt-transport-https \<br>                ca-certificates \<br>                curl \<br>                software-properties-common<br><br>            # Add Docker’s official GPG key<br>            curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg<br><br>            # Add Docker’s official APT repository<br>            echo \<br>              &quot;deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \<br>              $(lsb_release -cs) stable&quot; | sudo tee /etc/apt/sources.list.d/docker.list &gt; /dev/null<br><br>            # Update the package index again to include Docker packages<br>            sudo apt-get update -y<br><br>            # Install Docker<br>            sudo apt-get install -y docker-ce docker-ce-cli containerd.io<br><br>            # Start Docker service<br>            sudo systemctl start docker<br><br>            # Enable Docker to start on boot<br>            sudo systemctl enable docker<br><br>            # Add the ec2-user to the docker group to run Docker without sudo<br>            sudo usermod -aG docker ubuntu<br><br>            # Pull the Jenkins Docker Image<br>            sudo docker pull jenkins/jenkins:lts<br><br>            # Create a Jenkins directory for persistent storage<br>            sudo mkdir -p /var/jenkins_home<br>            sudo chown -R 1000:1000 /var/jenkins_home<br><br>            # Run Jenkins in a Docker container<br>            sudo docker run -d -p 8080:8080 -p 50000:50000 \<br>              --name jenkins \<br>              -v /var/jenkins_home:/var/jenkins_home \<br>              jenkins/jenkins:lts<br>            EOF<br>}</pre><p><strong>networking.tf</strong></p><pre>resource &quot;aws_security_group&quot; &quot;my_ip_ssh&quot; {<br>  name        = &quot;sg-my-ip-ssh&quot;<br>  description = &quot;Allow SSH from my IP&quot;<br>  vpc_id      = data.aws_vpc.default.id<br><br>  tags = {<br>    Name = &quot;sg-my-ip-ssh&quot;<br>  }<br>}<br><br>resource &quot;aws_vpc_security_group_ingress_rule&quot; &quot;my_ip_ssh_ingress&quot; {<br>  security_group_id = aws_security_group.my_ip_ssh.id<br>  description       = &quot;SSH from my IP&quot;<br>  from_port         = 22<br>  to_port           = 22<br>  protocol          = &quot;tcp&quot;<br>  cidr_blocks       = [&quot;${var.my_ip_address}/32&quot;]<br>}<br><br>resource &quot;aws_vpc_security_group_egress_rule&quot; &quot;my_ip_ssh_egress&quot; {<br>  security_group_id = aws_security_group.my_ip_ssh.id<br>  description       = &quot;Allow all outbound&quot;<br>  from_port         = 0<br>  to_port           = 0<br>  protocol          = &quot;-1&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}</pre><p><strong>outputs.tf</strong></p><pre>output &quot;instance_ip&quot; {    # I will see the instance public IP as an output in my console<br>  value = aws_instance.jenkins_server.public_ip<br>}</pre><p><strong>variables.tf</strong></p><pre># the values are in terraform.tfvars file<br>variable &quot;my_ip_address&quot; {<br>  description = &quot;My IP address&quot;<br>  type        = string<br>  sensitive   = true<br>}<br><br>variable &quot;key_pair_name&quot; {<br>  description = &quot;My key pair name&quot;<br>  type        = string<br>  sensitive   = true<br>}</pre><p><strong>terraform.tfvars</strong></p><pre>my_ip_address = &quot;your-IP-address&quot;   # Replace with your IP address<br>key_pair_name = &quot;your-ec2-key&quot;   # Replace with your key-pair name</pre><p>Note that terraform.tfvars file contains sensitive data and should be a part of the <a href="https://git-scm.com/docs/gitignore">.gitignore</a> file.</p><p><a href="https://github.com/github/gitignore/blob/main/Terraform.gitignore">Here you can find</a> the example of Terraform .gitignore file.</p><p>Now apply these Terraform configurations:</p><ul><li>Run <strong>terraform init</strong> to initialize the Terraform configurations.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dMiAd9N3mmQMxTFpOq4Ljg.jpeg" /></figure><ul><li>Run <strong>terraform plan</strong> to preview the changes Terraform will make before you apply them.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H9nA3NzH6TVFdE7bkbP3iQ.jpeg" /></figure><ul><li>Run <strong>terraform apply</strong> to create the defined resources.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CqUinTtLIDGGhl_pH08Mpg.jpeg" /></figure><p>As you see, there is an instance_ip in the Outputs.</p><p>You should be able to access Jenkins by typing the instance IP followed by port 8080 in your browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XMBGDDi7ePeAThm2AlL0pA.jpeg" /></figure><p>You can get the Unlock Jenkins password in the Docker container log.<br>You also can include the command to output the Docker logs in your Terraform file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gb0QaawRe0KbdIi5DQ76Dw.jpeg" /></figure><p>Once you apply the password, unlock Jenkins, install the recommended plugins, and you&#39;re ready to go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A1CslPcie_YsOFsFbyeEPQ.jpeg" /></figure><h4>Step 7: Create an EC2 Instance (server-01) that will build the Python app’s Docker Image and push it to Docker Hub. Install Docker on it.</h4><p>Use Terraform as well to do that.</p><p>You can use the same *.tf files from the previous step, all you need is to add another Terraform file dedicated to server-01 and add new networking file for security group that will let Jenkins to access the EC2 instance.</p><p>Also add jenkins_ip_address variable to the <strong>variables.tf </strong>and define it in<strong> terraform.tfvars.</strong></p><h4>Here is the Terraform files:</h4><p><strong>server-01.tf</strong></p><pre>resource &quot;aws_instance&quot; &quot;docker-server-01&quot; {<br>  ami           = &quot;ami-0e86e20dae9224db8&quot;<br>  instance_type = &quot;t2.micro&quot;<br>  key_name      = var.key_pair_name<br>  <br>  vpc_security_group_ids = [<br>    aws_security_group.jenkins_access.id,<br>    aws_security_group.my_ip_ssh.id<br>  ]<br>  tags = {<br>    Name = &quot;docker-server-1&quot;<br>  }<br>user_data = &lt;&lt;-EOF<br>            #!/bin/bash<br>            # Update the package index<br>            sudo apt-get update -y<br><br>            # Install prerequisite packages<br>            sudo apt-get install -y \<br>                apt-transport-https \<br>                ca-certificates \<br>                curl \<br>                software-properties-common<br>            # -----------------------------<br>            ####    INSTALL DOCKER     ####<br>            # -----------------------------<br>            # Add Docker’s official GPG key<br>            curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg<br>            <br>            # Add Docker’s official APT repository<br>            echo \<br>              &quot;deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \<br>              $(lsb_release -cs) stable&quot; | sudo tee /etc/apt/sources.list.d/docker.list &gt; /dev/null<br>            <br>            # Update the package index again to include Docker packages<br>            sudo apt-get update -y<br><br>            # Install Docker<br>            sudo apt-get install -y docker-ce docker-ce-cli containerd.io<br><br>            # Start Docker service<br>            sudo systemctl start docker<br><br>            # Enable Docker to start on boot<br>            sudo systemctl enable docker<br><br>            # Add the ec2-user to the docker group to run Docker without sudo<br>            sudo usermod -aG docker ubuntu<br><br>            EOF<br>}</pre><p><strong>networking.tf</strong></p><blockquote>In this example I added new resources to the same networking.tf file.</blockquote><pre>resource &quot;aws_security_group&quot; &quot;my_ip_ssh&quot; {<br>  name        = &quot;sg-my-ip-ssh&quot;<br>  description = &quot;Allow SSH from my IP&quot;<br>  vpc_id      = data.aws_vpc.default.id<br><br>  tags = {<br>    Name = &quot;sg-my-ip-ssh&quot;<br>  }<br>}<br><br>resource &quot;aws_vpc_security_group_ingress_rule&quot; &quot;my_ip_ssh_ingress&quot; {<br>  security_group_id = aws_security_group.my_ip_ssh.id<br>  description       = &quot;SSH from my IP&quot;<br>  from_port         = 22<br>  to_port           = 22<br>  protocol          = &quot;tcp&quot;<br>  cidr_blocks       = [&quot;${var.my_ip_address}/32&quot;]<br>}<br><br>resource &quot;aws_vpc_security_group_egress_rule&quot; &quot;my_ip_ssh_egress&quot; {<br>  security_group_id = aws_security_group.my_ip_ssh.id<br>  description       = &quot;Allow all outbound&quot;<br>  from_port         = 0<br>  to_port           = 0<br>  protocol          = &quot;-1&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}<br><br>resource &quot;aws_security_group&quot; &quot;jenkins_access&quot; {<br>  name        = &quot;sg-jenkins-access&quot;<br>  description = &quot;Allow SSH from Jenkins IP&quot;<br>  vpc_id      = data.aws_vpc.default.id<br><br>  tags = {<br>    Name = &quot;sg-jenkins-access&quot;<br>  }<br>}<br><br>resource &quot;aws_vpc_security_group_ingress_rule&quot; &quot;jenkins_access_ingress&quot; {<br>  security_group_id = aws_security_group.jenkins_access.id<br>  description       = &quot;SSH from Jenkins server&quot;<br>  from_port         = 22<br>  to_port           = 22<br>  protocol          = &quot;tcp&quot;<br>  cidr_blocks       = [&quot;${var.jenkins_ip_address}/32&quot;]<br>}<br><br>resource &quot;aws_vpc_security_group_egress_rule&quot; &quot;jenkins_access_egress&quot; {<br>  security_group_id = aws_security_group.jenkins_access.id<br>  description       = &quot;Allow all outbound&quot;<br>  from_port         = 0<br>  to_port           = 0<br>  protocol          = &quot;-1&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}</pre><p><strong>variables.tf</strong></p><pre># the values are in terraform.tfvars file<br>variable &quot;my_ip_address&quot; {<br>  description = &quot;My IP address&quot;<br>  type        = string<br>  sensitive   = true<br>}<br><br>variable &quot;key_pair_name&quot; {<br>  description = &quot;My key pair name&quot;<br>  type        = string<br>  sensitive   = true<br>}<br><br>variable &quot;jenkins_ip_address&quot; {<br>  description = &quot;Jenkins IP address&quot;<br>  type        = string<br>  sensitive   = true<br>}</pre><p><strong>terraform.tfvars</strong></p><pre>my_ip_address = &quot;your-IP-address&quot;   # Replace with your IP address<br>key_pair_name = &quot;your-ec2-key&quot;   # Replace with your key-pair name<br>jenkins_ip_address = &quot;jenkins-IP-address&quot;   # Replace with Jenkins IP address</pre><p>If you run terraform apply, Terraform will launch another EC2 instance with the name &quot;docker-server-1,&quot; and install/run the Docker engine on it.</p><h4>Step 8: Create an EC2 Instance (server-02) that will run the Python app container on POD. Install Minikube on it.</h4><p>You can also use Terraform for that, all you need is to create one more file for the server-02, and you need to create a new file for the security group to allow access to this server for anyone on port 443 (and also keep access to the server on SSH from your IP and access from Jenkins).</p><p>The file should launch the EC2 Instance with the <a href="https://minikube.sigs.k8s.io/docs/start/?arch=%2Fmacos%2Farm64%2Fstable%2Fbinary+download">minimum requirements for minikube</a>. Then it should install the Docker engine, install kubectl, and install Minikube.</p><p>I&#39;m using t3.medium Instance type because I need a more powerful machine for Minikube as I&#39;m running different projects on it, you should choose the minimum compatible machine.</p><blockquote><strong><em>Note that t3.medium costs money.</em></strong></blockquote><p><strong>server-02.tf</strong></p><pre>resource &quot;aws_instance&quot; &quot;minikube-server-02&quot; {<br>  ami           = &quot;ami-0e86e20dae9224db8&quot;<br>  instance_type = &quot;t3.medium&quot;<br>  vpc_security_group_ids = [<br>    aws_security_group.my_ip_ssh.id,<br>    aws_security_group.jenkins_access.id,<br>    aws_security_group.https_world.id<br>  ]<br>  key_name               = &quot;${var.key_pair_name}&quot;<br>  root_block_device {<br>    volume_size = 20  ## 20GB is the minimum requirements for minikube<br>    volume_type = &quot;gp3&quot;<br>  }<br>  tags = {<br>    Name = &quot;minikube-server-2&quot;<br>  }<br>user_data = &lt;&lt;-EOF<br>            #!/bin/bash<br>            # Update the package index<br>            sudo apt-get update -y<br><br>            # Install prerequisite packages<br>            sudo apt-get install -y \<br>                apt-transport-https \<br>                ca-certificates \<br>                curl \<br>                software-properties-common<br><br>            # -----------------------------<br>            ####    INSTALL DOCKER     ####<br>            # -----------------------------<br>            # Add Docker&#39;s official GPG key<br>            curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg<br>            <br>            # Add Docker&#39;s official APT repository<br>            echo \<br>              &quot;deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \<br>              $(lsb_release -cs) stable&quot; | sudo tee /etc/apt/sources.list.d/docker.list &gt; /dev/null<br>            <br>            # Update the package index again to include Docker packages<br>            sudo apt-get update -y<br><br>            # Install Docker<br>            sudo apt-get install -y docker-ce docker-ce-cli containerd.io<br><br>            # Start Docker service<br>            sudo systemctl start docker<br><br>            # Enable Docker to start on boot<br>            sudo systemctl enable docker<br><br>            # Add the ec2-user to the docker group to run Docker without sudo<br>            sudo usermod -aG docker ubuntu<br><br>            # -----------------------------<br>            ###    INSTALL KUBECTL    ####<br>            # -----------------------------<br>            # Install kubectl binary with curl on Linux<br>            curl -LO &quot;https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl&quot;<br><br>            # Validate the binary (optional)<br>            # Download the kubectl checksum file:<br>            curl -LO &quot;https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256&quot;<br>            <br>            # Validate the kubectl binary against the checksum file:<br>            echo &quot;$(cat kubectl.sha256)  kubectl&quot; | sha256sum --check<br>            # If valid, the output is: kubectl: OK<br><br>            # Install kubectl:<br>            sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl<br><br>            # Test to ensure the version you installed is up-to-date:<br>            kubectl version --client<br><br>            # -----------------------------<br>            ###    INSTALL MINIKUBE    ####<br>            # -----------------------------<br>            # Download the latest Minikube binary<br>            curl -LO https://github.com/kubernetes/minikube/releases/latest/download/minikube-linux-amd64<br>            sudo install minikube-linux-amd64 /usr/local/bin/minikube &amp;&amp; rm minikube-linux-amd64<br>            <br>            # To make docker the default driver:<br>            minikube config set driver docker<br><br>            # Start a cluster using the docker driver:<br>            minikube start --driver=docker<br><br>            EOF<br>}<br></pre><p><strong>networking.tf</strong></p><blockquote>In this example I added new resources to the same networking.tf file.</blockquote><pre>resource &quot;aws_security_group&quot; &quot;my_ip_ssh&quot; {<br>  name        = &quot;sg-my-ip-ssh&quot;<br>  description = &quot;Allow SSH from my IP&quot;<br>  vpc_id      = data.aws_vpc.default.id<br><br>  tags = {<br>    Name = &quot;sg-my-ip-ssh&quot;<br>  }<br>}<br><br>resource &quot;aws_vpc_security_group_ingress_rule&quot; &quot;my_ip_ssh_ingress&quot; {<br>  security_group_id = aws_security_group.my_ip_ssh.id<br>  description       = &quot;SSH from my IP&quot;<br>  from_port         = 22<br>  to_port           = 22<br>  protocol          = &quot;tcp&quot;<br>  cidr_blocks       = [&quot;${var.my_ip_address}/32&quot;]<br>}<br><br>resource &quot;aws_vpc_security_group_egress_rule&quot; &quot;my_ip_ssh_egress&quot; {<br>  security_group_id = aws_security_group.my_ip_ssh.id<br>  description       = &quot;Allow all outbound&quot;<br>  from_port         = 0<br>  to_port           = 0<br>  protocol          = &quot;-1&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}<br><br>resource &quot;aws_security_group&quot; &quot;jenkins_access&quot; {<br>  name        = &quot;sg-jenkins-access&quot;<br>  description = &quot;Allow SSH from Jenkins IP&quot;<br>  vpc_id      = data.aws_vpc.default.id<br><br>  tags = {<br>    Name = &quot;sg-jenkins-access&quot;<br>  }<br>}<br><br>resource &quot;aws_vpc_security_group_ingress_rule&quot; &quot;jenkins_access_ingress&quot; {<br>  security_group_id = aws_security_group.jenkins_access.id<br>  description       = &quot;SSH from Jenkins server&quot;<br>  from_port         = 22<br>  to_port           = 22<br>  protocol          = &quot;tcp&quot;<br>  cidr_blocks       = [&quot;${var.jenkins_ip_address}/32&quot;]<br>}<br><br>resource &quot;aws_vpc_security_group_egress_rule&quot; &quot;jenkins_access_egress&quot; {<br>  security_group_id = aws_security_group.jenkins_access.id<br>  description       = &quot;Allow all outbound&quot;<br>  from_port         = 0<br>  to_port           = 0<br>  protocol          = &quot;-1&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}<br><br>resource &quot;aws_security_group&quot; &quot;https_world&quot; {<br>  name        = &quot;sg-https-world&quot;<br>  description = &quot;Allow HTTPS from anyone&quot;<br>  vpc_id      = data.aws_vpc.default.id<br><br>  tags = {<br>    Name = &quot;sg-https-world&quot;<br>  }<br>}<br><br>resource &quot;aws_vpc_security_group_ingress_rule&quot; &quot;https_world_ingress&quot; {<br>  security_group_id = aws_security_group.https_world.id<br>  description       = &quot;HTTPS from anywhere&quot;<br>  from_port         = 443<br>  to_port           = 443<br>  protocol          = &quot;tcp&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}<br><br>resource &quot;aws_vpc_security_group_egress_rule&quot; &quot;https_world_egress&quot; {<br>  security_group_id = aws_security_group.https_world.id<br>  description       = &quot;Allow all outbound&quot;<br>  from_port         = 0<br>  to_port           = 0<br>  protocol          = &quot;-1&quot;<br>  cidr_blocks       = [&quot;0.0.0.0/0&quot;]<br>}</pre><p>If you run terraform apply, Terraform will launch another EC2 Instance with the name &quot;minikube-server-2,&quot; install/run the Docker engine on it, install kubectl, and install Minikube.</p><p>After this step, you should have 3 EC2 Instances running.</p><ul><li>First with Jenkins running on it.</li><li>The Second will behave as a Build Server.</li><li>The Third one will be the Deploy Server.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W0f2HI9mBvmOt27OhV9Ujw.png" /></figure><h4>Step 9: Create a Docker Hub repository.</h4><p>You can use ECR as a Container Registry, but I used it on my <a href="https://medium.com/devops-technical-notes-and-manuals/devops-example-project-for-your-resume-198e34d874b4">first DevOps example project</a>, so this time I decided to use Docker Hub.</p><p>If you don’t know how to create the repository in Docker Hub, read about it here: <a href="https://docs.docker.com/docker-hub/repos/create/">Create a repository</a>.</p><p>I created a public repository because of the limitation of a free Docker account, but in the real world you should create private repositories only.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r21ZFp0YtO68DdcvCzyOFQ.png" /></figure><h4>Step 10: Setup Jenkins</h4><p>Now you should set up your Jenkins to use your GitHub repository and Docker Hub. Then you need to create two Nodes.</p><ul><li>First, you need to set up credentials to let Jenkins connect to the EC2 Instances (SSH key). I’m using the same SSH key for both servers. However, in a non-educational environment, you should consider to use a key per server. Follow <a href="https://www.jenkins.io/doc/book/using/using-credentials/">this documentation</a> to add the SSH key to the Jenkins credentials.</li><li>Next, add Docker Hub credentials to make Jenkins reach Docker Hub.</li></ul><p>Create the <a href="https://docs.docker.com/security/for-developers/access-tokens/">personal access token</a> in Docker Hub with Read &amp; Write permissions and define it in <a href="https://www.jenkins.io/doc/book/pipeline/docker/">Jenking Credentials</a>.</p><p>Next, create <a href="https://www.jenkins.io/doc/book/managing/nodes/">Nodes</a>.</p><ul><li>Proceed to Dashboard &gt; Manage Jenkins &gt; Nodes.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rezZ5j20byDj9XMTg59ong.png" /></figure><ul><li>Press on New Node &gt; Give Node a name &gt; Choose Permanent Agent &gt; Press Create.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c250pjMMckLcQauZTqFahg.png" /></figure><ul><li>You will need to fill a few fields.<br>- Give a description.<br>- I defined /home/ubuntu/jenkins_agent as a remote root directory.<br>- Enter EC2 IP as a host.<br>- Launch method: Launch agents via SSH and set your key as a credentials.<br>- Save and repeat for the server-2.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GRMNTM8T1vUSdaXArTbJJQ.png" /></figure><p>Now when you have 2 Nodes in Jenkins, create the <a href="https://www.jenkins.io/doc/book/pipeline/getting-started/">Jenkins pipeline</a>.</p><p>The pipeline definition should be the <strong>Pipeline script from SCM</strong>. This way, Jenkins will push the code from the GitHub repository every time you execute it.</p><p>Define the Repository URL, Branch Specifier and Jenkinsfile as a Script Path.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FlXE8DUttHlsXuITK2zHyA.png" /></figure><h4>Step 11: Create a Jenkinsfile.</h4><p>Next, create a Jenkinsfile that does the following:</p><p><strong>server-1 (Build Node)</strong></p><ul><li>Build a Docker Image and push it to Docker Hub.</li></ul><p><strong>server-2 (Deploy Node)</strong></p><ul><li>Install Nginx.</li><li>Install Certbot to create the certificate for the Route 53 domain.</li><li>Authenticate and install certificate.</li><li>Add Minikube Nginx configurations to sites-available.</li><li>Add Minikube Nginx configurations to sites-enabled and remove the default configurations.</li><li>Restart Nginx.</li><li>Start the POD and Service.</li></ul><p>Create the Jenkinsfile in the root of the GitHub repository.</p><p>Here is my file:</p><pre>pipeline {<br>    agent none<br><br>    environment {<br>        DOCKER_IMAGE = &quot;andreybyhalenko/exercise-python-application:latest&quot;<br>    }<br><br>    stages {<br>        stage(&#39;Build &amp; Push Docker Image&#39;) {<br>            agent { label &#39;build-node-1&#39; }<br>            steps {<br>                checkout scm<br><br>                script {<br>                    withCredentials([usernamePassword(<br>                        credentialsId: &#39;dockerHub&#39;,<br>                        usernameVariable: &#39;DOCKERHUB_USER&#39;,<br>                        passwordVariable: &#39;DOCKERHUB_PASS&#39;<br>                    )]) {<br>                        // Build and push the Docker Image<br>                        sh &quot;&quot;&quot;<br>                          docker build -t \$DOCKER_IMAGE .<br>                          docker login -u \$DOCKERHUB_USER -p \$DOCKERHUB_PASS<br>                          docker push \$DOCKER_IMAGE<br>                          docker logout<br>                        &quot;&quot;&quot;<br>                    }<br>                }<br>            }<br>        }<br><br>        stage(&#39;Deploy&#39;) {<br>            agent { label &#39;deploy-node-1&#39; }<br>            steps {<br>                checkout scm<br><br>                // Install Nginx and Certbot<br>                sh &quot;&quot;&quot;<br>                  sudo apt update<br>                  sudo apt install -y certbot python3-certbot-nginx nginx<br>                &quot;&quot;&quot;<br><br>                // Authenticate and install certificate<br>                sh &quot;&quot;&quot;<br>                  sudo certbot --nginx -d andreyexercise.com --non-interactive --agree-tos --email byhalenko@gmail.com<br>                &quot;&quot;&quot;<br><br>                // Add minikube Nginx configurations to sites-available<br>                sh &#39;&#39;&#39;<br>                  MINIKUBE_IP=$(minikube ip)<br>                  <br>                  cat &lt;&lt;EOF | sudo tee /etc/nginx/sites-available/minikube<br>                  server {<br>                      listen      443 ssl;<br>                      server_name andreyexercise.com;<br><br>                      ssl_certificate /etc/letsencrypt/live/andreyexercise.com/fullchain.pem;<br>                      ssl_certificate_key /etc/letsencrypt/live/andreyexercise.com/privkey.pem;<br>                      ssl_protocols TLSv1.2 TLSv1.3;<br>                      ssl_prefer_server_ciphers on;<br><br>                      location / {<br>                          proxy_pass http://${MINIKUBE_IP}:30443;<br>                          proxy_set_header Host $host;<br>                          proxy_set_header X-Real-IP $remote_addr;<br>                          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;<br>                          proxy_set_header X-Forwarded-Proto $scheme;<br>                      }<br>                  }<br><br>                  server {<br>                      listen      80;<br>                      server_name andreyexercise.com;<br><br>                      # Redirect HTTP to HTTPS<br>                      return 301 https://\$host\$request_uri;<br>                  }<br>                  EOF<br>                &#39;&#39;&#39;<br><br>                // Add minikube Nginx configurations to sites-enabled and remove the default configurations<br>                sh &quot;&quot;&quot;<br>                  sudo ln -s /etc/nginx/sites-available/minikube /etc/nginx/sites-enabled/<br>                  sudo rm -f /etc/nginx/sites-enabled/default<br>                &quot;&quot;&quot;<br><br>                // Restart Nginx<br>                sh &quot;&quot;&quot;<br>                  sudo systemctl restart nginx<br>                &quot;&quot;&quot;<br><br>                // Start the POD and Service<br>                sh &quot;&quot;&quot;<br>                  kubectl apply -f k8s/deployment.yml<br>                  kubectl apply -f k8s/service.yml<br>                &quot;&quot;&quot;<br>            }<br>        }<br>    }<br>}</pre><h4>Step 12: Add the Sonar Cloud and Snyk tests to CI using GitHub Actions.</h4><p>I was not asked to do this step on my homework. However, I always try to do extra during my homework, so I decided to add a simple test.</p><p>My idea was to add tests to the early stages of our CI/CD pipeline, before my Docker Image and code gets to the servers.</p><p>This is why I decided to add it to GitHub using <a href="https://github.com/features/actions">GitHub Actions</a>. Every time someone will push code to the Development branch, the test will run.</p><ul><li>How to integrate Sonar Cloud in GitHub Actions: <a href="https://github.com/SonarSource/sonarcloud-github-action">read here</a>.</li><li>How to integrate Snyk in GitHub Actions: <a href="https://github.com/snyk/actions">read here</a>.</li></ul><p><strong>Here is my YAML file for Snyk:</strong></p><pre>name: Snyk Container Test<br><br>on:<br>  push:<br>    branches:<br>      - development<br><br>permissions:<br>  contents: read<br><br>jobs:<br>  snyk:<br>    permissions:<br>      contents: read<br>      security-events: write<br>      actions: read<br>    runs-on: ubuntu-latest<br>    steps:<br>    - uses: actions/checkout@v4<br>    - name: Build a Docker image<br>      run: docker build -t your/image-to-test .<br>    - name: Run Snyk to check Docker image for vulnerabilities<br>      continue-on-error: true<br>      uses: snyk/actions/docker@14818c4695ecc4045f33c9cee9e795a788711ca4<br>      env:<br>        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}<br>      with:<br>        image: your/image-to-test<br>        args: --file=Dockerfile --severity-threshold=medium --sarif-file-output=snyk.sarif<br>    - name: Debug SARIF File<br>      run: cat snyk.sarif<br>    - name: Fix SARIF File<br>      run: |<br>        sed -i &#39;s/&quot;severity&quot;: null/&quot;severity&quot;: 0/g&#39; snyk.sarif<br>    - name: Upload result to GitHub Code Scanning<br>      uses: github/codeql-action/upload-sarif@v3<br>      with:<br>        sarif_file: snyk.sarif</pre><p><strong>Here is my YAML file for Sonar Cloud:</strong></p><pre>name: Sonar Cloud Code Test<br>on:<br>  push:<br>    branches:<br>      - development<br>  pull_request:<br>    types: [opened, synchronize, reopened]<br>jobs:<br>  sonarqube:<br>    name: SonarQube<br>    runs-on: ubuntu-latest<br>    steps:<br>      - uses: actions/checkout@v4<br>        with:<br>          fetch-depth: 0  # Shallow clones should be disabled for a better relevancy of analysis<br>      - name: SonarQube Scan<br>        uses: SonarSource/sonarqube-scan-action@v4<br>        env:<br>          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}</pre><p><a href="https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions">Read here about how to use secrets in GitHub Actions</a>.</p><h4>Step 13: Connect the AWS Route 53 domain name to the server-02 EC2 Instance. This is an optional step, as a domain name costs money.</h4><p>Open the EC2 Instance windows in AWS console and check for the Public IP address of the Deploy server (the one that runs your app).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*endEM2UBzC_k3jLo_3MqQg.png" /></figure><p>You need to connect your Route 53 domain to the public IP address. Actually, you can connect any domain from any domain provider, but I’m using AWS Route 53.</p><p>The process is to create the A record and route it to the EC2 Instance public IP address. <a href="https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-ec2-instance.html">Read more about it here</a>.</p><p>If you did everything right:</p><ul><li>Choose your Jenkins pipeline.</li><li>Press build now.</li><li>You should see the success log.</li><li>You should access your application using your domain.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qOqxJl0k8i9sMPgHyGvdMA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/964/1*Q0rrvFLj1xCFzSjOk_HpKw.png" /></figure><p>If it will fail, start by by checking the Jenkins pipeline logs. Then you can connect to each EC2 Instance using SSH and check the Linux logs, Docker logs, kubectl logs, etc.</p><blockquote><em>One tip from me. If you will face any issues during this exercise (I’m sure you will), avoid using ChatGPT to solve them. Instead, use official documentation and Google to find answers in forums like </em><a href="https://stackoverflow.com/"><em>StackOverflow</em></a><em>.</em></blockquote><p>I hope you enjoyed it, and I hope this project will be another step towards your DevOps career.</p><p>Do not hesitate to contact me with any questions.</p><p>You can contact me on <a href="https://www.linkedin.com/in/andrey-byhalenko/">LinkedIn</a> or send me an email at <a href="/service/mailto:andrey@juniordevopshub.com">andrey@juniordevopshub.com</a>.</p><p>If you liked my articles, you can join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=db45a77d6607" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/devops-example-project-for-your-resume-db45a77d6607">DevOps Example Project with Source Code. AWS, Terraform, Jenkins, Docker, GitHub Actions, Nginx.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 DevOps Concept-Related Interview Questions I got asked during 2024.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/devops-technical-notes-and-manuals/10-devops-interview-questions-i-got-asked-during-2024-8c8a7ca2dbe6?source=rss----07c04f098b7e---4"><img src="https://cdn-images-1.medium.com/max/2600/1*EADm3S23kE-tpRV2T5IQ6w.jpeg" width="6000"></a></p><p class="medium-feed-snippet">In this article, I will give examples of 10 common technical questions I got asked on the DevOps interviews during the past year.</p><p class="medium-feed-link"><a href="https://medium.com/devops-technical-notes-and-manuals/10-devops-interview-questions-i-got-asked-during-2024-8c8a7ca2dbe6?source=rss----07c04f098b7e---4">Continue reading on DevOps Manuals and Technical Notes »</a></p></div>]]></description>
            <link>https://medium.com/devops-technical-notes-and-manuals/10-devops-interview-questions-i-got-asked-during-2024-8c8a7ca2dbe6?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/8c8a7ca2dbe6</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Mon, 11 Nov 2024 14:35:14 GMT</pubDate>
            <atom:updated>2024-12-11T13:50:38.066Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to run Jenkins using Docker on AWS EC2 instance with Terraform. Step-by-step DevOps Guide.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/devops-technical-notes-and-manuals/how-to-run-jenkins-using-docker-on-aws-ec2-instance-with-terraform-step-by-step-devops-guide-d51f09eec7c2?source=rss----07c04f098b7e---4"><img src="https://cdn-images-1.medium.com/max/1188/1*rXiAHMIGVbRij-WMrUtMBA.jpeg" width="1188"></a></p><p class="medium-feed-snippet">In my previous guide, &#x201C;How to run Jenkins using Docker on an AWS EC2 Ubuntu instance&#x201D; I wrote about the step-by-step manual process to do&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/devops-technical-notes-and-manuals/how-to-run-jenkins-using-docker-on-aws-ec2-instance-with-terraform-step-by-step-devops-guide-d51f09eec7c2?source=rss----07c04f098b7e---4">Continue reading on DevOps Manuals and Technical Notes »</a></p></div>]]></description>
            <link>https://medium.com/devops-technical-notes-and-manuals/how-to-run-jenkins-using-docker-on-aws-ec2-instance-with-terraform-step-by-step-devops-guide-d51f09eec7c2?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/d51f09eec7c2</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Mon, 11 Nov 2024 14:34:51 GMT</pubDate>
            <atom:updated>2025-02-07T06:54:57.830Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How the traffic gets from the public internet to the Kubernetes pod. DevOps interview question.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/how-the-traffic-gets-from-the-public-internet-to-the-kubernetes-pod-devops-interview-question-7f0736dfe1c0?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/7f0736dfe1c0</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[technical-interview]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Mon, 11 Nov 2024 14:34:26 GMT</pubDate>
            <atom:updated>2024-08-26T15:40:29.761Z</atom:updated>
            <content:encoded><![CDATA[<p>The question “Explain how traffic gets from the public internet to the Kubernetes pod” is quite common in the technical DevOps interviews.</p><p>A correct and detailed answer to this question shows how familiar a DevOps engineer is with the processes of Kubernetes.</p><p>In this article, I will go through this process, assuming I am running Kubernetes on AWS EKS.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/871/1*y8z6qonhGSl84P16nKE1ZA.png" /></figure><p>The short answer can be summarized as:</p><p><strong>User Request → DNS → ELB → Ingress Controller (Optional) → Kubernetes Service → Pod.</strong></p><p>This setup allows traffic from the public internet to reach a specific pod running within your EKS cluster. The ELB and Ingress Controller manage the external exposure, while Kubernetes services manage the internal routing to the correct pod.</p><p>Let’s dig into the details.</p><h4>1. DNS Resolution</h4><p>When a user attempts to access your application via a domain name, the DNS system resolves this domain to an IP address. This IP address typically belongs to an Elastic Load Balancer (ELB) provided by AWS.</p><h4>2. Elastic Load Balancer</h4><p>EKS integrates with AWS services, including ELB (Elastic Load Balancer). The ELB distributes incoming traffic across multiple targets, such as EC2 instances and, in this case, Kubernetes pods. The ELB can be configured as an Application Load Balancer (ALB) for HTTP/HTTPS traffic or a Network Load Balancer (NLB) for TCP/UDP traffic.</p><p>The ELB has a public IP address that is accessible from the internet. When the DNS resolves the domain name to this IP address, the traffic is routed to the ELB.</p><h4>3. Ingress Controller (Optional)</h4><p>In Kubernetes, an <strong>Ingress resource</strong> defines rules for routing external traffic to services within the cluster. The <strong>Ingress Controller</strong> is responsible for fulfilling these rules. On EKS, you might use an ALB Ingress controller, which integrates with AWS’s ALB to route traffic based on the Ingress rules.</p><p>The Ingress controller listens for incoming traffic on the ELB and routes it to the appropriate Kubernetes service based on the specified Ingress rules (e.g., path-based routing, host-based routing).</p><h4>4. Kubernetes Service</h4><p>Within Kubernetes, services abstract access to pods. Depending on how the service is exposed, it will either:</p><ul><li><strong>ClusterIP:</strong> Route traffic only within the cluster.</li><li><strong>NodePort:</strong> Expose the service on a specific port on each node in the cluster.</li><li><strong>LoadBalancer:</strong> Provision a cloud load balancer (like ELB) to route external traffic to the service.</li></ul><p>The service routes incoming traffic to the appropriate pods using a mechanism called kube-proxy, which forwards the traffic based on IP tables.</p><h4>5. Network Traffic within EKS</h4><p>Once the service receives the traffic, it forwards it to one of the pods that match the service’s selector. The pod’s IP address is managed by the Kubernetes networking model.</p><p>On EKS, the AWS VPC CNI (Container Networking Interface) plugin is used, which provides each pod with an IP address within the VPC, enabling communication within the cluster and with external services.</p><h4>6. Pod Receiving Traffic</h4><p>Finally, the pod running your application receives traffic. The application inside the pod processes the request and sends back a response, which is routed back through the service, Ingress Controller (if applicable), and the ELB back to the client.</p><h4>Why the Ingress Controller is optional:</h4><p>The need for an Ingress Controller depends on how you want to expose your services to external traffic and the complexity of your routing requirements.</p><p><strong>Without an Ingress Controller:</strong> Suitable for simple setups where each service can have its own ELB (LoadBalancer service) or where you manage routing through external load balancers with NodePort service.</p><p>W<strong>ith an Ingress Controller:</strong> Ideal for more complex applications with multiple services, needing advanced routing, centralized management, and cost efficiency by using fewer ELBs.</p><p>If you like my articles, join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>As a bonus, you will receive a free step-by-step DevOps CI/CD project, which you can use in your portfolio.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7f0736dfe1c0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/how-the-traffic-gets-from-the-public-internet-to-the-kubernetes-pod-devops-interview-question-7f0736dfe1c0">How the traffic gets from the public internet to the Kubernetes pod. DevOps interview question.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[20 Essential Linux Debugging Commands and Tricks for DevOps Engineers.]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/devops-technical-notes-and-manuals/20-essential-linux-debugging-commands-and-tricks-for-devops-engineers-bf51ad224614?source=rss----07c04f098b7e---4"><img src="https://cdn-images-1.medium.com/max/2600/1*RjaMk_Hnrh6ifR4Ektu5PQ.jpeg" width="6000"></a></p><p class="medium-feed-snippet">In this article, I would like to share 20 Linux debugging commands with the examples.</p><p class="medium-feed-link"><a href="https://medium.com/devops-technical-notes-and-manuals/20-essential-linux-debugging-commands-and-tricks-for-devops-engineers-bf51ad224614?source=rss----07c04f098b7e---4">Continue reading on DevOps Manuals and Technical Notes »</a></p></div>]]></description>
            <link>https://medium.com/devops-technical-notes-and-manuals/20-essential-linux-debugging-commands-and-tricks-for-devops-engineers-bf51ad224614?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/bf51ad224614</guid>
            <category><![CDATA[software]]></category>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Mon, 11 Nov 2024 14:32:03 GMT</pubDate>
            <atom:updated>2024-12-11T13:22:51.247Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How to use the git diff command. DevOps manual to compare changes in your code.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/how-to-use-the-git-diff-command-devops-manual-to-compare-changes-in-your-code-b658647c2dc5?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/b658647c2dc5</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[devops-practice]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Mon, 25 Mar 2024 17:02:31 GMT</pubDate>
            <atom:updated>2024-03-26T07:24:07.793Z</atom:updated>
            <content:encoded><![CDATA[<p>The git diff command is used to show the differences between various commits, the working directory, and the staging area in a Git repository.</p><p>In this manual, I will show you some useful examples of this command.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*BCZkmZR1_YzDZy22Vn4uUw.png" /></figure><p>The git diff command It&#39;s a powerful tool for examining changes over time, understanding what has been modified, added, or removed, and preparing changes to be committed.</p><p>The git diff can show changes between the working tree and the index or a tree, changes between the index and a tree, changes between two trees, changes resulting from a merge, changes between two blob objects, or changes between two files on disk.</p><h4>SYNOPSIS:</h4><pre>git diff [&lt;options&gt;] [&lt;commit&gt;] [--] [&lt;path&gt;…​]<br>git diff [&lt;options&gt;] --cached [--merge-base] [&lt;commit&gt;] [--] [&lt;path&gt;…​]<br>git diff [&lt;options&gt;] [--merge-base] &lt;commit&gt; [&lt;commit&gt;…​] &lt;commit&gt; [--] [&lt;path&gt;…​]<br>git diff [&lt;options&gt;] &lt;commit&gt;…​&lt;commit&gt; [--] [&lt;path&gt;…​]<br>git diff [&lt;options&gt;] &lt;blob&gt; &lt;blob&gt;<br>git diff [&lt;options&gt;] --no-index [--] &lt;path&gt; &lt;path&gt;</pre><p>I’ve created a sample git repository and cloned it to my local pc.</p><p>It contains only the README.md file.</p><p>Let me run git diff to see the unstaged changes since the last commit:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/606/1*LPAd3XRer_VrBNkC9PoCFw.png" /></figure><p>As you see, there is nothing here.</p><p>I will create first_file file, add some text to it, run git add . and check git diff again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/644/1*Art1KIfdxTLafVit7vn9lQ.png" /></figure><p>Still nothing when I run the git diff. Let’s make another change to the file and run git diff again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*PKD24XnIxgdnnmo3VkmS1Q.png" /></figure><p>Now I see the changes in the output.</p><p>git diff lists all the differences between changes in the working directory and staging area. When I ran the git diff for the first time, it showed nothing because I hadn’t changed anything in the working area after I staged the first changes.<br>So there was nothing to compare.</p><p>Then I added the second line to the working area without adding to the staging area and ran the git diff.</p><p>Understanding the output results:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lhxySpQ2S-5nIyDsdB4ljg.png" /></figure><p><strong>diff — git a/first-file b/first-file:</strong><br>a/first_file = old file version<br>b/first_file = new file version</p><p><strong>@@ -1 +1,2 @@: </strong><br>chunk of modified lines. <br>- for the old file (what was removed/changed)<br>+ for the new file (what was added)</p><p><strong>“-# this is the first comment inside the first file”:</strong><br>This line remains in the old file.</p><p><strong>“+# this is the first comment inside the first file<br>+# This is the second line inside the first file”:</strong><br>These lines are from the new file.</p><p><strong>Here are a few examples of </strong><strong>git diff usage:</strong></p><p>List all the differences between changes in your working directory and staging area:</p><pre>git diff</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*CtD147iAXnj6AsZHmHM7kg.png" /></figure><p>List all changes staged for the commit:</p><pre>git diff --staged</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*j8xTTgc3BuZgS5gfWrigWQ.png" /></figure><p>List both staged and unstaged changes:</p><pre>git diff HEAD</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/647/1*3TCKOh9A82uKkTJsXM2isA.png" /></figure><p>Compare two branches:</p><pre>git diff branch1..branch2</pre><p>This will show you all the commits that <strong>branch2</strong> has that <strong>branch1</strong> doesn’t.</p><p>In the example below, you see all the commits that <strong>diff-test</strong> has compared to the <strong>Development</strong> branch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*0BflJtZ1oLMV9UmlUT5vcQ.png" /></figure><p>In the image below, you can see the changes the <strong>Development</strong> branch has, compared to the<strong> diff-test</strong> branch.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/785/1*Ey-_lZl7ldofzbL195BfLg.png" /></figure><p>Compare between the two most recent commits:</p><pre>git diff HEAD HEAD~1</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*GB2MEaJTE2ScIdWVycoXzQ.png" /></figure><p>Compare between the two specific commits:</p><pre>git diff &lt;commit1&gt; &lt;commit2&gt;</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*pU50ccSBR4wf9NbAbFnZPg.png" /></figure><p>Get a summary of the changes, including how many lines have been added, modified, and removed:</p><pre>git diff --stat</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/1*8aLkojVPlmeCHQkKYRdd1w.png" /></figure><p><strong>Use Git Aliases for Common Diffs:</strong></p><p>If you find yourself using complex git diff commands frequently, consider adding aliases to your .gitconfig for quick access.</p><p>For example, you could alias difflast to git diff HEAD^ HEAD to quickly compare the last commit to the one before it.</p><p>As you see, git diff is a powerful tool.</p><p>By mastering git diff and its various options, you can significantly improve your workflow, making it easier to review changes, catch errors early, and understand the history of your project.</p><p>If you liked my articles, you can join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>As a bonus, you will receive a free step-by-step DevOps CI/CD project, which you can use in your portfolio.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b658647c2dc5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/how-to-use-the-git-diff-command-devops-manual-to-compare-changes-in-your-code-b658647c2dc5">How to use the git diff command. DevOps manual to compare changes in your code.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DevOps Example Project for your Resume. CI/CD for the Python service using GitHub Actions and AWS.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/devops-example-project-for-your-resume-198e34d874b4?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/198e34d874b4</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Thu, 14 Mar 2024 09:39:13 GMT</pubDate>
            <atom:updated>2025-04-24T05:12:57.149Z</atom:updated>
            <content:encoded><![CDATA[<h3>DevOps Example Project for your Portfolio. Create a CI/CD for the Python service using GitHub Actions and AWS.</h3><p>One of the best ways to show off your DevOps skills is to create an example project and include it in your portfolio.</p><p>In this tutorial, I will show you how to write a simple Python service and how to define and create a CI/CD pipeline for the service.</p><p>I will develop a service called “counter-service.” It will maintain a web page with a counter for the number of POST requests it has served and return it for every GET request it gets.</p><p>I will use GitHub Actions, Docker, Sonar Cloud, Snyk, and AWS services.</p><p>The project idea comes from a real-life exercise I received during one of my job interview processes.</p><blockquote>Note that most probably during the project implementation, you will face an issues that you will need to solve by yourself. Use the official documentation, Google, Stack Overflow, or ChatGPT to search for the solution. It’s a part of the learning process and imitates real-life DevOps challenges. The ability to find the necessary information is one of the most important skills for a DevOps engineer.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*1rIdixSAFu4X6ZsCZvJSwA.png" /></figure><p>Here is my code for this project:<br><a href="https://github.com/Andrey-B-lab/counter-service-exercise">https://github.com/Andrey-B-lab/counter-service-exercise</a></p><p>Here is how my app looks like:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*BI3fnZ7NDBuD_oI7yi-tlA.png" /></figure><h4><strong>Prerequisites.</strong></h4><p>Basic understanding in:</p><ul><li>CI/CD: <a href="https://medium.com/devops-technical-notes-and-manuals/what-is-ci-cd-and-why-we-need-it-guide-for-devops-engineer-d81dcf1abd1f">read here</a></li><li><a href="https://git-scm.com/">git</a></li><li><a href="https://github.com/features/actions">GitHub Actions</a></li><li><a href="https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html">AWS ECR</a></li><li><a href="https://docs.aws.amazon.com/ec2/?icmpid=docs_homepage_featuredsvcs">AWS EC2</a></li><li>Docker: <a href="https://medium.com/devops-technical-notes-and-manuals/understanding-docker-a-beginners-guide-for-junior-devops-69c02a78fa78">read here</a></li><li>docker-compose: <a href="https://medium.com/devops-technical-notes-and-manuals/what-is-docker-compose-and-why-do-you-need-it-as-devops-engineer-adf544d892f7">read here</a></li><li><a href="https://docs.docker.com/develop/develop-images/dockerfile_best-practices/">Dockerfile</a></li><li><a href="https://sonarcloud.io/">Sonar Cloud</a></li><li><a href="https://snyk.io/">Snyk</a></li></ul><p>Free accounts:</p><ul><li>GitHub account:<br><a href="https://github.com/">https://github.com/</a></li><li>AWS account:<br><a href="https://aws.amazon.com/free/">https://aws.amazon.com/free/</a></li><li>Sonar Cloud account:<br><a href="https://www.sonarsource.com/products/sonarcloud/signup/">https://www.sonarsource.com/products/sonarcloud/signup/</a></li><li>Snyk account:<br><a href="https://app.snyk.io/login/">https://app.snyk.io/login/</a></li></ul><p>Install on your local PC:</p><ul><li>git</li><li>Docker</li><li>Python</li><li>Visual Studio Code or any other source-code editor you prefer</li></ul><h4>Project goals and definitions:</h4><ul><li>Develop a service called “counter-service.”<br>It should maintain a web page with a counter for the number of POST requests it has served and return it for every GET request it gets.</li><li>The code should be as simple as possible, yet well documented and robust.</li><li>The counter-service needs to be exposed on port 80.</li><li>Build the counter-service into a Docker image and deploy it as a Docker container to the EC2 instance.</li><li>Short service downtime is acceptable when re-deploying a service.</li><li>Upon commit &amp; push, the code should pass CI/CD and end up as a running Docker container on the EC2 instance.</li></ul><h4>Project implementation steps:</h4><ol><li>Create a GitHub repository on <a href="https://github.com/">github.com</a> and clone it locally.</li><li>Develop the counter-service app in Python, run it, and test it locally.</li><li>Create a Dockerfile for the counter-service app, build an image, run it, and test it locally.</li><li>Create an EC2 Ubuntu instance for the counter-service app and install Docker on it.</li><li>Push your code to GitHub.</li><li>Create an ECR repository.</li><li>Write GitHub Actions for CI (build the image and push it to ECR).</li><li>Add the Sonar Cloud and Snyk tests to CI.</li><li>Write GitHub Actions for CD (pull the image from the ECR to the EC2 instance and run it using Docker Compose).</li><li>Purchase a domain name on AWS Route 53 and connect it to the EC2 instance. This is an optional step, as a domain name costs money.</li></ol><h4>Step 1: Create a GitHub repository for the project and clone it.</h4><p>If you want to see my final code for this exercise, it’s here in this repository: <a href="https://github.com/Andrey-B-lab/counter-service-exercise">https://github.com/Andrey-B-lab/counter-service-exercise</a></p><p>You better don’t fork mine, but create an empty repository and start from scratch. This way, you will learn about the project and better understand it.</p><p>Remember, if you plan to show this project in an interview, be ready to talk about every piece of code in it and explain why you wrote it.</p><p>So create an empty repository, clone it to your local PC, and write the code yourself.</p><h4>Step 2: Develop the counter-service app in Python and test it locally.</h4><p>Firstly, I hope you are always creating a virtual environment when developing new applications. If not, you should start to do it.</p><p>Read about it here: <a href="https://docs.python.org/3/library/venv.html">https://docs.python.org/3/library/venv.html</a>.</p><p>I called my file counter-service.py.</p><p>I assumed the counter number needed to be permanent, so I added a counter file to the code and a Docker volume to docker-compose.yml.</p><p>counter-service.py contains four functions:</p><ol><li><strong>read_counter</strong>—reads and returns the current counter value from the file. If the file doesn’t exist, it returns 0.</li><li><strong>update_counter</strong>—updates the counter file with the new counter value.</li><li><strong>handle_request</strong>—handles GET and POST requests to the root endpoint.<br>GET request returns the current count of POST requests.<br>POST request increments the counter and returns the updated count.</li><li><strong>health_check</strong>—performs a simple health check of the application.<br>It tries to read the counter file as a basic check.</li></ol><p>From my point of view, it’s a quick and acceptable solution for the current setup. However, if you are planning to scale the app or make another usage except proof of concept, you should consider a NoSQL database or an in-memory datastore to maintain the counter state, or both.</p><p>If you want the count number to reset on every container restart, you can remove the volume.</p><p>If you want the counter to reset on the new image version release, you can achieve it by adding version tracking to the Docker image and, in the app, implementing logic that runs at startup to compare the current version (from the Docker image) with the version stored in the persistent volume.</p><p>I also added a health check function that performs a simple health check on the application. It tries to read the counter file as a basic check.</p><p>Here is the application code. It’s well commented, so you can read what each function does:</p><pre>from flask import Flask, request, jsonify<br>import os<br><br>app = Flask(__name__)<br><br># Define the path for the counter file to store the data in Docker Volume<br>COUNTER_FILE = &quot;/data/counter.txt&quot;<br><br>def read_counter():<br>    &quot;&quot;&quot;<br>    Reads and returns the current counter value from the file.<br>    If the file doesn&#39;t exist, it return 0.<br>    <br>    Returns:<br>        int: The current counter value.<br>    &quot;&quot;&quot;<br>    if os.path.exists(COUNTER_FILE):<br>        with open(COUNTER_FILE, &quot;r&quot;) as f:<br>            return int(f.read().strip())<br>    else:<br>        return 0<br><br>def update_counter(counter):<br>    &quot;&quot;&quot;<br>    Updates the counter file with the new counter value.<br>    <br>    Args:<br>        counter (int): The new counter value to write to the file.<br>    &quot;&quot;&quot;<br>    with open(COUNTER_FILE, &quot;w&quot;) as f:<br>        f.write(str(counter))<br><br>@app.route(&#39;/&#39;, methods=[&#39;GET&#39;, &#39;POST&#39;])<br>def handle_request():<br>    &quot;&quot;&quot;<br>    Handles GET and POST requests to the root endpoint.<br>    GET request returns the current count of POST requests.<br>    POST request increments the counter and returns the updated count.<br>    <br>    Returns:<br>        str: The response message with the current or updated counter.<br>    &quot;&quot;&quot;<br>    counter = read_counter()<br>    if request.method == &#39;POST&#39;:<br>        # Increment the counter for each POST request and update the file.<br>        counter += 1<br>        update_counter(counter)<br>        return f&quot;POST requests counter updated. Current count: {counter}&quot;<br>    else:<br>        # For GET requests, just return the current count.<br>        return f&quot;Current POST requests count: {counter}&quot;<br><br>@app.route(&#39;/health&#39;, methods=[&#39;GET&#39;])<br>def health_check():<br>    &quot;&quot;&quot;<br>    Performs a simple health check of the application.<br>    It tries to read the counter file as a basic check.<br>    <br>    Returns:<br>        tuple: A JSON response indicating the health status and the HTTP status code.<br>    &quot;&quot;&quot;<br>    try:<br>        # Basic health check: Ensure the counter file is accessible.<br>        read_counter()<br>        return jsonify({&quot;status&quot;: &quot;healthy&quot;}), 200<br>    except Exception as e:<br>        # Return an unhealthy status if any error occurs, e.g., file access issues.<br>        return jsonify({&quot;status&quot;: &quot;unhealthy&quot;, &quot;reason&quot;: str(e)}), 500<br><br>if __name__ == &#39;__main__&#39;:<br>    # Run the Flask app with binding to all interfaces on port 8080.<br>    # Debug mode is turned off for production use.<br>    app.run(host=&#39;0.0.0.0&#39;, port=8080, debug=False)</pre><p>Now, test the counter-service application locally.</p><p>Verify you are in the project’s directory. <br>Execute python counter-service.py.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*KWu-nsypdzYHbbwIXAFwmA.png" /></figure><p>Open your browser and proceed to <a href="http://127.0.0.1:8080/">http://127.0.0.1:8080/</a>. You should see the POST requests count.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/704/1*AcTE7Lv0j57FZpjtXm1S6A.png" /></figure><p>By reaching 127.0.0.1:8080 you sent a GET request.</p><p>Try to send a POST to the app.</p><p>If you are using a Windows PC, your POST request looks like this:</p><pre>Invoke-WebRequest -Uri http://127.0.0.1:8080 -Method POST</pre><p>If you are using Mac or Linux, you can use curl:</p><pre>curl -X POST http://127.0.0.1:8080</pre><p>You can use <a href="https://www.postman.com/downloads/">Postman</a> as well.</p><p>When sending the POST request, the counter gets updated:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gyMNJVZFIkr1vhITYNJNTA.png" /></figure><p>Refresh the browser (send GET request) and verify that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*JNyIYF0cot6idfm2w9BUNw.png" /></figure><p>It works locally. Let’s move on.</p><h4>Step 3: Create a Dockerfile for the counter-service app, build an image, run it, and test it locally.</h4><p>The easiest way to create the Dockerfile is to use the docker init command. Before you do that, you need to create a requirements.txt file for the counter-service application.</p><pre>pip3 freeze &gt; requirements.txt</pre><blockquote>Note that the pip3 freeze command includes all packages that are installed with pip install in your environment.</blockquote><blockquote>So if you don’t work with the virtual environment when developing in Python (you should!), you will get inside the file all the installed packages and not just the ones needed for the app.</blockquote><p>Now, run the docker init command.</p><p>You will have 4 questions to answer, just accept the defaults (verify that you defined Python in the first question):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/905/1*0pSFOTaU87vtYIAOJi3n9A.png" /></figure><p>Verify the created Dockerfile:</p><pre># syntax=docker/dockerfile:1<br><br># Comments are provided throughout this file to help you get started.<br># If you need more help, visit the Dockerfile reference guide at<br># https://docs.docker.com/go/dockerfile-reference/<br><br># Want to help us make this template better? Share your feedback here: https://forms.gle/ybq9Krt8jtBL3iCk7<br><br>ARG PYTHON_VERSION=3.12.2<br>FROM python:${PYTHON_VERSION}-slim as base<br><br># Prevents Python from writing pyc files.<br>ENV PYTHONDONTWRITEBYTECODE=1<br><br># Keeps Python from buffering stdout and stderr to avoid situations where<br># the application crashes without emitting any logs due to buffering.<br>ENV PYTHONUNBUFFERED=1<br><br>WORKDIR /app<br><br># Create a non-privileged user that the app will run under.<br># See https://docs.docker.com/go/dockerfile-user-best-practices/<br>ARG UID=10001<br>RUN adduser \<br>    --disabled-password \<br>    --gecos &quot;&quot; \<br>    --home &quot;/nonexistent&quot; \<br>    --shell &quot;/sbin/nologin&quot; \<br>    --no-create-home \<br>    --uid &quot;${UID}&quot; \<br>    appuser<br><br># Download dependencies as a separate step to take advantage of Docker&#39;s caching.<br># Leverage a cache mount to /root/.cache/pip to speed up subsequent builds.<br># Leverage a bind mount to requirements.txt to avoid having to copy them into<br># into this layer.<br>RUN --mount=type=cache,target=/root/.cache/pip \<br>    --mount=type=bind,source=requirements.txt,target=requirements.txt \<br>    python -m pip install -r requirements.txt<br><br># Switch to the non-privileged user to run the application.<br>USER appuser<br><br># Copy the source code into the container.<br>COPY . .<br><br># Expose the port that the application listens on.<br>EXPOSE 8000<br><br># Run the application.<br>CMD gunicorn &#39;counter-service:app&#39; --bind=0.0.0.0:8000</pre><p>Build the image:</p><pre>docker build --tag &quot;counter-service-local:1.0.0&quot; .</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h_et5as9wZPbJeAml2aD_Q.png" /></figure><p>Run the image:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*c0S6ZYw2MdXOWDRHdbGfvg.png" /></figure><p>It seems like it works.</p><p>However, there is an issue with the COUNTER_FILE, defined in the Python app.</p><p>The volume is not defined, so the application doesn’t work as expected. If you try to POST, you will see an error.</p><p>Don’t worry, you will fix it on the Linux server during the next steps.</p><p>A few notes from me.</p><p>I made some changes to the default Dockerfile.</p><ul><li>Instead of a slim image, I decided to use alpine because of vulnerability considerations, despite the fact that “python:slim” is smaller and it’s built and reacts faster than “python:alpine.”<br>In the current CI/CD setup, I use Snyk, which will find <a href="https://security.snyk.io/vuln/SNYK-DEBIAN12-ZLIB-6008963">critical vulnerabilities</a> in the slim image on the first “Merge to Main” run and will drop the run. In order to avoid that, I’m using alpine, just to show that the pipeline is working.</li><li>Another change is to use COPY counter-service.py . instead of COPY . . as best practice and vulnerability considerations.</li><li>I changed the application port to 8080, just for convenience, because it’s traditionally used as an alternative to port 80 for web traffic. Anyway, it&#39;s an internal TCP container port.</li><li>I enabled logs output.</li></ul><p>Here is my final Dockerfile:</p><pre># syntax=docker/dockerfile:1<br>ARG PYTHON_VERSION=3.13.0a4<br>FROM python:${PYTHON_VERSION}-alpine3.19 as base<br><br># Prevents Python from writing pyc files.<br>ENV PYTHONDONTWRITEBYTECODE=1<br><br># Keeps Python from buffering stdout and stderr to avoid situations where<br># the application crashes without emitting any logs due to buffering.<br>ENV PYTHONUNBUFFERED=1<br><br>WORKDIR /app<br><br># Create a non-privileged user that the app will run under.<br># See https://docs.docker.com/go/dockerfile-user-best-practices/<br>ARG UID=10001<br>RUN adduser \<br>    --disabled-password \<br>    --gecos &quot;&quot; \<br>    --home &quot;/nonexistent&quot; \<br>    --shell &quot;/sbin/nologin&quot; \<br>    --no-create-home \<br>    --uid &quot;${UID}&quot; \<br>    appuser<br><br># Download dependencies as a separate step to take advantage of Docker&#39;s caching.<br># Leverage a cache mount to /root/.cache/pip to speed up subsequent builds.<br># Leverage a bind mount to requirements.txt to avoid having to copy them into<br># into this layer.<br>RUN --mount=type=cache,target=/root/.cache/pip \<br>    --mount=type=bind,source=requirements.txt,target=requirements.txt \<br>    python -m pip install -r requirements.txt<br><br># Switch to the non-privileged user to run the application.<br>USER appuser<br><br># Copy the source code(counter-service.py) into the container.<br>COPY counter-service.py .<br><br># Expose the port that the application listens on.<br># In most Unix-like operating systems, binding to ports below 1024 requires elevated privileges.<br># This application runs on Docker as an appuser, which is restricted to binding to ports below 1024.<br>EXPOSE 8080<br><br># Run the application.Logs enabled to see the output logs<br>CMD [&quot;gunicorn&quot;, &quot;counter-service:app&quot;, &quot;--bind&quot;, &quot;0.0.0.0:8080&quot;, &quot;--access-logfile&quot;, &quot;-&quot;, &quot;--error-logfile&quot;, &quot;-&quot;]</pre><h4>Step 4: Create an EC2 Ubuntu 22.04 instance for the counter-service app.</h4><blockquote>For simplicity, I will use the AWS Dashboard to create the instance and not the Terraform script.</blockquote><blockquote>In the next tutorial, I will add Terraform, Load Balancer, Redis, etc. For now, let’s keep it simple.</blockquote><ul><li>Follow the tutorial about how to create an EC2 instance and install Docker on it:<br><a href="https://medium.com/devops-technical-notes-and-manuals/how-to-run-jenkins-using-docker-on-aws-ec2-ubuntu-instance-step-by-step-guide-for-junior-devops-766d382ec86e">link to the tutorial</a></li><li>Define the following network rules:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_j9p3I452-1tM7nf2GGelw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4XoZeErKv7gND3Nt3Uh51A.jpeg" /></figure><p>Prepare the EC2 instance for Continuous Deployment.</p><ul><li>Create a docker directory in /home/&lt;user&gt;/. You will copy the Docker files inside it.</li><li>Install AWS CLI: <a href="https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html">https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html</a></li><li>Configure AWS credentials on the instance and login to the ECR:<br>run aws configure command.</li><li>Retrieve an authentication token and authenticate your Docker client to your registry:</li></ul><pre>aws ecr get-login-password - region &lt;your-region&gt; | docker login - username AWS - password-stdin &lt;account-id&gt;.dkr.ecr.eu-west-1.amazonaws.com</pre><p>Now that you have installed Docker on the EC2 instance and verified that it works, let’s push your code to GitHub.</p><h4>Step 5: Push your code to GitHub.</h4><p>The main branch in the repository is the default one. Create an additional branch for the tests, name it “development”.</p><p>Do not push to the main or development branches. Create a new branch and push to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JxxmC3RnpdMmvxAzowNGvw.png" /></figure><p>It’s done, now create the ECR repository for the Docker images.</p><h4>Step 6: Create an ECR repository.</h4><p>You should name the ECR repository the same as your GitHub repository.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gz878PVTysv8o8I508sRag.png" /></figure><p>The next step is creating CI (Continuous Integration).</p><h4>Step 7: Write GitHub Actions for CI (build the image and push it to ECR).</h4><p>The goal of this step is to create the CI process using GitHub Actions in the following order:</p><ol><li>Create a new release:</li></ol><ul><li>Fetch all tags.</li><li>Get the latest tag, assume semver, and sort.</li><li>If there’s no tag yet, start with v0.0.0.</li><li>Increment the patch version.</li><li>Output the next version.</li><li>Create the release.</li></ul><blockquote>For simplicity, I increment the patch version only for each push to development in GitHub Actions. In case needed, there is an option to increment any version by merging from the branches with specific syntax. If pushed from patch/, increment patch version, if pushed from feature/, increment feature version; and so on.</blockquote><p>2. Build a Docker image.</p><ul><li>Configure AWS credentials.</li><li>Login to Amazon ECR.</li><li>Extract the repository name.</li><li>Build a Docker image.</li><li>Push the image to ECR.</li></ul><p>GitHub Actions yml files stored in .github/workflows directory in GitHub repository.</p><p>It’s easier to do it on the <a href="https://github.com/">github.com</a> website, because you can access the Marketplace for Actions.</p><p>Once you are in the repository, press Actions &gt; Simple Workflow.</p><p>GitHub will create a file with the minimum necessary structure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TeF62a-oGnliDjwDzejibQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vVX72SFQtfqgeBCfVu8awA.png" /></figure><p>Change the push to “development” instead of “main”.</p><p>Push to development will be a trigger on which GitHub Action will start.</p><p>Now you need to write the code for the earlier defined steps.</p><blockquote>For the first step (creating a new release), I wrote a custom script that automatically determines the next version number for the project, based on the semantic versioning (semver) scheme. It does not directly reference a GitHub Market Action, but instead, it’s written as a shell script to be executed within a GitHub Actions workflow step. For the next steps, search for the syntax in the Marketplace on the right panel.</blockquote><p><strong>Create a new release:</strong></p><ul><li>Fetch all tags.</li></ul><pre>git fetch --tags</pre><ul><li>Get the latest tag, assume semver, and sort.</li></ul><pre>LATEST_TAG=$(git tag -l | grep -E &#39;^v[0-9]+\.[0-9]+\.[0-9]+$&#39; | sort -V | tail -n1)</pre><ul><li>If there’s no tag yet, start with v0.0.0.</li></ul><pre>if [ -z &quot;$LATEST_TAG&quot; ]; then<br>  LATEST_TAG=&quot;v0.0.0&quot;<br>fi</pre><ul><li>Increment the patch version.</li></ul><pre>NEXT_TAG=$(echo $LATEST_TAG | awk -F. &#39;{print $1&quot;.&quot;$2&quot;.&quot;$3+1}&#39;)</pre><ul><li>Output the next version.</li></ul><pre>echo &quot;::set-output name=tag::$NEXT_TAG&quot;<br>echo &quot;Next version: $NEXT_TAG&quot;</pre><ul><li>Create the release.</li></ul><pre>    - name: Create Release<br>      id: create_release<br>      uses: actions/create-release@v1<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN_2 }}<br>      with:<br>        tag_name: ${{ steps.next_version.outputs.tag }}<br>        release_name: Release ${{ steps.next_version.outputs.tag }}<br>        draft: false<br>        prerelease: false</pre><p><strong>Build a Docker image.</strong></p><ul><li>Configure AWS credentials.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CP1QgQm1lRkKChAuDfREng.png" /></figure><pre>    - name: Configure AWS credentials<br>      uses: aws-actions/configure-aws-credentials@v1<br>      with:<br>        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>        aws-region: eu-west-1</pre><ul><li>Login to Amazon ECR.</li></ul><pre>    - name: Login to Amazon ECR<br>      id: login-ecr<br>      uses: aws-actions/amazon-ecr-login@v1</pre><ul><li>Extract the repository name.</li></ul><pre>    - name: Extract repository name<br>      id: repo-name<br>      run: |<br>        REPO_NAME=&quot;${GITHUB_REPOSITORY##*/}&quot;<br>        echo &quot;REPO_NAME=$REPO_NAME&quot; &gt;&gt; $GITHUB_ENV<br>        echo &quot;::set-output name=repo_name::$REPO_NAME&quot;</pre><ul><li>Build a Docker image.</li></ul><pre>    - name: Build Docker image<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: ${{ env.REPO_NAME }}<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .<br>        echo &quot;IMAGE_NAME=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG&quot; &gt;&gt; $GITHUB_ENV</pre><ul><li>Push the image to ECR.</li></ul><pre>    - name: Push Docker image to Amazon ECR<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: counter-service-exercise<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        # Tag the image as latest<br>        docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:latest<br>        # Push the specific version tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG<br>        # Push the latest tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest</pre><blockquote>As you noticed, I’m using the latest tag in my code.</blockquote><blockquote>It’s worth noting that relying on the latest tag in production can be risky, as it might lead to unpredicted behaviors if the image changes unexpectedly.</blockquote><blockquote>Despite this, I decided to use the latest tag for training purposes.</blockquote><blockquote>I’m using docker-compose, and the easiest way to ensure docker-compose restating with the new image version is to use tag latest.</blockquote><p>Here is the full GitHub Action for the CI (I added my comments to the code as well):</p><pre>name: Build and Push Docker image to AWS ECR<br><br>on:<br>  push:<br>    branches:<br>      - development<br><br>jobs:<br>  build-and-push:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>    - name: Check out the repo<br>      uses: actions/checkout@v2<br>      with:<br>        fetch-depth: 0 # Necessary to fetch all tags and history<br><br>################################################################<br>###               DETERMINE NEXT VERSION                     ###<br>###   Used for creating new releases and image tags          ###<br>################################################################<br><br>    - name: Determine Next Version<br>      id: next_version<br>      run: |<br>        # Fetch all tags<br>        git fetch --tags<br>        <br>        # Get the latest tag, assume semver, and sort.<br>        LATEST_TAG=$(git tag -l | grep -E &#39;^v[0-9]+\.[0-9]+\.[0-9]+$&#39; | sort -V | tail -n1)<br>        <br>        # If there&#39;s no tag yet, start with v0.0.0. Used for new repos<br>        if [ -z &quot;$LATEST_TAG&quot; ]; then<br>          LATEST_TAG=&quot;v0.0.0&quot;<br>        fi<br>        <br>        # Increment the patch version<br>        NEXT_TAG=$(echo $LATEST_TAG | awk -F. &#39;{print $1&quot;.&quot;$2&quot;.&quot;$3+1}&#39;)<br>        <br>        # Output the next version<br>        echo &quot;::set-output name=tag::$NEXT_TAG&quot;<br>        echo &quot;Next version: $NEXT_TAG&quot;<br><br>################################################################<br>###                     CREATE RELEASE                       ###<br>###  Creating release with the tag from the previous step    ###<br>################################################################<br><br>    - name: Create Release<br>      id: create_release<br>      uses: actions/create-release@v1<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN_2 }}<br>      with:<br>        tag_name: ${{ steps.next_version.outputs.tag }}<br>        release_name: Release ${{ steps.next_version.outputs.tag }}<br>        draft: false<br>        prerelease: false<br><br>################################################################<br>###                  BUILD DOCKER IMAGE                      ###<br>###         Build Docker image from the Dockefile            ###<br>################################################################<br><br>    - name: Configure AWS credentials<br>      uses: aws-actions/configure-aws-credentials@v1<br>      with:<br>        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>        aws-region: eu-west-1<br><br>    - name: Login to Amazon ECR<br>      id: login-ecr<br>      uses: aws-actions/amazon-ecr-login@v1<br><br>    - name: Extract repository name<br>      id: repo-name<br>      run: |<br>        REPO_NAME=&quot;${GITHUB_REPOSITORY##*/}&quot;<br>        echo &quot;REPO_NAME=$REPO_NAME&quot; &gt;&gt; $GITHUB_ENV<br>        echo &quot;::set-output name=repo_name::$REPO_NAME&quot;<br><br>    - name: Build Docker image<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: ${{ env.REPO_NAME }}<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .<br>        echo &quot;IMAGE_NAME=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG&quot; &gt;&gt; $GITHUB_ENV<br><br>###########################################################<br>###                  PUSH IMAGE TO ECR                  ###<br>###     Tag Docker image as &quot;latest&quot; and push to ECR    ###<br>########################################################### <br><br>    - name: Push Docker image to Amazon ECR<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: counter-service-dev<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        # Tag the image as latest<br>        docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:latest<br>        # Push the specific version tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG<br>        # Push the latest tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest</pre><p>Note that I’m using the following secrets in this code:</p><ul><li>GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN_2 }}</li><li>aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}</li><li>aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}</li></ul><blockquote>If you are not familiar with secrets in GitHub Actions, read about it here:<br><a href="https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions">https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions</a></blockquote><p>You need to create an IAM user in AWS and assign the necessary permissions (AmazonEC2ContainerRegistryPowerUser).</p><p>When you create an Access key for this user, you’ll be provided with an Access Key ID and a Secret Access Key. Make sure to record these securely. The Secret Access Key is only shown once and cannot be retrieved later, though you can always generate a new one if needed.</p><p>Save the AWS_ACCESS_KEY_ID and the AWS_SECRET_ACCESS_KEY as GitHub repository secrets.</p><p>About the GITHUB_TOKEN.</p><p>The actions/create-release action, which is being used to create a release, requires authentication to interact with the GitHub API. While GitHub provides a default GITHUB_TOKEN for workflows, which has a scoped set of permissions for the repository that is running the action, I prefer to create my personal access token and use it.</p><p>Read about the personal access tokens here:<br><a href="https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens">https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens</a></p><p>Push the GitHub Actions code to the Development branch and see if the image will be delivered to ECR by executing GutHub Action.</p><p>You can see the GitHub Actions output in real-time in the Actions tab.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l_HpU8iM85eLefNNCVsFRw.png" /></figure><p>The GitHub Action run finished without any errors, let’s check the ECR.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kcqPU-icW_fE1FOGBnqtBQ.png" /></figure><blockquote>You should see v0.0.1 if your GitHub Action worked on the first try. The tag will be created for each Action run, it doesn’t matter if it fails or succeeds. The release will be created only if the Action succeeds.</blockquote><blockquote>This is why you see v0.0.3 in my repository.</blockquote><p>Good job! You have configured Continuous Integration!</p><p>Let’s add the code and vulnerability tests to your GitHub Action.</p><h4>Step 8: Add the Sonar Cloud and Snyk tests.</h4><p>The Sonar Cloud scans the code for bugs or vulnerabilities. Snyk scans the code and the base Docker image from the Dockerfile for vulnerabilities.</p><p><strong>Sonar Cloud:</strong></p><ul><li>Login with your GitHub account to <a href="https://sonarcloud.io/">Sonar Cloud</a>.</li><li><a href="https://docs.sonarsource.com/sonarcloud/organizations/overview/">Create an Organization</a> (optionally).</li><li>Press + on the right upper corner, then press Analyze new project.</li><li>Search for your project, select it, and press Set Up.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yS-Bbe97T7-L1k9cqnnD4g.png" /><figcaption>Set Up new project</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4zqKWQgX4UhG2iAU-e3DGA.png" /><figcaption>Choose Previous version and press Create project</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MsJMeIqZtFMij4LmKIyA1Q.png" /><figcaption>Choose GitHub Action as Analysis Method</figcaption></figure><p>You will get exact instructions on what you need to do in order to run the Sonar Cloud test with GitHub Actions.</p><p>You will need to create a new SONAR_TOKEN secret, copy the SonarCloud step to your GitHub Actions yml file, and create a sonar-project.properties file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ijQqMpslZ2MS4QtBFtaFA.png" /></figure><p>Here is my SonarCloud step:</p><pre>    - name: SonarCloud Scan<br>      uses: SonarSource/sonarcloud-github-action@master<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}<br>        SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}<br><br>    - name: Setup Git<br>      run: |<br>        git config --global user.name &#39;github-actions&#39;<br>        git config --global user.email &#39;github-actions@github.com&#39;</pre><p>Make sure the Automatic Analysis is off in Administration/Analysis Method.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3TqYEH-6sFFq6feMJO528A.png" /></figure><p><strong>Snyk:</strong></p><ul><li>Login to <a href="https://snyk.io/">https://snyk.io/</a> with GitHub.</li><li>Press “Add project ”on the right upper corner &gt; GitHub, choose your repository, and press “Add selected repository.”</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vzzp57DO-JA00MioJLujEA.png" /></figure><p>Search for the Snyk GitHub Action syntax in Actions Marketplace on GitHub:<br><a href="https://github.com/snyk/actions/tree/master/python">https://github.com/snyk/actions/tree/master/python</a></p><p>How to configure SNYK_TOKEN:<br><a href="https://docs.snyk.io/snyk-api/revoking-and-regenerating-snyk-api-tokens">https://docs.snyk.io/snyk-api/revoking-and-regenerating-snyk-api-tokens</a></p><p>Here is my Snyk step:</p><pre>    - name: Run Snyk to check Docker image for vulnerabilities<br>      uses: snyk/actions/docker@master<br>      env:<br>        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}<br>      with:<br>         image: ${{ env.IMAGE_NAME }}<br>         args: --severity-threshold=high --policy-path=.snyk<br>      continue-on-error: false</pre><p>Here is my updated GitHub Action that includes SonarCloud and Snyk:</p><pre>name: Build and Push Docker image to AWS ECR<br><br>on:<br>  push:<br>    branches:<br>      - development<br><br>jobs:<br>  build-and-push:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>    - name: Check out the repo<br>      uses: actions/checkout@v2<br>      with:<br>        fetch-depth: 0 # Necessary to fetch all tags and history<br><br>################################################################<br>###               SONAR CLOUD SCAN                           ###<br>### Drops the build if any bugs or vulnerabilities are found.###<br>###            Using the default quality gate.               ###<br>###        Connected to my personal Sonar Cloud account      ###<br>################################################################<br><br>    - name: SonarCloud Scan<br>      uses: SonarSource/sonarcloud-github-action@master<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}<br>        SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}<br><br>    - name: Setup Git<br>      run: |<br>        git config --global user.name &#39;github-actions&#39;<br>        git config --global user.email &#39;github-actions@github.com&#39;<br><br>################################################################<br>###               DETERMINE NEXT VERSION                     ###<br>###   Used for creating new releases and image tags          ###<br>################################################################<br><br>    - name: Determine Next Version<br>      id: next_version<br>      run: |<br>        # Fetch all tags<br>        git fetch --tags<br>        <br>        # Get the latest tag, assume semver, and sort.<br>        LATEST_TAG=$(git tag -l | grep -E &#39;^v[0-9]+\.[0-9]+\.[0-9]+$&#39; | sort -V | tail -n1)<br>        <br>        # If there&#39;s no tag yet, start with v0.0.0. Used for new repos<br>        if [ -z &quot;$LATEST_TAG&quot; ]; then<br>          LATEST_TAG=&quot;v0.0.0&quot;<br>        fi<br>        <br>        # Increment the patch version<br>        NEXT_TAG=$(echo $LATEST_TAG | awk -F. &#39;{print $1&quot;.&quot;$2&quot;.&quot;$3+1}&#39;)<br>        <br>        # Output the next version<br>        echo &quot;::set-output name=tag::$NEXT_TAG&quot;<br>        echo &quot;Next version: $NEXT_TAG&quot;<br><br>################################################################<br>###                     CREATE RELEASE                       ###<br>###  Creating release with the tag from the previous step    ###<br>################################################################<br><br>    - name: Create Release<br>      id: create_release<br>      uses: actions/create-release@v1<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN_2 }}<br>      with:<br>        tag_name: ${{ steps.next_version.outputs.tag }}<br>        release_name: Release ${{ steps.next_version.outputs.tag }}<br>        draft: false<br>        prerelease: false<br><br>################################################################<br>###                  BUILD DOCKER IMAGE                      ###<br>###         Build Docker image from the Dockefile            ###<br>################################################################<br><br>    - name: Configure AWS credentials<br>      uses: aws-actions/configure-aws-credentials@v1<br>      with:<br>        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>        aws-region: eu-west-1<br><br>    - name: Login to Amazon ECR<br>      id: login-ecr<br>      uses: aws-actions/amazon-ecr-login@v1<br><br>    - name: Extract repository name<br>      id: repo-name<br>      run: |<br>        REPO_NAME=&quot;${GITHUB_REPOSITORY##*/}&quot;<br>        echo &quot;REPO_NAME=$REPO_NAME&quot; &gt;&gt; $GITHUB_ENV<br>        echo &quot;::set-output name=repo_name::$REPO_NAME&quot;<br><br>    - name: Build Docker image<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: ${{ env.REPO_NAME }}<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .<br>        echo &quot;IMAGE_NAME=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG&quot; &gt;&gt; $GITHUB_ENV<br><br>###########################################################<br>###  Docker image Snyk scan | If fails, drop the action ###<br>###       Connected to my personal Snyk account         ###<br>###     The code owner receives an email notification   ###<br>### Possible to configure Slack notification if needed  ###<br>###########################################################               <br>               <br>    - name: Run Snyk to check Docker image for vulnerabilities<br>      uses: snyk/actions/docker@master<br>      env:<br>        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}<br>      with:<br>         image: ${{ env.IMAGE_NAME }}<br>         args: --severity-threshold=high --policy-path=.snyk<br>      continue-on-error: false<br><br>###########################################################<br>###                   PUSH IMAGE TO ECR                 ###<br>###     Tag Docker image as &quot;latest&quot; and push to ECR    ###<br>########################################################### <br><br>    - name: Push Docker image to Amazon ECR<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: counter-service-dev<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        # Tag the image as latest<br>        docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:latest<br>        # Push the specific version tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG<br>        # Push the latest tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest</pre><p>Push to Development branch and verify it’s working.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DhRmVCpJNadcvdcLDxAnAQ.png" /></figure><p>If all is set, it’s time to create a CD (Continuous Deployment).</p><h4>Step 9: Write GitHub Actions for CD (pull the image from the ECR to the EC2 instance and run it).</h4><p><strong>Pull the image from the ECR to the EC2 instance and run it.</strong></p><p>To pull the image from the ECR, we need to define and save as GitHub Action Secrets the following parameters:</p><ol><li>EC2 instance PEM key to connect to it (EC2_PEM_KEY).</li><li>EC2 instance IP address or hostname (EC2_HOST).</li></ol><blockquote>Note that the IP of the EC2 instance is temporary by default, and the instance will get a new IP once restarted. You can purchase a static IP if you want.</blockquote><ol><li>EC2 instance user (EC2_USER). The default user for Ubuntu servers is ubuntu.</li></ol><p>Here is a detailed explanation of the Deploy to EC2 GitHub Action:<br><a href="https://github.com/marketplace/actions/deploy-docker-to-aws-ec2">https://github.com/marketplace/actions/deploy-docker-to-aws-ec2</a></p><p>I will use custom commands to be executed on an EC2 instance within a GitHub Actions workflow step. Here is my GitHub Action step for CD:</p><pre>    - name: Deploy to EC2<br>      env:<br>        EC2_PEM_KEY: ${{ secrets.EC2_PEM_KEY }}<br>        EC2_HOST: ${{ secrets.EC2_HOST }}<br>        EC2_USER: ${{ secrets.EC2_USER }}<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: counter-service-exercise<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        # Save PEM key to file and set permissions<br>        echo &quot;$EC2_PEM_KEY&quot; &gt; ec2.pem<br>        chmod 400 ec2.pem<br><br>        # SSH, SCP commands<br>        SSH_COMMAND=&quot;ssh -i ec2.pem -o StrictHostKeyChecking=no $EC2_USER@$EC2_HOST&quot;<br>        SCP_COMMAND=&quot;scp -i ec2.pem -o StrictHostKeyChecking=no&quot;<br><br>        #Login to the Docker Registry (ECR)<br>        $SSH_COMMAND &quot;aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin $ECR_REGISTRY&quot;<br>        <br>        # Copy docker-compose.yml to EC2 server<br>        $SCP_COMMAND docker-compose.yml $EC2_USER@$EC2_HOST:/home/centos/docker/<br>        <br>        # Pull and run the Docker container on EC2<br>        $SSH_COMMAND &quot;cd /home/ubuntu/docker/ &amp;&amp; docker pull $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG &amp;&amp; docker compose -f docker-compose.yml up -d --force-recreate&quot;<br>        <br>        # Cleanup PEM key<br>        rm -f ec2.pem</pre><p>As you see, I’m using EC2_PEM_KEY to connect to the server. It’s the private key of the EC2 instance from step 4. You should add it as a secret to the GitHub repository.</p><p>There is also the step “Login to the Docker Registry (ECR)”. You need it because the ECR login lasts for 12 hours only.</p><p>The GitHub Action will also copy docker-compose.yml from the GitHub repository to the EC2 server. So add this file to the GitHub repository.</p><p>Here is my docker-compose.yml:</p><pre>version: &#39;2.4&#39;  # The last version of Docker Compose file format that directly supports mem_limit and cpus<br>services:<br>  counter-service:<br>    container_name: counter-service-exercise<br>    image: &lt;aws account id&gt;.dkr.ecr.eu-west-1.amazonaws.com/counter-service-exercise:latest<br>    volumes:<br>      - ./data:/data<br>    ports:<br>      - &quot;80:8080&quot;<br>    restart: always<br>    mem_limit: 256M<br>    cpus: 0.5</pre><p>Here is what the final GitHub Action file looks like:</p><pre>name: Build and Push Docker image to AWS ECR<br><br>on:<br>  push:<br>    branches:<br>      - development<br><br>jobs:<br>  build-and-push:<br>    runs-on: ubuntu-latest<br><br>    steps:<br>    - name: Check out the repo<br>      uses: actions/checkout@v2<br>      with:<br>        fetch-depth: 0 # Necessary to fetch all tags and history<br><br>################################################################<br>###               SONAR CLOUD SCAN                           ###<br>### Drops the build if any bugs or vulnerabilities are found.###<br>###            Using the default quality gate.               ###<br>###        Connected to my personal Sonar Cloud account      ###<br>################################################################<br><br>    - name: SonarCloud Scan<br>      uses: SonarSource/sonarcloud-github-action@master<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}<br>        SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}<br><br>    - name: Setup Git<br>      run: |<br>        git config --global user.name &#39;github-actions&#39;<br>        git config --global user.email &#39;github-actions@github.com&#39;<br><br>################################################################<br>###               DETERMINE NEXT VERSION                     ###<br>###   Used for creating new releases and image tags          ###<br>################################################################<br><br>    - name: Determine Next Version<br>      id: next_version<br>      run: |<br>        # Fetch all tags<br>        git fetch --tags<br>        <br>        # Get the latest tag, assume semver, and sort.<br>        LATEST_TAG=$(git tag -l | grep -E &#39;^v[0-9]+\.[0-9]+\.[0-9]+$&#39; | sort -V | tail -n1)<br>        <br>        # If there&#39;s no tag yet, start with v0.0.0. Used for new repos<br>        if [ -z &quot;$LATEST_TAG&quot; ]; then<br>          LATEST_TAG=&quot;v0.0.0&quot;<br>        fi<br>        <br>        # Increment the patch version<br>        NEXT_TAG=$(echo $LATEST_TAG | awk -F. &#39;{print $1&quot;.&quot;$2&quot;.&quot;$3+1}&#39;)<br>        <br>        # Output the next version<br>        echo &quot;::set-output name=tag::$NEXT_TAG&quot;<br>        echo &quot;Next version: $NEXT_TAG&quot;<br><br>################################################################<br>###                     CREATE RELEASE                       ###<br>###  Creating release with the tag from the previous step    ###<br>################################################################<br><br>    - name: Create Release<br>      id: create_release<br>      uses: actions/create-release@v1<br>      env:<br>        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN_2 }}<br>      with:<br>        tag_name: ${{ steps.next_version.outputs.tag }}<br>        release_name: Release ${{ steps.next_version.outputs.tag }}<br>        draft: false<br>        prerelease: false<br><br>################################################################<br>###                  BUILD DOCKER IMAGE                      ###<br>###         Build Docker image from the Dockefile            ###<br>################################################################<br><br>    - name: Configure AWS credentials<br>      uses: aws-actions/configure-aws-credentials@v1<br>      with:<br>        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}<br>        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}<br>        aws-region: eu-west-1<br><br>    - name: Login to Amazon ECR<br>      id: login-ecr<br>      uses: aws-actions/amazon-ecr-login@v1<br><br>    - name: Extract repository name<br>      id: repo-name<br>      run: |<br>        REPO_NAME=&quot;${GITHUB_REPOSITORY##*/}&quot;<br>        echo &quot;REPO_NAME=$REPO_NAME&quot; &gt;&gt; $GITHUB_ENV<br>        echo &quot;::set-output name=repo_name::$REPO_NAME&quot;<br><br>    - name: Build Docker image<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: ${{ env.REPO_NAME }}<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .<br>        echo &quot;IMAGE_NAME=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG&quot; &gt;&gt; $GITHUB_ENV<br><br>###########################################################<br>###  Docker image Snyk scan | If fails, drop the action ###<br>###       Connected to my personal Snyk account         ###<br>###     The code owner receives an email notification   ###<br>### Possible to configure Slack notification if needed  ###<br>###########################################################               <br>               <br>    - name: Run Snyk to check Docker image for vulnerabilities<br>      uses: snyk/actions/docker@master<br>      env:<br>        SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}<br>      with:<br>         image: ${{ env.IMAGE_NAME }}<br>         args: --severity-threshold=high --policy-path=.snyk<br>      continue-on-error: false<br><br>###########################################################<br>###         PUSH IMAGE TO ECR AND DEPLOY TO EC2         ###<br>###     Tag Docker image as &quot;latest&quot; and push to ECR    ###<br>###               Deploy to EC2 using SSH               ###<br>########################################################### <br><br>    - name: Push Docker image to Amazon ECR<br>      env:<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: counter-service-dev<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        # Tag the image as latest<br>        docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:latest<br>        # Push the specific version tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG<br>        # Push the latest tag<br>        docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest<br><br>    - name: Deploy to EC2<br>      env:<br>        EC2_PEM_KEY: ${{ secrets.EC2_PEM_KEY }}<br>        EC2_HOST: ${{ secrets.EC2_HOST }}<br>        EC2_USER: ${{ secrets.EC2_USER }}<br>        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}<br>        ECR_REPOSITORY: counter-service-dev<br>        IMAGE_TAG: ${{ steps.next_version.outputs.tag }}<br>      run: |<br>        # Save PEM key to file and set permissions<br>        echo &quot;$EC2_PEM_KEY&quot; &gt; ec2.pem<br>        chmod 400 ec2.pem<br><br>        # SSH, SCP commands<br>        SSH_COMMAND=&quot;ssh -i ec2.pem -o StrictHostKeyChecking=no $EC2_USER@$EC2_HOST&quot;<br>        SCP_COMMAND=&quot;scp -i ec2.pem -o StrictHostKeyChecking=no&quot;<br><br>        #Login to Docker registry<br>        $SSH_COMMAND &quot;aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin $ECR_REGISTRY&quot;<br>        <br>        # Copy docker-compose.yml to EC2 server<br>        $SCP_COMMAND docker-compose.yml $EC2_USER@$EC2_HOST:/home/ubuntu/docker/<br>        <br>        # Pull and run the Docker container on EC2<br>        $SSH_COMMAND &quot;cd /home/ubuntu/docker/ &amp;&amp; docker pull $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG &amp;&amp; docker compose -f docker-compose.yml up -d --force-recreate&quot;<br>        <br>        # Cleanup PEM key<br>        rm -f ec2.pem</pre><p>Push the changes to the Development branch and verify CI/CD works.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Px6Gby8yi08JqUvG2KigNQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/1*aOMRcT9_JnxSHqN9I8m9og.png" /></figure><p>If you type the Public IPv4 address of your EC2 instance into the browser, you should see your app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/882/1*GWs4-qlmUwxbADCF3BQzGw.png" /></figure><p>If you try to POST to the app now, you will see an error:<br>PermissionError: [Errno 13] Permission denied: ‘/data/counter.txt’</p><p>This is because the application, running inside a Docker container, does not have the necessary permissions to write to the file /data/counter.txt. There is a mismatch between the user ID (UID) of the process inside the container and the ownership or permissions of the mounted volume on the host machine.</p><p>In the real world, you should use a NoSQL database for this kind of application, but it’s out of the scope of this exercise.</p><p>Just give ./data full permissions (sudo chmod -R 777 ./data), and it will work.</p><p>POST to it now, and you should see the counter updates:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y-sq_HfQ91UmVh1oTHXe8Q.png" /></figure><p>Now, in order to make it work during the push to the main branch, you should only replace the branch name on the top of the yml file.</p><pre>on:<br>  push:<br>    branches:<br>      - main</pre><p>Look at your code again, make sure it has good comments, write good Readme.md, and that’s it.</p><p>Now you have one more project for your portfolio.</p><p>Good job!</p><h4>Step 10: Create a DNS in Route 53 and connect it to the EC2 instance.</h4><p>This is an optional step, as a DNS record costs money.</p><ul><li>Proceed to AWS Route 53 and register a new domain.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2U7tro2HyZ7aeJ8e7N6Kfg.png" /></figure><ul><li>Create a new hosted zone.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5HDYY_JseDgb5Dj2kfUKqA.png" /></figure><ul><li>Create A record and route it to the EC2 instance IP.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E8q9U1QZxbW0fm2o8aHwmw.png" /></figure><blockquote>Note that the IP of the EC2 instance is temporary by default, and the instance will get a new IP once restarted. You can purchase a static IP if you want.</blockquote><p>Verify you can access the app by typing the domain name into the browser. It might take some time for AWS to set it up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*erKXhTa7OY_9yvNWfXtA2w.png" /></figure><h4>General Notes:</h4><ul><li>For simplicity, I increment the patch version only for each merge to main in GitHub Actions.</li></ul><p>In case needed, there is an option to increment any version by merging from the branches with specific syntax.</p><p>If pushed from patch/, increment patch version, if pushed from feature/, increment feature version; and so on.</p><pre>if: startsWith(github.head_ref, &#39;patch/&#39;) || startsWith(github.head_ref, &#39;feature/&#39;)</pre><ul><li>I assumed the counter number needed to be permanent, so I added Docker volume to docker-compose.yml.</li></ul><p>From my point of view, it’s a quick and acceptable solution for the current setup.</p><p>However, if there are plans to scale the app or make another usage except proof of concept, you should consider a NoSQL database or an in-memory datastore to maintain the counter state, or both.</p><p>If there is a need for the counter to reset on every container restart, you can remove the volume.</p><p>If there is a need for the counter to reset on the new image version release, this can be achieved by adding version tracking to the Docker image and, in the app, implementing logic that runs at startup to compare the current version (from the Docker image) with the version stored in the persistent volume.</p><ul><li>I chose to use Gunicorn over Flask to run the app because I wanted to keep the option to scale the app or use it for production in the future, although in the current setup, a flask would be more than enough.</li><li>When the build fails, the code owner receives an email notification by default. There is an option to send instant messages via Slack, Signal, MS Teams, Discord, SMS, voice calls, and more.</li><li>I added the “restart:always” key to docker-compose.yml to always restart the container regardless of the exit status.</li><li>I added a memory and CPU limit to the docker-compose.yml, which works well for simple, local development scenarios.</li></ul><p>I hope you enjoyed it.</p><p>Do not hesitate to contact me with any questions.</p><p>You can contact me on <a href="https://www.linkedin.com/in/andrey-byhalenko/">LinkedIn</a> or send me an email at <a href="/service/mailto:andrey@juniordevopshub.com">andrey@juniordevopshub.com</a>.</p><p>If you liked my articles, you can join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=198e34d874b4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/devops-example-project-for-your-resume-198e34d874b4">DevOps Example Project for your Resume. CI/CD for the Python service using GitHub Actions and AWS.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[10 Quick Linux tips and tricks for DevOps: How to Make the Search Insensitive and Others.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/10-quick-linux-tips-for-devops-how-to-make-the-search-insensitive-and-others-9f59a374bb67?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/9f59a374bb67</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[linux-tips]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[linux-tutorial]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Sat, 02 Mar 2024 15:31:28 GMT</pubDate>
            <atom:updated>2024-03-26T08:19:34.687Z</atom:updated>
            <content:encoded><![CDATA[<h3>10 Quick Linux Tips and Tricks for DevOps💡: How to Make the Search Insensitive and Others.</h3><p>Here are some quick tips for better work with Linux, used in my DevOps practice.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tu7KbM3B05o3Ad-T3HcePw.jpeg" /></figure><p>As you know, Linux and the UNIX system are case-sensitive.<br>What if you need to find any specific text but are not sure if it’s lowercase or uppercase?<br> <br>Simply use the grep -i command; it makes the search insensitive to case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1018/1*BxL4S92KNj8ubEvOAkDsRQ.jpeg" /></figure><p>Instead of doing this:</p><pre>mkdir mydir<br>cd mydir</pre><p>Add this function to the ~/.bashrc and call it next time:</p><pre>function mkcd {<br> if [ ! -n &quot;$1&quot; ]; then<br> echo &quot;Enter mkcd followed by a directory name&quot;<br> elif [ -d $1 ]; then<br> echo &quot;\`$1&#39; already exists&quot;<br> else<br> mkdir $1 &amp;&amp; cd $1<br> fi<br>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*2-66JEu_Xpdz_a11dLgZAA.jpeg" /></figure><p>Find all files that were edited in the last 10 minutes:</p><pre>find . -type f -mmin -10</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/841/1*4IBjAJJ_hmsZnBvhLDHnbg.jpeg" /></figure><p>Check the disk usage to find out which files are taking the most disk space:</p><pre>sudo du -hsx /* | sort -rh | head -n 5</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/626/1*2NfnbNH2i_pPDqS87XMcZQ.jpeg" /></figure><p>Run multiple commands with one command:</p><pre>command1 ; command2 ; command3 ; command4</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/532/1*stRYUe9FxovddEUBOyTspQ.jpeg" /></figure><p>Run the following command only if the previous one runs successfully:</p><pre>command1 &amp;&amp; command2 &amp;&amp; command3 &amp;&amp; command4</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*7ow15YIHkhTG-Kuov1fhDg.jpeg" /></figure><p>List files sorted by size:</p><pre>ls -laSh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*Mka0Dfd_ND0frxEh10lLUA.jpeg" /></figure><p>Get the virtual memory statistics report:</p><pre>vmstat 1 5<br># 1 = the values will be re-measured and reported every second.<br># 5 = the values will be reported five times, and then the program will stop.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/655/1*6T9qX9Gp-Dq_SMYfBOqGgA.jpeg" /></figure><p>Display file permissions in a user-friendly format.</p><pre>getfacl</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/370/1*8-9la3wGGyQuG3mKRXER4w.jpeg" /></figure><p>Add a timestamp to your bash history, execute:</p><pre>export HISTTIMEFORMAT=&quot;%d/%m/%y %T &quot;</pre><p>Add this line to the .bashrc if you want to make it permanent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/977/1*XaTzGmVtdlKoZyyYEWhfFA.jpeg" /></figure><p>If you liked my articles, you can join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>As a bonus, you will receive a free step-by-step DevOps CI/CD project, which you can use in your portfolio.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f59a374bb67" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/10-quick-linux-tips-for-devops-how-to-make-the-search-insensitive-and-others-9f59a374bb67">10 Quick Linux tips and tricks for DevOps💡: How to Make the Search Insensitive and Others.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linux tip for DevOps: how to create a directory and move into it with one command.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/devops-linux-tip-how-to-create-a-directory-and-move-into-it-with-one-command-514c98575c05?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/514c98575c05</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[linux-tips]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[linux]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Tue, 20 Feb 2024 13:26:15 GMT</pubDate>
            <atom:updated>2024-03-26T08:19:46.600Z</atom:updated>
            <content:encoded><![CDATA[<p>In this short article, I will share with you the way you can create a directory and move into it with one command.</p><p>I will use my Ubuntu 22.04 server to demonstrate this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tu7KbM3B05o3Ad-T3HcePw.jpeg" /></figure><p>Instead of doing this:</p><pre>mkdir mydir<br>cd mydir</pre><p>Do the following:</p><ul><li>Use any editor you want to add the mkcd function to the bottom of the bashrc file.</li></ul><pre># edit the bashrc file<br>vim ~/.bashrc<br><br># add the mkcd function to the bottom of the file<br>function mkcd {<br> if [ ! -n &quot;$1&quot; ]; then<br> echo &quot;Enter mkcd followed by a directory name&quot;<br> elif [ -d $1 ]; then<br> echo &quot;\`$1&#39; already exists&quot;<br> else<br> mkdir $1 &amp;&amp; cd $1<br> fi<br>}<br><br># save the file and exit</pre><ul><li>Reload the bashrc settings by executing the following command:</li></ul><pre>. ~/.bashrc</pre><ul><li>Call the mkcd function next time you need to create a folder.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/680/1*wPUAqI_MQ2fiS0LkYnMcaw.jpeg" /></figure><p>If you liked my articles, you can join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>As a bonus, you will receive a free step-by-step DevOps CI/CD project, which you can use in your portfolio.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=514c98575c05" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/devops-linux-tip-how-to-create-a-directory-and-move-into-it-with-one-command-514c98575c05">Linux tip for DevOps💡: how to create a directory and move into it with one command.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What are What are RPO and RTO, and what is the difference between them? DevOps interview question.]]></title>
            <link>https://medium.com/devops-technical-notes-and-manuals/what-are-what-are-rpo-and-rto-and-what-is-the-difference-between-them-devops-interview-question-3c05080b5155?source=rss----07c04f098b7e---4</link>
            <guid isPermaLink="false">https://medium.com/p/3c05080b5155</guid>
            <category><![CDATA[it]]></category>
            <category><![CDATA[tor]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[rpo]]></category>
            <dc:creator><![CDATA[Andrey Byhalenko]]></dc:creator>
            <pubDate>Mon, 19 Feb 2024 15:55:53 GMT</pubDate>
            <atom:updated>2024-03-26T08:19:57.484Z</atom:updated>
            <content:encoded><![CDATA[<h3>What are RPO and RTO, and what is the difference between them? DevOps interview question.</h3><p>Questions about RPO (Recovery Point Objective) and RTO (Recovery Time Objective) are often asked in DevOps and IT interviews.</p><p>In this short article, I will explain what RPO and RTO are and what is the difference between the two terms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hp7hhS7053b4wsXaSmRZPg.jpeg" /></figure><p>Both RPO and RTO are part of the disaster recovery terminology.</p><p><strong>RPO (Recovery Point Objective)</strong>: acceptable data loss or how much data loss your application can allow in case of disaster.</p><p><strong>RTO (Recovery Time Objective)</strong>: acceptable downtime, or measure how quickly, after a failure, the application must be available again.</p><p>Imagine you did a backup of your data at 13:00. Unfortunately, the server with your data burned down at 16:00, and you need to recover the last backup on a new server now.</p><p>It took 1 hour to install the new server and restore data from the backup, everything was up and running by 17:00.</p><p>Based on the example, you lost 3 hours of data because you did the last backup at 13:00. The server wasn’t available for an hour.</p><p>So the RPO in this example is 3 hours (which is a lot, not realistic time), and the RTO is one hour.</p><p>That means if the company policy allows 10 minutes of RPO, you need to back up data every 10 minutes.</p><p>For the RTO, the IT department must have clear instructions on how to act in a disaster situation (RTO Policies and Procedures). Who does what, what scripts and backups need to be prepared, in what sequence to run the procedures, etc.</p><p>As you see, the answer to the RPO/RTO interview question is pretty simple, despite the fact that the procedure itself can be very complex and requires clear guidelines and instructions.</p><p>If you liked my articles, you can join my newsletter, and you will receive weekly DevOps tutorials, articles, and tips every Saturday.</p><p>As a bonus, you will receive a free step-by-step DevOps CI/CD project, which you can use in your portfolio.</p><p>Subscribe here: <a href="https://junior-devops-hub.ck.page/">https://junior-devops-hub.ck.page</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c05080b5155" width="1" height="1" alt=""><hr><p><a href="https://medium.com/devops-technical-notes-and-manuals/what-are-what-are-rpo-and-rto-and-what-is-the-difference-between-them-devops-interview-question-3c05080b5155">What are What are RPO and RTO, and what is the difference between them? DevOps interview question.</a> was originally published in <a href="https://medium.com/devops-technical-notes-and-manuals">DevOps Manuals and Technical Notes</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>