<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Moses Musinde on Medium]]></title>
        <description><![CDATA[Stories by Moses Musinde on Medium]]></description>
        <link>https://medium.com/@musinde?source=rss-79430f618a44------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*3TJ56T1bUUzcg6k92hO_ng.jpeg</url>
            <title>Stories by Moses Musinde on Medium</title>
            <link>https://medium.com/@musinde?source=rss-79430f618a44------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:31:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@musinde/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[NETFLIX CLONE DEVSECOPS PROJECT- PART 2]]></title>
            <link>https://medium.com/@musinde/netflix-clone-devsecops-project-part-2-82ff28b80a94?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/82ff28b80a94</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[aws-eks]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[argo-cd]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Thu, 16 Jan 2025 15:31:58 GMT</pubDate>
            <atom:updated>2025-01-16T15:31:58.639Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uezLJtSJN3ltyV_s.png" /></figure><h4>Project Overview</h4><p>In <a href="https://medium.com/@musinde/netflix-clone-devsecops-project-part-1-120551cb9c0c">Part 1</a> of this series, we created a Netflix application and launched it on Docker by the use of Jenkins CICD pipeline. We integrated SonarQube, Trivy, OWASP Dependency Check, Grafana, Prometheus and Email notification.</p><p>In this project, we shall deploy the same Netflix app on Kubernetes leveraging EKS. We will use ArgoCD as the pipeline. Monitoring will be done through Grafana and Prometheus.</p><p><strong>Prerequisites</strong></p><ol><li>Installation of ArgoCD, Prometheus and Grafana</li><li>Knowledge on Kubernetes and EKS</li><li>You should have Helm, AWS CLI and kubectl installed on your PC.</li></ol><h4><strong>Kubernetes Cluster</strong></h4><p>Before creating our EKS Cluster, let us first create the necessary IAM Roles required for the cluster and node groups.</p><ol><li><a href="https://docs.aws.amazon.com/eks/latest/userguide/cluster-iam-role.html#create-service-role">EKSClusterRole</a> with these two permissions:<br> — AmazonEKS_CNI_Policy<br> — AmazonEKSClusterPolicy</li><li><a href="https://docs.aws.amazon.com/eks/latest/userguide/create-node-role.html">AmazonEKSNodeRole</a> with the following permissions:<br> — AmazonEKS_CNI_Policy<br> — AmazonEBSCSIDriverPolicy<br> — AmazonEC2ContainerRegistryReadOnly<br> — AmazonEKSWorkerNodePolicy<br> — AmazonEC2ContainerRegistryPullOnly</li></ol><p>Once the IAM Roles are done, we go ahead to create the Cluster.</p><ul><li>Use Custom configuration option and turn off the EKS Auto Mode.</li><li>Use the IAM Role you created and K8S version 1.28 for this project.</li><li>Use the default VPC and SG</li><li>Use Public cluster endpoint access</li><li>Leave everything else as default and create cluster.</li></ul><p>Wait for the cluster to be created normally between 5 to 10 minutes then create a node group.</p><p>Go to the cluster under compute and select a T2.medium instance. Set the Desired, Minimum and Maximum sizes to 1.</p><p>We will <a href="https://archive.eksworkshop.com/intermediate/290_argocd/install/">install ArgoCD</a> for our cluster from our PC. We first have to set the context to start using the cluster we have created. Run the following commands on the terminal:</p><pre>aws eks update-kubeconfig --name Netflix_Clone --region us-east-1</pre><p>Input the name of your cluster and the region it was deployed in.</p><p>Ensure you have AWS configured, as well as kubectl.</p><pre>kubectl create namespace argocd<br>kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml</pre><p>By default argocd server is not publicly exposed. Let us expose the ArgoCD server to a Loadbalancer:</p><pre>kubectl patch svc argocd-server -n argocd -p &#39;{&quot;spec&quot;: {&quot;type&quot;: &quot;LoadBalancer&quot;}}&#39;</pre><p>After about 2 minutes, the loadbalancer is created:</p><pre>export ARGOCD_SERVER=`kubectl get svc argocd-server -n argocd -o json | jq --raw-output &#39;.status.loadBalancer.ingress[0].hostname&#39;`<br># retrieve the LB endpoint<br>echo ARGOCD_SERVER</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HQFzTVZBZVCLrHm8ioGC2g.png" /></figure><p>Copy the endpoint and open on browser. To login use the username admin and passworsd as illustrated below:</p><pre>export ARGO_PWD=`kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath=&quot;{.data.password}&quot; | base64 -d`<br><br># Retrieve the password<br>echo ARGO_PWD</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9eHuivT1h5XJeML91nTSLw.png" /></figure><p>Navigate to Settings &gt; Repositories &gt; Connect Repo &gt; VIA HTTPS</p><ul><li>Type: git</li><li>Project: default</li><li>Repo URL : <a href="https://github.com/N4si/DevSecOps-Project">https://github.com/N4si/DevSecOps-Project</a></li><li>Connect</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*meJukCnXWF_ujRjfzDTQqg.png" /></figure><p>Create an ArgoCD application that will fetch data from the K8S manifest files in the repo. Make sure to have the path correctly named.</p><p>Name: Netflix</p><p>Project Name: default</p><p>Repo URL: <a href="https://github.com/N4si/DevSecOps-Project">https://github.com/N4si/DevSecOps-Project</a></p><p>Revision: HEAD</p><p>Path: Kubernetes</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WS2_czGHQD_8Tne1e31QZg.png" /></figure><p>deployment.yml</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: netflix-app<br>  labels:<br>    app: netflix-app<br>spec:<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      app: netflix-app<br>  template:<br>    metadata:<br>      labels:<br>        app: netflix-app<br>    spec:<br>      containers:<br>      - name: netflix-app<br>        image: nasi101/netflix:latest<br>        ports:<br>        - containerPort: 80</pre><p>service.yml</p><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: netflix-app<br>  labels:<br>    app: netflix-app<br>spec:<br>  type: NodePort<br>  ports:<br>  - port: 80<br>    targetPort: 80<br>    nodePort: 30007<br>  selector:<br>    app: netflix-app</pre><p>Nodeport is running on port 30007 and therefore we must enable this port on the EKS Node SG.</p><p>node-service.yml</p><pre>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: node-exporter<br>  namespace: prometheus-node-exporter<br>spec:<br>  selector:<br>      app: node-exporter<br>  ports:<br>  - name: node-exporter<br>    protocol: TCP<br>    port: 9100<br>    targetPort: 9100<br>  type: NodePort</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A3e7cj2j4H5u4vfWKU0hdw.png" /><figcaption>ArgoCD Application</figcaption></figure><p>We shall also enable port 9100 in the SG. This is the port required by prometheus node_exprter for metrics data collection.</p><p>Monitor Kubernetes with Prometheus<br>Prometheus is a powerful monitoring and alerting toolkit, and you’ll use it to monitor your Kubernetes cluster. Additionally, you’ll install the node exporter using Helm to collect metrics from your cluster nodes.</p><p>Install Node Exporter using Helm<br>To begin monitoring your Kubernetes cluster, you’ll install the Prometheus Node Exporter. This component allows you to collect system-level metrics from your cluster nodes. Here are the steps to install the Node Exporter using Helm:</p><p>Add the Prometheus Community Helm repository:</p><pre>helm repo add prometheus-community https://prometheus-community.github.io/helm-charts</pre><p>Create a Kubernetes namespace for the Node Exporter:</p><pre>kubectl create namespace prometheus-node-exporter</pre><p>Install the Node Exporter using Helm:</p><pre>helm install prometheus-node-exporter prometheus-community/prometheus-node-exporter --namespace prometheus-node-exporter</pre><p>Add a Job to Scrape Metrics on nodeip:9100/metrics in prometheus.yml:</p><p>The Node_exporter is installed in the EKS Node. Hence the block of code added should use the NodeIP. Prometheus is however installed on another server as per PART 1 tutorial.</p><pre># my global config<br>global:<br>  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.<br>  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.<br>  # scrape_timeout is set to the global default (10s).<br><br># Alertmanager configuration<br>alerting:<br>  alertmanagers:<br>    - static_configs:<br>        - targets:<br>          # - alertmanager:9093<br><br># Load rules once and periodically evaluate them according to the global &#39;evaluation_interval&#39;.<br>rule_files:<br>  # - &quot;first_rules.yml&quot;<br>  # - &quot;second_rules.yml&quot;<br><br># A scrape configuration containing exactly one endpoint to scrape:<br># Here it&#39;s Prometheus itself.<br>scrape_configs:<br>  # The job name is added as a label `job=&lt;job_name&gt;` to any timeseries scraped from this config.<br>  - job_name: &quot;prometheus&quot;<br><br>    # metrics_path defaults to &#39;/metrics&#39;<br>    # scheme defaults to &#39;http&#39;.<br><br>    static_configs:<br>      - targets: [&quot;localhost:9090&quot;]<br>  <br>  - job_name: &quot;node_exporter&quot;<br>    static_configs:<br>      - targets: [&quot;localhost:9100&quot;]<br>  <br>  - job_name: &quot;jenkins&quot;<br>    metrics_path: &quot;/prometheus&quot;<br>    static_configs:<br>      - targets: [&quot;52.86.180.139:8080&quot;]<br>  - job_name: &quot;K8s&quot;<br>    metrics_path: &#39;/metrics&#39;<br>    static_configs:<br>      - targets: [&#39;35.175.246.60:9100&#39;]</pre><p>Reload or restart Prometheus to apply these changes to your configuration.<br>You will see the k8s target on Prometheus:</p><pre>curl -X POST http://localhost:9090/-/reload</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sI4zfKQn1FbSMe5w4b9EbA.png" /></figure><p>Now on the EKS Node compute instance, get the Public IP and use the port 30007 to launch the Netflix application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cuncjjD81aO6pWBJBfSzaQ.png" /></figure><p>Success!</p><p><strong>NOTE: These resources are costly. Make sure to delete:</strong></p><ol><li>EKS Node Group and Cluster</li><li>Load Balancer for the ArgoCD</li><li>Elastic IPs</li><li>Compute Instances</li></ol><p>Feel free to leave a comment in case of any clarification or input.</p><p>Until next article</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=82ff28b80a94" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[NETFLIX CLONE DEVSECOPS PROJECT- PART 1]]></title>
            <link>https://medium.com/@musinde/netflix-clone-devsecops-project-part-1-120551cb9c0c?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/120551cb9c0c</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[devsecops]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Thu, 16 Jan 2025 09:06:01 GMT</pubDate>
            <atom:updated>2025-01-16T09:06:01.182Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KsGD3xsF7VG8t7Z-DZbMJw.png" /></figure><p><strong>Project Overview</strong></p><p>This project is a step-by-step guide on launching a Netflix Clone app on a Docker Container through a CICD Jenkins pipeline and also on a Kubernetes Cluster. To ensure that both our code and applications are secure, we shall integrate security features during the deployment that will scan and cross-check our pipelines. Finally, we will be able to monitor the application and resources utilized using tools such as Grafana and Prometheus.</p><p>We will also be able to receive an email notification to how the status of our deployment whenever any changes occur.</p><p>Note: Due to the length of the project, I will divide the implementation into two parts:</p><ol><li>The Jenkins CICD Pipeline</li><li>The Kubernetes Cluster Deployment.</li></ol><p><strong>Prerequisites</strong></p><ol><li>Installation of Jenkins, Prometheus, Grafana.</li><li>Some knowledge on docker, Jenkins, Prometheus and Grafana</li><li>Knowledge on launching AWS instances and IAM.</li><li>TMDB Account.</li></ol><h3><strong>JENKINS CICD PIPELINE</strong></h3><h4>Part 1: DEV PART</h4><p>Provision TWO EC2 (Ubuntu 22.04) instances with the following requirements:</p><blockquote>i. <strong>Netflix_Clone Instance.</strong></blockquote><ul><li>Instance type: T2.Large</li><li>Enable HTTP/S and SSH</li><li>Storage 25GiB</li><li>Associate an Elastic IP to the instance</li><li>Allow ports 8081 (docker), 9000 (SonarQube) and 8080 (Jenkins) in the Security Group.</li></ul><p>Connect to the instance and update all packages. We shall also clone the below git repo where our code is, and install docker:</p><pre>sudo apt update -y<br><br>https://github.com/N4si/DevSecOps-Project.git<br></pre><pre>#install docker<br>sudo apt-get update<br>sudo apt-get install docker.io -y<br>sudo usermod -aG docker $USER  <br>newgrp docker<br>sudo chmod 777 /var/run/docker.sock</pre><p>Verify the docker installation by running the command “docker version”</p><p>In the “DevSecOps-Project” directory run the below command to create a docker image and container.</p><pre>docker buid -t netflix_clone .<br>docker run -d -p 8081:80 netflix_clone</pre><p>The below Dockerfile will be executed:</p><pre>FROM node:16.17.0-alpine as builder<br>WORKDIR /app<br>COPY ./package.json .<br>COPY ./yarn.lock .<br>RUN yarn install<br>COPY . .<br>ARG TMDB_V3_API_KEY<br>ENV VITE_APP_TMDB_V3_API_KEY=${TMDB_V3_API_KEY}<br>ENV VITE_APP_API_ENDPOINT_URL=&quot;https://api.themoviedb.org/3&quot;<br>RUN yarn build<br><br>FROM nginx:stable-alpine<br>WORKDIR /usr/share/nginx/html<br>RUN rm -rf ./*<br>COPY --from=builder /app/dist .<br>EXPOSE 80<br>ENTRYPOINT [&quot;nginx&quot;, &quot;-g&quot;, &quot;daemon off;&quot;]</pre><p>A docker image will be created as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q79Ifd4Qc1t5owCQADO5GQ.png" /></figure><p>Using the Elastic IP address earlier created and port 8081, we should be able to see a blank Netflix page. In case the page is unavailable, be sure to add the port 8081 to your security group inbound rules.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qeg_Esd08Cj4ygO6wReS4w.png" /></figure><p>Checking our Dockerfile, we notice some arguments that have to be added for the API to connect to TMDB. ie. TMDB_v3_API_KEY. TMDB is the site that we will be using to fetch our movies. TMDB will provide the API required to retrieve the movies. Go to TMDB site and create an account. Navigate to API under settings to create an API Key. Copy the key and recreate your Docker image with the new API Key argument:<br>Run docker stop and docker rm to remove the running container before recreating it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*86hDZARhbg4ALoZOh4qpQg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bPxGVnUbaBVNJNGdIMv4ew.png" /></figure><p>With the KEY now deploy a new image and container:</p><pre>docker build --build-arg TMDB_V3_API_KEY=&lt;your-api-key&gt; -t netflix_clone .<br>docker run -d -p 8081:80 netflix_clone<br><br>#example#<br>docker build --build-arg TMDB_V3_API_KEY=01084d73aea97ae196799b7c7b0dade7 -t netflix_clone .</pre><p>Now our Netflix application should be working as expected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4qT9M2BuaFCDt12mtNORjg.png" /></figure><h4>PART 2: SECURITY</h4><p>We will be installing SonarQube and Trivy.</p><p>SonarQube is a platform for analyzing software for bugs, vulnerabilities, and code smells. In addition to performing a variety of static analysis checks on your source, it presents the results in the form of rich reports that make it easy for you to improve your application’s security and stability.</p><p>Trivy is an open-source vulnerability scanner specifically designed for containers. It is a lightweight and easy-to-use tool that helps identify vulnerabilities in container images and filesystems. Trivy focuses on scanning container images for known vulnerabilities in the installed packages and libraries.</p><pre>#SonarQube<br><br>docker run -d --name sonar -p 9000:9000 sonarqube:lts-community<br><br>#Trivy<br><br>udo apt-get install wget apt-transport-https gnupg lsb-release<br>wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -<br>echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list<br>sudo apt-get update<br>sudo apt-get install trivy</pre><p>Login to SonarQube by port 9000 and username and password set to admin.</p><p>You can check for any vulnerabilities in the Netflix image and the file systems using Trivy.</p><pre># check all file systems<br>trivy fs .<br># check netflix image<br>trivy image netflix_clone</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KqPiD5hi5we_87wJd9F2vQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m52jAjglsWj9htV0wGKOxQ.png" /></figure><h4>PART 3: OPS</h4><p>Install Jenkins</p><pre>sudo apt update<br>sudo apt install fontconfig openjdk-17-jre<br>java -version<br>openjdk version &quot;17.0.8&quot; 2023-07-18<br>OpenJDK Runtime Environment (build 17.0.8+7-Debian-1deb12u1)<br>OpenJDK 64-Bit Server VM (build 17.0.8+7-Debian-1deb12u1, mixed mode, sharing)<br><br>sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \<br>https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key<br>echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \<br>https://pkg.jenkins.io/debian-stable binary/ | sudo tee \<br>/etc/apt/sources.list.d/jenkins.list &gt; /dev/null<br>sudo apt-get update<br>sudo apt-get install jenkins<br>sudo systemctl start jenkins<br>sudo systemctl enable jenkins<br><br>#Verify running status<br>sudo service jenkins status<br>#retrieve pasword for initial login<br>sudo cat /var/lib/jenkins/secrets/initialAdminPassword</pre><p>Using port 8080, open the webpage and login to Jenkins. Install the suggested plugins.</p><p>We will also install the following:</p><ul><li>Eclipse Temurin Installer (Install without restart)</li><li>SonarQube Scanner (Install without restart)</li><li>NodeJs Plugin (Install Without restart)</li></ul><p>After the installation, go to Tools under Manage Jenkins to set them up:</p><ul><li>Under JDK installations:<br> — Name: jdk17 (use the same name as this is the name in the Jenkins CICD pipeline)<br> — Check the Install Automatically box and select Install from adoptium.net<br> — Version: jdk-17.0.8.1+1<br> — Under NodeJS Installations:<br> — Name: node16 <br> — Version: NodeJS16.2.0</li></ul><p>Apply then Save.</p><p>Go to SonarQube under Administration &gt; Security &gt; Users. Under admin user click on “Update Token”. Give your token a name and click generate. Copy it and return to Jenkins.<br> <br> Manage Jenkins &gt; Credentials &gt; System &gt; Global credentials &gt; Create New. Enter details as per screenshot. Paste the token in the “Secret” box.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cK2H0-5iqrW_N_-Je4BsVw.png" /></figure><p>Manage Jenkins &gt; System &gt; Add SonarQube. Server URL should be the same as the SonarQube URL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D73dTsx4TXzcKoiMgUHktw.png" /></figure><p>Go to Tools and configure sonar scanner as per screenshot.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YlyD_FJ3yiliya8VrSFZTQ.png" /></figure><p>In SonarQube. Go to Projects and Manually create the project named “Netflix”. This is referenced in the “SonarQube Analysis” stage of the Pipeline.</p><p>Now we can create our pipeline for the build. Paste the below script under the pipeline box. Save and Build.</p><blockquote><strong>NOTE</strong>: Make sure the naming for the tools is corresponding to the script.</blockquote><pre>pipeline {<br>    agent any<br>    tools {<br>        jdk &#39;jdk17&#39;<br>        nodejs &#39;node16&#39;<br>    }<br>    environment {<br>        SCANNER_HOME = tool &#39;sonar-scanner&#39;<br>    }<br>    stages {<br>        stage(&#39;clean workspace&#39;) {<br>            steps {<br>                cleanWs()<br>            }<br>        }<br>        stage(&#39;Checkout from Git&#39;) {<br>            steps {<br>                git branch: &#39;main&#39;, url: &#39;https://github.com/N4si/DevSecOps-Project.git&#39;<br>            }<br>        }<br>        stage(&quot;Sonarqube Analysis&quot;) {<br>            steps {<br>                withSonarQubeEnv(&#39;sonar-server&#39;) {<br>                    sh &#39;&#39;&#39;$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \<br>                    -Dsonar.projectKey=Netflix&#39;&#39;&#39;<br>                }<br>            }<br>        }<br>        stage(&quot;quality gate&quot;) {<br>            steps {<br>                script {<br>                    waitForQualityGate abortPipeline: false, credentialsId: &#39;Sonar-token&#39;<br>                }<br>            }<br>        }<br>        stage(&#39;Install Dependencies&#39;) {<br>            steps {<br>                sh &quot;npm install&quot;<br>            }<br>        }<br>    }<br>}</pre><p>SonarQube is able to show us all the bugs, vulnerabilities, security hotpots etc in our code. Explore further under the various tabs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-ImeMzl6nooU15I1oQsnLw.png" /></figure><p>Let us install docker and OWASP Dependency Check. Docker will be used to push the image to Docker Hub while OWASP is used to detect publicly disclosed vulnerabilities contained within a project’s dependencies.</p><p>Input your DockerHub credentials in the Credentials section of Jenkins. Username and Passowrd are your dockerhub credentials.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p-gzRmI0bkBTNFWsclj6GA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*80O9dEWdJd19q4boQTjRBA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWLn7-i2xT1wksgChDG3QA.png" /></figure><p><strong>Correction: Name is DP-Check and not with an underscore. Naming syntax is important as the pipeline uses the name to build every stage.</strong></p><p>The previous build is stuck on Quality Gate as some checks are not met.</p><p>Let us modify our pipeline to include Trivy for image scanning, OWASP for dependencies check and Docker for push the image to Dockerhub and building the images.</p><p>We shall delete the docker container so that it can be automatically created by the pipeline.</p><pre>pipeline{<br>    agent any<br>    tools{<br>        jdk &#39;jdk17&#39;<br>        nodejs &#39;node16&#39;<br>    }<br>    environment {<br>        SCANNER_HOME=tool &#39;sonar-scanner&#39;<br>    }<br>    stages {<br>        stage(&#39;clean workspace&#39;){<br>            steps{<br>                cleanWs()<br>            }<br>        }<br>        stage(&#39;Checkout from Git&#39;){<br>            steps{<br>                git branch: &#39;main&#39;, url: &#39;https://github.com/N4si/DevSecOps-Project.git&#39;<br>            }<br>        }<br>        stage(&quot;Sonarqube Analysis &quot;){<br>            steps{<br>                withSonarQubeEnv(&#39;sonar-server&#39;) {<br>                    sh &#39;&#39;&#39; $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \<br>                    -Dsonar.projectKey=Netflix &#39;&#39;&#39;<br>                }<br>            }<br>        }<br>        stage(&quot;quality gate&quot;){<br>           steps {<br>                script {<br>                    waitForQualityGate abortPipeline: false, credentialsId: &#39;Sonar-token&#39; <br>                }<br>            } <br>        }<br>        stage(&#39;Install Dependencies&#39;) {<br>            steps {<br>                sh &quot;npm install&quot;<br>            }<br>        }<br>        stage(&#39;OWASP FS SCAN&#39;) {<br>            steps {<br>                dependencyCheck additionalArguments: &#39;--scan ./ --disableYarnAudit --disableNodeAudit&#39;, odcInstallation: &#39;DP-Check&#39;<br>                dependencyCheckPublisher pattern: &#39;**/dependency-check-report.xml&#39;<br>            }<br>        }<br>        stage(&#39;TRIVY FS SCAN&#39;) {<br>            steps {<br>                sh &quot;trivy fs . &gt; trivyfs.txt&quot;<br>            }<br>        }<br>        stage(&quot;Docker Build &amp; Push&quot;){<br>            steps{<br>                script{<br>                   withDockerRegistry(credentialsId: &#39;docker&#39;, toolName: &#39;docker&#39;){   <br>                       sh &quot;docker build --build-arg TMDB_V3_API_KEY=01084d73aea97ae196799b7c7b0dade7 -t netflix_clone .&quot;<br>                       sh &quot;docker tag netflix_clone musinde/netflix_clone:latest &quot;<br>                       sh &quot;docker push musinde/netflix_clone:latest &quot;<br>                    }<br>                }<br>            }<br>        }<br>        stage(&quot;TRIVY&quot;){<br>            steps{<br>                sh &quot;trivy image musinde/netflix_clone:latest &gt; trivyimage.txt&quot; <br>            }<br>        }<br>        stage(&#39;Deploy to container&#39;){<br>            steps{<br>                sh &#39;docker run -d -p 8081:80 musinde/netflix_clone:latest&#39;<br>            }<br>        }<br>    }<br>}</pre><p>In case of a Docker Login Error during the build, run the below commands on your EC2 terminal:</p><pre>sudo su<br>sudo usermod -aG docker jenkins<br>sudo systemctl restart jenkins</pre><p>Remember to change the Dockerhub credentials to your own and use your own TMDB API KEY in the code above.</p><p>The Build should be successful. Always check the console output to check for any failures.</p><p>Now our application can be automatically deployed from Jenkins.</p><p>On DockerHub, the image is publicly available in our repo.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6jcm1xck2EjOAEcrgkaxRA.png" /></figure><p>ii. MONITORING EC2 INSTANCE</p><ul><li>T2.medium server</li><li>Allow SSH and HTTPS</li><li>Enable ports 3000 (Grafana), 9090 (Prometheus) and 9100 (Node Exporter)</li><li>Assocaiate an EIP with the instance</li></ul><p>Installing Prometheus:</p><pre>sudo useradd --system --no-create-home --shell /bin/false prometheus<br>wget https://github.com/prometheus/prometheus/releases/download/v2.47.1/prometheus-2.47.1.linux-amd64.tar.gz<br><br>tar -xvf prometheus-2.47.1.linux-amd64.tar.gz<br>cd prometheus-2.47.1.linux-amd64/<br>sudo mkdir -p /data /etc/prometheus<br>sudo mv prometheus promtool /usr/local/bin/<br>sudo mv consoles/ console_libraries/ /etc/prometheus/<br>sudo mv prometheus.yml /etc/prometheus/prometheus.yml<br><br>sudo chown -R prometheus:prometheus /etc/prometheus/ /data/<br></pre><p>Create a systemd unit configuration file for Prometheus:</p><p>sudo nano /etc/systemd/system/prometheus.service</p><pre>#######################################################<br>[Unit]<br>Description=Prometheus<br>Wants=network-online.target<br>After=network-online.target<br><br>StartLimitIntervalSec=500<br>StartLimitBurst=5<br><br>[Service]<br>User=prometheus<br>Group=prometheus<br>Type=simple<br>Restart=on-failure<br>RestartSec=5s<br>ExecStart=/usr/local/bin/prometheus \<br>  --config.file=/etc/prometheus/prometheus.yml \<br>  --storage.tsdb.path=/data \<br>  --web.console.templates=/etc/prometheus/consoles \<br>  --web.console.libraries=/etc/prometheus/console_libraries \<br>  --web.listen-address=0.0.0.0:9090 \<br>  --web.enable-lifecycle<br><br>[Install]<br>WantedBy=multi-user.target</pre><p>Here’s a brief explanation of the key parts in this prometheus.service file:</p><p>User and Group specify the Linux user and group under which Prometheus will run.</p><p>ExecStart is where you specify the Prometheus binary path, the location of the configuration file (prometheus.yml), the storage directory, and other settings.</p><p>web.listen-address configures Prometheus to listen on all network interfaces on port 9090.</p><p>web.enable-lifecycle allows for management of Prometheus through API calls.</p><p>Enable and start Prometheus:</p><pre>sudo systemctl enable prometheus<br>sudo systemctl start prometheus</pre><p>Use port 9090 with the EIP to access Prometheus.</p><p>We will install Node Exporter in a similar fashion to Prometheus. Node Exporter is used to monitor hardware and kernel-related metrics on Prometheus.</p><pre>sudo useradd --system --no-create-home --shell /bin/false node_exporter<br>wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz<br><br>tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz<br>sudo mv node_exporter-1.6.1.linux-amd64/node_exporter /usr/local/bin/<br>rm -rf node_exporter*<br><br>sudo nano /etc/systemd/system/node_exporter.service</pre><p>Configuration for the file node_exporter.service</p><pre>[Unit]<br>Description=Node Exporter<br>Wants=network-online.target<br>After=network-online.target<br><br>StartLimitIntervalSec=500<br>StartLimitBurst=5<br><br>[Service]<br>User=node_exporter<br>Group=node_exporter<br>Type=simple<br>Restart=on-failure<br>RestartSec=5s<br>ExecStart=/usr/local/bin/node_exporter --collector.logind<br><br>[Install]<br>WantedBy=multi-user.target</pre><p>Enable and start Node Exporter:</p><pre>sudo systemctl enable node_exporter<br>sudo systemctl start node_exporter<br># Verify the Node Exporter&#39;s status:<br>sudo systemctl status node_exporter</pre><p>You can access Node Exporter metrics:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k82GOahVmpJe8GPBZCa3bg.png" /></figure><p>Configure Prometheus Plugin Integration:</p><p>Integrate Jenkins with Prometheus to monitor the CI/CD pipeline.</p><p>Prometheus Configuration:</p><p>To configure Prometheus to scrape metrics from Node Exporter and Jenkins, you need to modify the prometheus.yml file. Here is an example prometheus.yml configuration for your setup:</p><pre>cd /etc/prometheus<br>sudo nano prometheus.yml</pre><pre># my global config<br>global:<br>  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.<br>  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.<br>  # scrape_timeout is set to the global default (10s).<br><br># Alertmanager configuration<br>alerting:<br>  alertmanagers:<br>    - static_configs:<br>        - targets:<br>          # - alertmanager:9093<br><br># Load rules once and periodically evaluate them according to the global &#39;evaluation_interval&#39;.<br>rule_files:<br>  # - &quot;first_rules.yml&quot;<br>  # - &quot;second_rules.yml&quot;<br><br># A scrape configuration containing exactly one endpoint to scrape:<br># Here it&#39;s Prometheus itself.<br>scrape_configs:<br>  # The job name is added as a label `job=&lt;job_name&gt;` to any timeseries scraped from this config.<br>  - job_name: &quot;prometheus&quot;<br><br>    # metrics_path defaults to &#39;/metrics&#39;<br>    # scheme defaults to &#39;http&#39;.<br><br>    static_configs:<br>      - targets: [&quot;localhost:9090&quot;]<br>  <br>  - job_name: &quot;node_exporter&quot;<br>    static_configs:<br>      - targets: [&quot;localhost:9100&quot;]<br>  <br>  - job_name: &quot;jenkins&quot;<br>    metrics_path: &quot;/prometheus&quot;<br>    static_configs:<br>      - targets: [&quot;52.86.180.139:8080&quot;]</pre><p>Since Node Exporter is on the same EC2 instance as Prometheus, use localhost. For Jenkins, retrieve the same IP and port that Jenkins is connected to.</p><pre># Check the validity of the configuration file:<br>promtool check config /etc/prometheus/prometheus.yml<br># Reload the Prometheus configuration without restarting:<br>curl -X POST http://localhost:9090/-/reload</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VVwCh-wWPUquUnNk5i5Ovw.png" /></figure><p>Node_Exporter has been added as a Target. Jenkins is still DOWN<br>Integrate Jenkins with Prometheus to monitor the CI/CD pipeline.<br>Manage Jenkins &gt; Plugins &gt; Available plugins &gt; Prometheus metrics and then restart Jenkins.<br>Manage Jenkins &gt; System &gt; Prometheus. Leave as default and Apply then Save.</p><p>After this configuration, the Jenkins Target on Prometheus should be UP</p><p>GRAFANA</p><p>Step 1: Install Dependencies:</p><pre>sudo apt-get update<br>sudo apt-get install -y apt-transport-https software-properties-common</pre><p>Step 2: Add the GPG Key:</p><pre>wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -<br>Expect the output to be OK.</pre><p>Step 3: Add Grafana Repository:</p><pre>echo &quot;deb https://packages.grafana.com/oss/deb stable main&quot; | sudo tee -a /etc/apt/sources.list.d/grafana.list</pre><p>Step 4: Update and Install Grafana:</p><pre>sudo apt-get update<br>sudo apt-get -y install grafana<br>Step 5: Enable and Start Grafana Service:</pre><p>Step 5: Enable and Start Grafana Service:</p><pre>sudo systemctl enable grafana-server<br>sudo systemctl start grafana-server<br># Verify the status of the Grafana service to ensure it&#39;s running correctly:<br>sudo systemctl status grafana-server</pre><p>Step 6: Access Grafana Web Interface:</p><p>Open a web browser and navigate to Grafana using your server’s IP address. The default port for Grafana is 3000.</p><p>You’ll be prompted to log in to Grafana. The default username is “admin,” and the default password is also “admin.”</p><p>Step 7: Change the Default Password:</p><p>Step 8: Add Prometheus Data Source:</p><p>To visualize metrics, you need to add a data source. Follow these steps:<br>Click on the gear icon (⚙️) in the left sidebar to open the “Configuration” menu.</p><p>Select “Data Sources.”</p><p>Click on the “Add data source” button.</p><p>Choose “Prometheus” as the data source type.</p><p>In the “HTTP” section:</p><p>Set the “URL” to <a href="http://localhost:9090/">http://localhost:9090</a> (assuming Prometheus is running on the same server).<br>Click the “Save &amp; Test” button to ensure the data source is working.</p><p>Step 9: Import a Dashboard:</p><p>To make it easier to view metrics, you can import a pre-configured dashboard. Follow these steps:</p><p>Click on the “+” (plus) icon in the left sidebar to open the “Create” menu.</p><p>Select “Dashboard.”</p><p>Click on the “Import” dashboard option.</p><p>Enter the dashboard code you want to import. Google the code for node_exporter grafana ie. 1860</p><p>Click the “Load” button.</p><p>Select the data source you added (Prometheus) from the dropdown.</p><p>Click on the “Import” button.</p><p>You should now have a Grafana dashboard set up to visualize metrics from Prometheus.</p><p>Add the Jenkins dashboard by importing it with code 9964. These codes can be retrieved from Google.</p><p>Grafana is a powerful tool for creating visualizations and dashboards, and you can further customize it to suit your specific monitoring needs.</p><p>That’s it! You’ve successfully installed and set up Grafana to work with Prometheus for monitoring and visualization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X7MiuPAI9Bcc72SBhm8y6w.png" /></figure><h4>PART 4: NOTIFICATION</h4><p>Implement Notification Services:<br><br>Step 1: Google App Password</p><p>Go to Manage Your Google Account &gt; Security &gt; Turn on 2-Step Verification. Search for app passwords and create one. We will use the password to send notifications from Jenkins.</p><p>Step 2: Jenkins Credentials</p><p>Create a new credential &gt; username with password &gt; Email address as username and email password as password.</p><p>Step 3: Extended Email Configuration</p><p>Manage Jenkins &gt; System &gt; Extended Email Notification.<br>SMTP server: smtp.gmail.com<br>SMTP Port: 465<br>Under advanced, select the credential , check Use SSL.<br>Default Content Type : HTML<br>Default Triggers: Always and Failure — Any</p><p>Step 4: Email Notification Configuration</p><p>Go to Manage Jenkins &gt; System &gt; E-Mail Notification.<br>SMTP server: smtp.gmail.com<br>Default user e-mail suffix: &lt;your email address&gt;<br>Advanced. Check both Use SMTP Authentication and Use SSL boxes.<br>Username: &lt;email address&gt;<br>Password: App password.<br>SMTP Port: 465</p><p>Update our pipeline by adding the “post” block that will send a notification during any events.<br>Our code now looks like this:</p><pre>pipeline{<br>    agent any<br>    tools{<br>        jdk &#39;jdk17&#39;<br>        nodejs &#39;node16&#39;<br>    }<br>    environment {<br>        SCANNER_HOME=tool &#39;sonar-scanner&#39;<br>    }<br>    stages {<br>        stage(&#39;clean workspace&#39;){<br>            steps{<br>                cleanWs()<br>            }<br>        }<br>        stage(&#39;Checkout from Git&#39;){<br>            steps{<br>                git branch: &#39;main&#39;, url: &#39;https://github.com/N4si/DevSecOps-Project.git&#39;<br>            }<br>        }<br>        stage(&quot;Sonarqube Analysis &quot;){<br>            steps{<br>                withSonarQubeEnv(&#39;sonar-server&#39;) {<br>                    sh &#39;&#39;&#39; $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \<br>                    -Dsonar.projectKey=Netflix &#39;&#39;&#39;<br>                }<br>            }<br>        }<br>        stage(&quot;quality gate&quot;){<br>           steps {<br>                script {<br>                    waitForQualityGate abortPipeline: false, credentialsId: &#39;Sonar-token&#39; <br>                }<br>            } <br>        }<br>        stage(&#39;Install Dependencies&#39;) {<br>            steps {<br>                sh &quot;npm install&quot;<br>            }<br>        }<br>        stage(&#39;OWASP FS SCAN&#39;) {<br>            steps {<br>                dependencyCheck additionalArguments: &#39;--scan ./ --disableYarnAudit --disableNodeAudit&#39;, odcInstallation: &#39;DP-Check&#39;<br>                dependencyCheckPublisher pattern: &#39;**/dependency-check-report.xml&#39;<br>            }<br>        }<br>        stage(&#39;TRIVY FS SCAN&#39;) {<br>            steps {<br>                sh &quot;trivy fs . &gt; trivyfs.txt&quot;<br>            }<br>        }<br>        stage(&quot;Docker Build &amp; Push&quot;){<br>            steps{<br>                script{<br>                   withDockerRegistry(credentialsId: &#39;docker&#39;, toolName: &#39;docker&#39;){   <br>                       sh &quot;docker build --build-arg TMDB_V3_API_KEY=01084d73aea97ae196799b7c7b0dade7 -t netflix_clone .&quot;<br>                       sh &quot;docker tag netflix_clone musinde/netflix_clone:latest &quot;<br>                       sh &quot;docker push musinde/netflix_clone:latest &quot;<br>                    }<br>                }<br>            }<br>        }<br>        stage(&quot;TRIVY&quot;){<br>            steps{<br>                sh &quot;trivy image musinde/netflix_clone:latest &gt; trivyimage.txt&quot; <br>            }<br>        }<br>        stage(&#39;Deploy to container&#39;){<br>            steps{<br>                sh &#39;docker run -d -p 8081:80 musinde/netflix_clone:latest&#39;<br>            }<br>        }<br>    }<br>    post {<br>     always {<br>        emailext attachLog: true,<br>            subject: &quot;&#39;${currentBuild.result}&#39;&quot;,<br>            body: &quot;Project: ${env.JOB_NAME}&lt;br/&gt;&quot; +<br>                &quot;Build Number: ${env.BUILD_NUMBER}&lt;br/&gt;&quot; +<br>                &quot;URL: ${env.BUILD_URL}&lt;br/&gt;&quot;,<br>            to: &#39;musinde*****@gmail.com&#39;, # replace with your email<br>            attachmentsPattern: &#39;trivyfs.txt,trivyimage.txt&#39;<br>        }<br>    }<br>}</pre><p>Now when we build our pipeline we should be able to receive an email update.</p><p><strong>Remember to always delete the containers before a new build as we are using the same port. Failure to do this your build will fail.</strong></p><p>Now let us recap what we have achieved with this project:</p><ol><li>Launched a Netflix app on a docker container</li><li>Fetched movies from TMDB</li><li>Installed and configured SonarQube and Trivy for vulnerability checks</li><li>Installed Jenkins and plugins for CICD.</li><li>Launched a Jenkins pipeline to automatically launch a new Netflix app.</li><li>Installed and configured Prometheus and its Targets for monitoring</li><li>Installed and integrated Grafana for monitoring</li><li>Send email notifications on the status of the deployment from Jenkins.</li></ol><p>Thanks you for walking this journey with me. In case of any questions, feel free to leave a comment and I will be sure to sort you out.</p><p>Watch out for Part 2 where we install the same application on EKS!</p><p>Reference<br><a href="https://github.com/N4si/DevSecOps-Project">https://github.com/N4si/DevSecOps-Project</a></p><p>Until next article</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=120551cb9c0c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[LAUNCHING “2048” GAME ON EKS CLUSTER AWS PROJECT]]></title>
            <link>https://medium.com/@musinde/launching-2048-game-on-eks-cluster-aws-project-13aec8b2f476?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/13aec8b2f476</guid>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[container-orchestration]]></category>
            <category><![CDATA[aws-eks-fargate]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Thu, 14 Sep 2023 01:31:37 GMT</pubDate>
            <atom:updated>2024-04-16T11:05:06.644Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/548/0*TUnAv937bIWU5HgK.png" /></figure><p>Today we shall have a step-by-step end-to-end deployment of the 2048 game on the AWS EKS platform.</p><h3><strong>Prerequisites</strong></h3><ul><li>Familiarity with K8S and AWS</li><li><a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">kubectl</a> — install kubectl command line tool for working with K8S clusters on your PC</li><li>eksctl — install this command line tool for working with EKS clusters and is very useful in automating individual tasks</li><li><a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html">AWS CLI </a>— command line tool for working with AWS services.</li></ul><p>Installation of these tools are beyond the scope of this article. Depending on your the platform you use, you can download and install the tools from the links above.</p><p><strong>Setting up your AWS IAM users</strong></p><ul><li>Go to the IAM (Identity and Access Management) service in the AWS Management Console.</li><li>Click on “Users” in the left-hand navigation pane and then click on “Add user.”</li><li>Enter a username for the new IAM user and select the access type (Programmatic access, AWS Management Console access, or both).</li><li>Choose the permissions for the IAM user by adding them to one or more IAM groups or attaching policies directly. You can use admin permissions for lab purposes but I do emphasize on the least privilege assignment.</li><li>If you selected “Programmatic access” during user creation, you will receive access keys (Access Key ID and Secret Access Key).</li><li>Store these access keys securely, as they will be used to authenticate API requests made to AWS services.</li></ul><p><strong>Configuring AWS CLI and Kubectl</strong></p><p>After downloading and installing AWS CLI as per the prerequisites above, configure the AWS CLI credentials:</p><pre>aws configure</pre><ul><li>Enter the access key ID and secret access key of the IAM user you created earlier.</li><li>Choose a default region and output format for AWS CLI commands e.g. us-west-2 and JSON respectively.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/776/1*1MgbtFL4eAj-qVtBSSUu0A.png" /></figure><p><strong>Installing EKS Cluster using Fargate</strong></p><pre>eksctl create cluster --name cluster-2048-game --region us-west-2 --fargate</pre><p>Fargate is a serverless compute engine that lets you focus on building applications without managing servers. While we can also leverage EC2 instances for installing our EKS cluster, it requires us to manually configure and manage all the compute and memory resources, build your container image, isolate applications in separate VMs etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XYTUzjDyklhGdaO9HIi9Og.png" /></figure><p>Depending on your internet connection, the cluster takes between 10 to 20 minutes to get created. Going back to the AWS console, you can see the cluster has been created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YOUb2CgUHGaTGPLNz50acw.png" /></figure><p><strong>Configuring kubectl for EKS</strong></p><ul><li>Once kubectl is installed, you need to configure it to work with your EKS cluster.</li><li>In the AWS Management Console, go to the EKS service and select your cluster.</li><li>Click on the “Config” button and follow the instructions to update your kubeconfig file. Alternatively, you can use the AWS CLI to update the kubeconfig file:</li></ul><pre>aws eks update-kubeconfig --name your-cluster-name --region your-region</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GC0RaICWnJkcjrivD5v7BQ.png" /></figure><p><strong>Create a Fargate Profile</strong></p><p>By default, the “fp-default” fargate profile is created. The “fp-default” has only two namespaces — default and kube-system. We shall create our own Fargate profile to enable us deploy pods on our own created namespace:</p><pre>eksctl create fargateprofile \<br>    --cluster cluster-2048-game \<br>    --region us-west-2 \<br>    --name alb-sample-app \<br>    --namespace game-2048</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a-miIPrWuEppGQMQx52Y1Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MNVGm3uPz6oRj0hCMeymUQ.png" /></figure><p>Once the fargate profile is created, we proceed to create our deployment using kubectl. The deployment file is linked <a href="https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/examples/2048/2048_full.yaml"><em>here</em></a>. Use the below command to directly create the deployment.</p><pre>kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/examples/2048/2048_full.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jUCZ1YT1ZDnqKKnaPFtnNQ.png" /></figure><p>Verify that the pods/service/ingress are getting created with the command</p><pre>kubectl get pods -n game-2048<br>kubectl get svc -n game-2048<br>kubectl get ingress -n game-2048</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aPiUCteVFETlrCD19Xj83A.png" /></figure><p>For the service, note that we do not have an external IP. Users outside the cluster cannot access our application. We therefore need to create an ingress from which our users can access our 2048 game.</p><p>Analyzing our ingress resource “ingress-2048”, we see that we have a wildcard (*) for the hosts implying that anyone can access. We have a port as well but no address. This is because our ingress resource requires an ingress controller to be deployed, which then reads the ingress resource creates and configures the load balancer before an IP address is assigned.</p><p>One prerequisite before creating an ingress controller is configuring IAM OIDC provider. The ingress controller (ALB controller) needs to access the Application Load Balancer. Note that a controller is just a K8S pod which needs to talk to AWS resources. This can only be achieved by providing the necessary IAM permissions hence the use of a connector (OIDC). Run the below command to associate the identity provider with our cluster:</p><pre>eksctl utils associate-iam-oidc-provider --cluster cluster_name --approve --region region_name</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QBSoqKBGXnA1Fv8SY3lNkQ.png" /></figure><p>Our ALB controller pod should have access to AWS services such as ALB in order to create the service. Let us grant permissions by running the <a href="https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json"><em>iam_policy.json</em></a> file from the official ALB controller documentation.</p><pre>curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json</pre><p>Create an IAM policy:</p><pre>aws iam create-policy \<br>    --policy-name AWSLoadBalancerControllerIAMPolicy \<br>    --policy-document file://iam_policy.json</pre><p>Create IAM Role. Remember to modify the AWS account ID and cluster name.</p><pre>eksctl create iamserviceaccount \<br>  --cluster=&lt;your-cluster-name&gt; \<br>  --namespace=kube-system \<br>  --name=aws-load-balancer-controller \<br>  --role-name AmazonEKSLoadBalancerControllerRole \<br>  --attach-policy-arn=arn:aws:iam::&lt;your-aws-account-id&gt;:policy/AWSLoadBalancerControllerIAMPolicy \<br>  --approve \<br>  --region=us-west-2</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dLRaNT6D2MJM8W8ENXUpTA.png" /></figure><p>Finally we can deploy the ALB Controller. We shall use a helm chart to create the controller and use the service account (iamserviceaccount) we created earlier for running the pod.</p><p>Let us add helm to our repo and update our helm repo</p><pre>helm repo add eks https://aws.github.io/eks-charts<br>helm repo update eks</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lpLTX1pE-xMNKhI4GlxxrA.png" /></figure><p>Now we can install the controller. Again, remember to fill in the variables as per your account and cluster details. VPCId is located in the Networking tab of your EKS cluster.</p><pre>helm install aws-load-balancer-controller eks/aws-load-balancer-controller -n kube-system \<br>  --set clusterName=&lt;your-cluster-name&gt; \<br>  --set serviceAccount.create=false \<br>  --set serviceAccount.name=aws-load-balancer-controller \<br>  --set region=&lt;region&gt; \<br>  --set vpcId=&lt;your-vpc-id&gt;</pre><p>Verifying the load balancer has been created:</p><pre>kubectl get deployment -n kube-system aws-load-balancer-controller</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2X01mlvxE7UZkt_moRE_MA.png" /></figure><p>We can see below that two replicas of the ALB have been created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X5tBQLkredSLzKieut0jNg.png" /></figure><p>From the Dashboard, we can verify that the LB has been created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MMdCnvK4AlW9VhfOxay8mg.png" /></figure><p>Now, if we check the ingress resource, we should see the LB in the Address section:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-uKCnaVUqUrPq-7Vl8Hwvw.png" /></figure><p>We can use this address in a browser and we should finally have our game!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vxHFBzfGSdQhZ_x4Lobo6g.png" /></figure><p>We have successfully deployed our 2048 game publicly. Remember to delete your EKS cluster after playing around with the game ;)</p><pre>eksctl delete cluster --name demo-cluster --region region-name</pre><p>Until next article</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13aec8b2f476" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[KUBERNETES FUNDAMENTALS]]></title>
            <link>https://medium.com/@musinde/kubernetes-fundamentals-a0c94ef7aefd?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/a0c94ef7aefd</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Wed, 05 Jul 2023 01:02:51 GMT</pubDate>
            <atom:updated>2024-04-16T11:06:00.370Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*C2VnfMUKaKTIdnNC" /></figure><p><strong>What is Kubernetes?</strong></p><p>Kubernetes is a container orchestration platform that offers capabilities such as auto scaling, auto healing, clustering and enterprise level support like load balancing. It is these capabilities that makes Kubernetes (K8s) such an important open-source containerization platform in comparison to Docker. K8s leverages multi-environment multi-container deployments.</p><p><strong>K8S ARCHITECTURE</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*U32t0d2aYyyPK5uh.png" /></figure><p>Kubernetes is divided into two main components — Control plane and Data Plane. Control plane makes global decisions about the cluster (a k8s deployment) and monitors and responds accordingly to the events of the cluster. A Data Plane implements the instructions forwarded from the control plane.</p><p>Control plane is composed of the following:</p><ol><li><strong>API Server-</strong> responsible for handling interactions with the user and is essentially the front end of the control plane</li><li><strong>Scheduler- </strong>handles the allocation of pods to nodes in a cluster.</li><li><strong>ETCD- </strong>This is an object tool that is highly available. All resources are stored in key-value pairs</li><li><strong>Control Manager- </strong>ensures that the actual state matches the desired state. In non-cloud setup, we use default k8s controllers e.g. replica sets but in the cloud, Cloud Controller Manager (CCM) is used. CCM is curated by specific cloud providers to integrate K8s into their platforms.</li></ol><p>Data Plane is composed of the following:</p><ol><li><strong>Kubelet- </strong>responsible for the management of pods (stopping, restarting, health)</li><li><strong>Kube-proxy-</strong> the networking component designed to manage the services such as updating the IP routing table for the pods.</li><li><strong>Container Engine (runtime)- </strong>responsible for starting and stopping containers, pulling images and exposing containers to ports specified in the manifest. Include <em>docker</em><strong>, </strong><em>containerd</em> and <em>CRI-O. </em>These runtime examples are not available out-of-the-box therefore they must be installed<em>.</em></li></ol><p><strong>What is a pod?</strong></p><p>A pod is the smallest deployable unit of computing that can be created and managed in k8s. It is a runtime specification of a container in Docker. Typically, you can run more than one container in a pod. All containers in a pod share resources.</p><h3><strong>PROJECT TIME!</strong></h3><p>We have understood the basic concepts behind kubernetes, but that is definitely not all. The rest of the concepts will be better of explained in a practical scenario. But first, we need to set up our environment (cluster) from which we will take advantage of the goodness that is K8s!</p><p>For our projects derived from the <a href="https://github.com/LondheShubham153/90DaysOfDevOps/blob/master/2023/day30/tasks.md">#90DaysofDevOps challenge</a>, we shall use minikube. Minikube is a dumb-down version of k8s that is lightweight and easy to deploy on our machines either as a VM, container or bare metal. It has multiple container runtimes and is cross-platform.</p><p>I shall be running minikube on Ubuntu but this <a href="https://minikube.sigs.k8s.io/docs/start/">guide</a> should be helpful in installing it anywhere.</p><h3><strong>Creating a pod</strong></h3><ol><li>Create a YAML file (pod.yaml)</li><li>Run command to create pod: kubectl apply -f pod.yaml</li><li>Run command to view pod: kubectl get pod</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/1*6GHIrprvxFEX1-4cgHk2Kw.png" /></figure><p>Remember that a pod is a runtime specification of a container in docker. Therefore, the pod.yaml file equivalent in docker would be:</p><pre>docker run -d nginx:1.14.2 --name nginx -p 80:80</pre><h3><strong>Use of Deployment</strong></h3><p>The pod.yaml file is a lot of work compared to the simple one-line command option docker provides. In K8s, we do not deploy pods as we have previously done. We use <em>Deployments.</em></p><p>A deployment is a YAML manifest file that contains the desired configuration state of resources. When applied, the deployment controller changes the actual state to the desired state. A deployment, when applied, usually creates an intermediate state called a <em>replica set (rs)</em> that in turn triggers the creation of pods. This keeps track of the actual vs desired state and always works to reconcile the two in what is called auto scaling or auto healing. Let us demonstrate this below:</p><p>We create a deployment file: deployment.yaml and apply using the command kubectl apply -f deployment.yaml.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2erbbbyLH5RfqitN-FU9rg.png" /></figure><p>We have 3 replicas running.</p><ol><li>Auto scaling</li></ol><p>Now, let us edit our file and reduce the number of replicas to 2 and apply it again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5ruIdqWP6nOI8INSvpyrTQ.png" /></figure><p>We see that one pod is terminated to keep up with the deployment file.</p><p>2. Auto Healing</p><p>Let us now delete one of the pods and observe what happens.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*05qqgwk7H2T51_QuTWDCSw.png" /></figure><p>K8S automatically starts a new pod. See the AGE of the new pod.</p><h3><strong>Namespaces and Services</strong></h3><p>Namespaces are used to create isolated environments for resources. Each namespace represents a virtual separate cluster within the physical cluster.</p><p>Services are used to expose your pods and deployments to the network. Labels are used as an abstraction ensuring that the ever-changing IPs assigned to pods do not affect the reachability from within the cluster, to the outside world, or to specific IP addresses or ranges.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*d7rBdgDqgVSLW6nE.png" /></figure><p>Let us create a namespace and update our deployment to include the namespace</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*04M72bHOq9p-vxqz18ojxw.png" /></figure><p>Notice now that when we apply the deployment, or want to check the pod status, we specify the namespace by either the — namespace flag or the -n flag followed by the name of the namespace.</p><p>To verify that the namespace is created:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/952/1*tPTSenxojiGTFkDE54jXeg.png" /></figure><p>The other namespaces are created by default within the minikube cluster. It is best practice to NOT assign your namespaces with the “kube-” notation.</p><p>Let us create a service for the todo app deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*P6ud5N_kl5NzJFMOLwZ3hQ.png" /></figure><p>Notice that Service type is set to NodePort. This will make the Service accessible from outside the cluster by using the IP address of any node in the cluster and the nodePort defined in the Service.</p><p><strong>spec: selector: app: todo</strong>: This section defines the selector for the pods that the Service should target.</p><p>Apply the service definition and verify the service</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G55vrbySQPuC1d22bodVTg.png" /></figure><p>I have used the -A flag to list all the services available. Alternatively, you can use the namespace flag to be more granular with your output.</p><p>To verify that we can access our todo app using the service IP and port in the namespace:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*drwWf7nKcCu7yKq2UffJog.png" /></figure><pre>minikube service &lt;service_name&gt; -n &lt;namespace_name&gt; --url <br>curl -L &lt;service_ip&gt;:&lt;port&gt;</pre><p>What if we only wanted to access the app within the Cluster? Well, this is usually the default, if we do not specify any Type option. See the clusterIP-todo-service.yaml file created and applied below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S9gQRv8jS5DtIDt8_0Qrcw.png" /></figure><p>Let us verify that we can access it through the IP provided. For this, we shall have to go inside one of the pods.</p><pre>kubectl exec -i -t &lt;pod_name&gt; -n &lt;namespace_name&gt; -- sh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XNFnJYjy2IhLxy5D0o4gMA.png" /></figure><p>Make a curl request to the Cluster-IP Service using its IP address:</p><pre>wget -q -O - http://10.105.116.104:80</pre><p><strong>LoadBalancer Service</strong></p><p>Load balancing is a technique used to distribute incoming network traffic across multiple servers to ensure that no single server is overwhelmed with requests. In Kubernetes, load balancing is achieved through the use of a load balancer service. The load balancer service routes incoming traffic to a set of backend Pods that are part of the service. Kubernetes provides several options for load balancing, including round-robin, session affinity, and IP-based balancing.</p><p>Let us create a LB service to access the todo app from outside our cluster.</p><p>We start by creating a LoadBalancer definition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aDyLKn4czreeB_t9neAKDA.png" /></figure><p>Now verify that you can use the IP and port to see your todo app</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dnUqmi70ZhefKKQnmuRpvA.png" /></figure><h3><strong>ConfigMaps and Secrets</strong></h3><p>A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume. A ConfigMap allows you to decouple environment-specific configuration from your container images, so that your applications are easily portable.</p><p>As an example, we shall create a ConfigMap for our todo app deployment, apply it to the pinkpanther namespace and update our deployment.yaml file to include the ConfigMap. This is shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jpitg3xrzUb0A2IanRZtqw.png" /></figure><p>We then verify that the configmap has been created as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NYXijQT1_78crSJKxSpfdw.png" /></figure><p>To see the key-value pairs of an environment variable in a ConfigMap inside a cluster or a pod, you can use the following command:</p><pre>kubectl get pod -n &lt;namespace-name&gt;<br>kubectl exec -i -t &lt;pod-name&gt; -n &lt;namespace-name&gt; -- sh</pre><p>Use the <strong>printenv inside the pod to see the key-value pairs available:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5yznq4e5ymbwFd83OfymLg.png" /></figure><p>This is a lot of information. using the <strong>grep </strong>parameter, we can filter the data to our needs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/396/1*R31DiLhXUeS__VD01HVLoQ.png" /></figure><p>A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image. Using a Secret means that you don’t need to include confidential data in your application code. Because Secrets can be created independently of the Pods that use them, there is less risk of the Secret (and its data) being exposed during the workflow of creating, viewing, and editing Pods. Kubernetes, and applications that run in your cluster, can also take additional precautions with Secrets, such as avoiding writing secret data to nonvolatile storage.</p><p>Let us create a basic secret and add it to out deployment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lPW0GXJbOpW_quMupl2U2Q.png" /></figure><p>In this example, we’re creating a Secret called secret-basic-auth with two keys: username and password. Updating the deployment involves referencing the secret using the valueFrom field in the container&#39;s env list or as a volume. Here&#39;s an example of using the Secret as environment variables:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*02EXh5zBZoQ5tWp4p7dtdQ.png" /></figure><p>ThevalueFrom field with the secretKeyRef object are used to get the values for the DB_USERNAME and DB_PASSWORD environment variables from the secret-basic-auth Secret.</p><p>Verifying that the secret has been created and describing the contents of the secret we get:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gOl-1oInCHgpwKyFiLZ9zA.png" /></figure><p>To see the key-value pairs of an environment variable in a ConfigMap inside a cluster or a pod.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uZli_mWPZjv2atBhO1Cx4A.png" /></figure><h3><strong>Persistent Volumes</strong></h3><p>A <em>PersistentVolume</em> (PV) is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes. It is a resource in the cluster just like a node is a cluster resource. PVs are volume plugins like Volumes, but have a lifecycle independent of any individual Pod that uses the PV. This API object captures the details of the implementation of the storage, be that NFS, iSCSI, or a cloud-provider-specific storage system.</p><p>A <em>PersistentVolumeClaim</em> (PVC) is a request for storage by a user. It is similar to a Pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific size and access modes (e.g., they can be mounted ReadWriteOnce, ReadOnlyMany or ReadWriteMany, see AccessModes).</p><h3>Task 1:</h3><p>Add a Persistent Volume to your Deployment todo app.</p><ul><li>Create a Persistent Volume using a file on your node.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GQmICDxk6qYVZoHb6KqmPg.png" /></figure><ul><li>Create a Persistent Volume Claim that references the Persistent Volume.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VKMVXB7VKkBB5Xb8--d7Eg.png" /></figure><p>Update the deployment file to include the PVC. Ensure the names match otherwise the pods will remain in pending state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tu4enUQPDYAmLABfPC7UbQ.png" /></figure><h3>Task 2:</h3><p>Accessing data in the Persistent Volume,</p><ul><li>Connect to a Pod in your Deployment using command : `kubectl exec &lt;podname&gt; -it — /bin/bash</li></ul><pre>kubectl exec -it todo-app-deployment-8645b4d6d7-vsvgt -- sh</pre><p>below inside the /tmp/data/ folder create a new file named test.txt</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zp9FEwxnIGURP9yK8JNLpQ.png" /></figure><ul><li>Verify that you can access the data stored in the Persistent Volume from within the Pod</li></ul><p>delete a pod which we used in above step and create a new pod using command kubectl apply -f deployment.yaml</p><p>Go inside a pod using kubectl exec command, then go to the folder and check test.txt file.</p><pre>kubectl delete -f deployment2.yaml<br>kubectl get pod <br>kubectl apply -f deployment2.yaml<br>kubectl get pod<br>kubectl exec -it todo-app-deployment-8645b4d6d7-8hhw7 -- sh</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4r_1M6M6bcJtzxoyWpfZpQ.png" /></figure><h3><strong>Conclusion</strong></h3><p>We have managed to successfully complete every single K8S task in the #90DaysofDevOpsChallenge. Thank you for walking with me throughout the journey and I hope it has been as captivating to you as it was for me.</p><p>Feel free to leave a comment or a question below and I will be sure to respond.</p><p>Until next article,</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a0c94ef7aefd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[JENKINS FUNDAMENTALS]]></title>
            <link>https://medium.com/@musinde/jenkins-fundamentals-bb247dc5adfd?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/bb247dc5adfd</guid>
            <category><![CDATA[jenkins]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[aws-ec2]]></category>
            <category><![CDATA[jenkins-pipeline]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Mon, 29 May 2023 19:32:35 GMT</pubDate>
            <atom:updated>2024-04-16T11:07:00.019Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>What is Jenkins?</strong></h3><p>Jenkins is an open-source CI/CD server used for automation. The tool is written in Java and it essentially allows developers to build, test and deploy software.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/0*RSgnwWuDv9VkLFtC.jpg" /></figure><p><strong>Why Jenkins?</strong></p><p>We have many CI/CD systems — Gitlab CI/CD, GitHub CI/CD, CircleCI, Travis CI, Bitbucket pipeline etc. Each tool has a different unique functionality and can be leveraged based on the scenario at hand. Jenkins, however, has a few advantages that makes it preferred in a general sense as compared to the competition.</p><p>To begin with, installation and configuration is swift and simple as we shall see when we do the set up. The platform has a simple web-based administration that allows for much more flexible build jobs and pipelines. It also uses plugin ecosystem for customization and flexibility. Jenkins offers scalability though distributed build agents and executors. Finally, the active user base and excellent online documentation is particularly helpful.</p><h4><strong>Jenkins Terminology</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sCPb8Fql1G67GFFS.png" /></figure><ol><li>Jenkins Master</li></ol><p>Jenkins controller/master is a central service from which the administration console is served from. It is responsible for the configuration of build jobs, pipelines, and plugins via the web administration console that it serves. Build jobs are scheduled and dispatched to build agents by the master.</p><p>2. Build Agent</p><p>Also know as the Slave, a build agent is a separate server that receives instructions in the form of build job dispatches from the Jenkins master. Each Agent can run its own OS, as illustrated in the diagram above.</p><p>3. Build Project</p><p>Defines the build configuration properties required to build a unique piece of software. Different project types exist for different build requirements. The current build project options that Jenkins provided are:</p><ul><li>Freestyle project</li><li>Pipeline</li><li>Multi-configuration project</li><li>Folder</li><li>GitHub Organization</li><li>Multibranch Pipeline</li></ul><p>Jenkins web-based platform offers a vivid description of each project option.</p><p>4. Label</p><p>User-defined metadata used to group together different build agents. For example, labels can be used group Build Agents by OS or by build technology.</p><p>5. Node</p><p>Defines by Label a machine which is capable of executing a build pipeline or project. This defines where the execution should take place. Nodes are used within S<strong>cripted pipelines.</strong></p><p><strong>6. </strong>Agent</p><p>Defines by Label a machine which is capable of executing a build pipeline or project. This defines where the execution should take place. Agents are used within the <strong>Declarative pipeline.</strong></p><p>7. Artifact</p><p>Output file generated as an outcome of a build. It could be a Java JAR file, a .NET assembly, or just a plain old ZIP file. Considered immutable, and Artifact should not be altered externally to the build process that created it. It is tracked and traced by applying a fingerprint, which is a checksum over the Artifact file in question.</p><p>8. Pipeline</p><p>A pipeline is a modelling concept that embodies and encapsulates the core build logic and workflow. Jenkins supports the creation of pipelines using two <a href="https://dsstream.com/declarative-vs-scripted-pipeline-key-differences/">different syntax</a>:</p><ul><li>Scripted Pipelines</li></ul><p>Created using the Groovy Language, providing you with freedom and flexibility to build very customized pipelines. However, to be productive requires you to be knowledgeable in Groovy. The extra required learning curve may not be desirable to build and release support personnel unfamiliar with Groovy.</p><ul><li>Declarative Pipelines</li></ul><p>An opinionated form of Syntax, meaning that they are more defined with a guiding structure which must be followed. They are less flexible than scripted pipelines, but easier to work with if lacking Groovy knowledge.</p><p>CI/CD (Continuous Integration/Continuous Deployment) pipeline is a set of automated processes that helps to integrate code changes, build, test, and deploy applications continuously. The primary goal of a CI/CD pipeline is to enable fast and reliable delivery of changes to production.</p><p>A typical CI/CD pipeline consists of several stages, including:</p><ol><li>Code Integration: In this stage, developers integrate their code changes into a shared repository.</li><li>Build: In this stage, the CI system builds the code and runs any necessary tests.</li><li>Test: In this stage, the code is tested using various testing techniques, including unit tests, integration tests, and end-to-end tests.</li><li>Deployment: In this stage, the code is deployed to production or a staging environment.</li><li>Monitoring: In this stage, the system monitors the deployed application for performance and stability.</li></ol><p>9. Stage</p><p>Used to segment distinct subsections of the overall pipeline. Defining individual stages has several benefits:</p><ul><li>Allows you as the author of the pipeline to apply structure</li><li>Used to provide build visuals to highlight progress during build execution</li></ul><p>Typical stages model key and important stages within the CI/CD cycle, such as Clone, Build, Test, Package and Deploy.</p><p>An individual Pipeline stage may consist of multiple pipeline <strong>steps.</strong></p><p>10. Step</p><p>A pipeline step defines a single build task and what specifically should happen. A pipeline stage typically wraps multiple pipeline steps.</p><p>11. Workspace</p><p>A dedicated file system directory where build work is performed for the current pipeline or build job. When troubleshooting builds, this particular directory and the sub directory structure within it is useful to interrogate. The workspace will be created on a particular build agent for which the pipeline or build job has been configured to run on.</p><p>12. Downstream</p><p>Represents another pipeline or build job that will be triggered as a consequence of the current running pipeline or build job.</p><p>13. Upstream</p><p>Refers to the pipeline which triggered the currently running pipeline or build job.</p><p>Jenkins Installation</p><p>Here is a blog post on how to install Jenkins on AWS</p><blockquote><a href="https://www.trainwithshubham.com/blog/install-jenkins-on-aws">https://www.trainwithshubham.com/blog/install-jenkins-on-aws</a></blockquote><p>Remember to save the “initialAdminPassword” content for initial log in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*pK3ZpIyZznypBVOAxoRtQA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*o_gF7eDaeD5SAId9gILe1g.png" /></figure><p>Use the suggested plugin to install commonly used plugins:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*NUp_zDR_b7bXQv_-p4vCMw.png" /></figure><p>After the plugins have downloaded, create your credentials and click “Save and Continue” to got to your dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Xnh_Z64Osk_MzsmnCt1pg.png" /></figure><h3>PROJECTS</h3><p>Now that we are familiar with the Jenkins basics, let us get our hands dirty with some tasks and projects from the <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704">#90DaysofDevops</a> challenge between Days 22 to 29.</p><blockquote><strong>Create a freestyle pipeline to print “Hello World!!</strong></blockquote><ol><li>Create a new item using the Freestyle project. Give it a name, say <em>HelloBuildJob. </em>Click OK on the bottom right.</li><li>Once project has been created, navigate to the Configuration section and farther to the “Build Steps” section where you will add “Execute Shell” option.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/979/1*H5ED3NX3pSsCCvue0zRsXw.png" /></figure><p>3. Add the command to print “Hello World!!”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/689/1*ehi2_7nO5K4LoyOvLRdQXw.png" /></figure><p>4. Save</p><p>5. Now, click on “Build Now”. If all is well, you will see a green tick with the #1 next to it to show the first build was successful.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/350/1*6RKGxTvZraKVCZhEOFb6og.png" /></figure><p>6. Click on the build number and check the console output for the “Hello World” text.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*AxiRZoMvNzHGOYSLg2Ue7w.png" /></figure><p>That is it for the first project!</p><h3><strong>Task-01</strong></h3><p>Create an agent for your app.</p><p>Create a new Jenkins freestyle project for your app.</p><ul><li>App is a to-do app present in the git repo <a href="https://github.com/Moses-Mathias/node-todo-cicd">here</a>.</li><li>As before we shall create a new item called toDoApp and choose Freestyle Project as the build job. Click OK</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mZx-T_lCQM7SnBau0wAPqQ.png" /></figure><ul><li>In the configuration tab, select GitHub Project and enter the GitHub URL. Do the same under the “Source Code Management” section.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uBYGuyl_8CRkrsyKv9pykw.png" /></figure><ul><li>Leave the rest as default as the source code is in the default master branch. If not, feel free to change the branch</li><li>Under build steps, add the command to build docker image for the container.</li><li>Also, add a second command to to start a container using the image created above.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1kOAsGx6NAdCVOmYvnvvCQ.png" /></figure><pre>docker build -t todoapp .<br>docker run -d --name todoapp_c -p 8000:8000 todoapp</pre><ul><li>Now you can build the project by clicking the “build now”.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kC2Fkb0uuE9HmFeTZqNREA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*jVdLKSC99QJgpvVu4fDGbQ.png" /></figure><ul><li>Examining the workspace folder under /var/lib/jenkins/workspace, we see that the source code has been cloned on the machine.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sYd7YVstVY8tr5K6qAGJUA.png" /></figure><h3><strong>Task 02</strong></h3><ul><li>Create Jenkins project to run “docker-compose up -d” command to start the multiple containers defined in the compose file.</li><li>Create docker-compose.yml file inside your project folder.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*AqQ1Ypl28PDFNaFnkqK58w.png" /></figure><ul><li>Add a build step “docker-compose down” to tear down any containers defined in the docker compose file. This helps to prevent conflicting containers</li><li>Now add “docker-cmpose up -d” to spin up new containers as per the docker-compose file</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U4bWsXAX4y1eTodgpD3fjA.png" /></figure><ul><li>The container has been started as below:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Cr5-DozLndgUm39ZTA-6eQ.png" /></figure><h3><strong>COMPLETE JENKINS CI/CD PROJECT</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*eKJ26bwGY_9RwXpm.png" /></figure><p>This project is derived from day 24 and 25 of <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704"><strong>#90DaysofDevops</strong></a><strong> challenge.</strong></p><p>We shall be deploying a node.js application on an EC2 instance and creating a CI/CD pipeline using Jenkins. We shall then create a connection to our Jenkins job and our GitHub repo via GitHub integration. Therefore, make sure you have a running EC2 instance with Jenkins and Docker installed and a GitHub repo.</p><p>In order for us to integrate Jenkins with GitHub, we are required to provide a public SSH key to GitHub. Do this by running the command <strong><em>ssh-keygen</em></strong> in the EC2 instance and copying the public key contents. Use the <strong><em>cat</em></strong> command to display the contents in the id_rsa.pub file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*iOO3MKgbWb1zVcqDJTd7KQ.png" /></figure><p>Navigate to GitHub -&gt; Settings -&gt; SSH and GPG Key _&gt; New SSH Key -&gt; Title (Name the Key) -&gt; Paste the contents of the id_rsa.pub file -&gt; Add SSH Key</p><p>In Jenkins, create a GitHub project and paste the GitHub repo URL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ulkGXzBm3E730nlApHwxgg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3peq8KnAgxL4FYcgqQ5wrA.png" /></figure><p>Go to Add credentials and input credentials as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vy6qAnHlMz8cYSM-M9PLEg.png" /></figure><p>Note that your username is the username used for the EC2 instance created. Copy the private key from your EC2 instance and paste it as shown.</p><p>Save your item and build it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vpf5k91qT9v96Yzz3_iFyA.png" /></figure><p>Now we need to install dependencies as per the README.md file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/559/1*RXY5fX5L8Fm8ErD2e3wzOQ.png" /></figure><p>The last command provides a URL from which the app is running. This port is not allowed in our security group and so we need to add it before accessing it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*JSS1CaCMSo0fgZToi1K_0Q.png" /></figure><p>Navigate to Instance -&gt; Security -&gt; Security groups -&gt; Edit inbound rules -&gt; Add rule -&gt; Save</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HfEK4Hovgz3IAECcxHQPMw.png" /></figure><p>Now, take the public IP of the EC2 instance and add port 8000 to access the application on a browser.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/935/1*vJXgQCv7T-4f9Kb2CNnU8w.png" /></figure><p>Now, let us create a docker application of the nodejs to-do app that can be accessed anywhere by anyone. Assuming we have docker installed:</p><p>Create your Dockerfile:</p><pre>FROM node:12.2.0-alpine<br>WORKDIR app<br>COPY . .<br>RUN npm install<br>RUN npm run test<br>EXPOSE 8000<br>CMD [&quot;node&quot;,&quot;app.js&quot;]</pre><p>Let us build the image from the Dockerfile</p><pre>docker build -t todoapp .</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*LUCf0JO-q5ZAPy5oan4i5Q.png" /></figure><p>Create a container:</p><pre>docker run -d --name todoapp_c -p 8000:8000 todoapp:latest</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b-_rYJAfxVNyXJp3u3nTig.png" /></figure><p>Let us try accessing the app from the public IP and port 8080. It should be accessible.</p><p>We now automate the process by adding the commands in Jenkins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UeF2hWvbB7JcVP3YV2xkFA.png" /></figure><p>Build the project and access it from the browser as before</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jJJrVaUi1pLek2xfjx6tXw.png" /></figure><p>The task is automated now. All we need to do is configure a web-hook which will handle the changes in the repository by automatically triggering the job. Remember to kill all containers otherwise the builds will be unsuccessful due to conflict.</p><p>Navigate to Jenkins -&gt; Manage Plugins -&gt; System Configuration -&gt; Plugins -&gt; Available Plugins -&gt; Search for GitHub Integration plugin -&gt; Install.</p><p>On GitHub, go to Repo settings -&gt; Webhooks -&gt; Add Webhook -&gt; Payload URL (Jenkins URL) -&gt; Content type -&gt; application/json -&gt; Add webhook</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*OZInoFhwdnEh21C8ClcYRQ.png" /></figure><p>In Jenkins under the Project, go to configure and select GitHub hook trigger option then save.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/494/1*tx-ZaJb23tvoveBmboqg8A.png" /></figure><p>Update something in the repo and your job will be automatically triggered.</p><p>That sums up our Jenkins projects.</p><p>Feel free to ask any questions and I will be happy to engage you.</p><p>Until next article.</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bb247dc5adfd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DOCKER FUNDAMENTALS]]></title>
            <link>https://medium.com/@musinde/docker-fundamentals-8b856f24aeb1?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b856f24aeb1</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-compose]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[bash]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Tue, 25 Apr 2023 10:05:47 GMT</pubDate>
            <atom:updated>2024-04-16T11:07:37.864Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Docker?</h3><p>Docker is a platform that packages software into standardized units called containers that allows you to separate your application from underlying infrastructure by bundling up your code and all of its dependencies into a self-contained entity that will run the same on any supported system.</p><h3>Why Docker?</h3><p>We can barely mention Docker without mentioning Virtual Machines (VMs). VMs run applications in their own Operating Systems and share same hardware. They use a hypervisor which is responsible for managing which OS to run.</p><p>Applications may run on my local machine but break while running on a different environment due to a difference in the dependencies used. I might be running Python 2.7 on my PC from which I developed and tested my applications. Another environment might be running Python 3.6 which might not necessarily run my application the same way.</p><p>Docker virtualizes the application layer and sits on top of the OS and solves the above mentioned problem by building containerized applications that hold the right versions of all dependencies allowing it to run on any environment correctly.</p><p>Docker containers, unlike VMs, can be spin up in seconds, are lightweight hence less resource intensive and are easily configurable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*jpWLvI1yQ2iU2ToF87217Q.png" /><figcaption>Source: cloudacademy</figcaption></figure><p><strong>Docker Architecture</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/1*Bf7SRkFS4fYjQfsGAvxlJA.png" /><figcaption>Source: cloudacademy</figcaption></figure><p>Docker uses a client-server architecture . The server part is a Docker Daemon which manages docker objects (images, containers, volumes and networks). The Daemon exposes a REST API that the client consumes over UNIX sockets or a network interface. The client is the Docker binary allowing you to run Docker commands.</p><p><strong>Docker image </strong>is a template that defines how a container will run while a <strong>Docker container</strong> is an isolated environment that that is running the code from the Docker images. A container is synonymous to a running application while an executable file is the image. Docker images can be added to a registry for others to use, and example being <strong>Docker Hub</strong>.</p><p><strong>Docker volumes </strong>are persistent storage options generated by running Docker containers. Saving data in persistent volumes allows applications to have the same data when the container state changes. Volumes also help different containers to have access to same data.</p><p><strong>Docker networks </strong>are used to isolate containers and are also used to allow containers to communicate with each other.</p><p><strong>Installing Docker</strong></p><p>Docker installation varies from one operating system to another. Docker Desktop can be downloaded for <a href="https://docs.docker.com/docker-for-windows/install/">Windows</a> and <a href="https://docs.docker.com/desktop/install/mac-install/">Mac</a> users.</p><p>For Linux, you can install Docker from the command line as per this <a href="https://docs.docker.com/engine/install/ubuntu/">guide</a>. Further, we need to start the docker daemon by using the command</p><pre>sudo systemctl start docker</pre><p><strong>Dockerfile</strong></p><p>Now before we dive into Docker commands, I would like to introduce a rather important part of Docker called Dockerfile. This is a file that holds commands that you can use to build an image. It has a simple and easy-to-read text format that begins with specifying a base image from which our image will be built. It is worth noting that Docker has a layered system that starts off at the base image and each customization from the base image is added upon the base image. Each layer is a set of file changes that Docker can merge into the previous layer. The layered system approach prevents duplicate files as each layer just merges in the changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/0*dFvnJ6Cl3EIA2n3G.png" /></figure><p>Example of a Docker</p><pre><br><br># Node 16 Base image that will be retrieved from online repo<br>FROM node:16<br># Specifies the working directory. All the paths referenced after this point will be relative to this directory.<br>WORKDIR /app<br># Copy all files with the package name on them into the Docker container&#39;s working directory<br>COPY package*.json ./<br># Installs some dependencies required for the Node.js API<br>RUN npm install<br># Copy all local source files into the Docker container&#39;s working directory<br>COPY . .<br># Indicates that the API exposes port 80<br>EXPOSE 80<br># Command to start the API which will get executed when a Docker container using this image is started.<br>CMD [&quot;node&quot;, &quot;index.js&quot;]</pre><p>Frequently used instructions are described in the table below. The full list of instructions can be found in the <a href="https://docs.docker.com/engine/reference/builder/">Dockerfile reference</a> documentation.</p><p>The .dockerignore file works the same way as Git’s .gitignore file and is created in the same directory as the Dockerfile to prevent some files from being copied into the Docker image.</p><h3>Docker commands</h3><p>We shall use the Docker Project from Day 17 of <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704">#90DaysofDevops</a> challenge to better illustrate some of the commands used for building containerized apps using Docker</p><h3>TASKS:</h3><ul><li><strong>Create a Dockerfile for a simple web application (e.g. a Node.js or Python app)</strong></li></ul><p>Assuming we have installed Docker and Git already, we need to clone the below repo from which we will get our nodejs application.</p><pre>git clone https://github.com/Moses-Mathias/nodejs-on-ec2.git</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VdfVWFhKj01I2cT-YHJ2mw.png" /></figure><p>Start the Docker daemon and check its status by running the below commands</p><pre>sudo systemctl start docker<br>sudo systemctl status docker</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gJUL77L7AWnuqaN-f9mpdg.png" /></figure><p>We shall use the Dockerfile we created above. Create the file in the directory you cloned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLuOgoPwiVPlh-8f32iFDQ.png" /></figure><p>Once we have created the Dockerfile, we shall use it to build the image.</p><p>We can check for any available images by using the command:</p><pre>docker ps <br>docker ps -a</pre><p>Using the “a” flag lists both the running and stopped processes, while the ps command only shows the running processes. In our case, we have no containers available</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/1*mddnl14wHxXKIHO3wvABhA.png" /></figure><p>Now, we can go ahead and build the image.</p><pre>docker build -t app .</pre><p>The “t” flag, is a tag flag to give a name for the image. The “fullstop” at the end shows the location of the Dockerfile which is the current directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aK_W-zR0OHSbg0U2RBVDhg.png" /></figure><p>Now we create and run a container from the built image:</p><pre>docker run -d --name node_c -p 4000:80 app:latest</pre><p>The run command is used to create the container. We use the “name” flag to give the container a name. The “d” flag is a detach flag that basically lets the commandline “exit” the standard I/O and interact with the shell. This is very important while running containers with infinite loops. We also have the “p” (publish) flag that allows us to dynamically bind exposed port, say 80&lt; to a port on the host, say 4000.</p><p>We also have a tag “v1” on the image which gives it a specific identifiers. One image can have more than one tag and if not specified, then Docker assigns the tag “latest” to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Ovp7d_Itiw7X-DIxDCUOQ.png" /></figure><pre>docker tag app app;v1</pre><p>Now we have a new version of the same image. We can run a new container using the image and assign a new port for the host. I will use port 80 as I am running an ec2 instance. On a local PC, running <a href="http://localhost:4000/">http://localhost:4000</a> should work just fine.</p><pre>docker run -d --name node_c2 -p 80:80 app:v1</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A016d8ifH8vM_Z2q6oZrXQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*7K-XNEqSX_HOoG_9RsZrow.png" /></figure><h4><strong>Pulling/Pushing Images to DockerHub</strong></h4><p>Docker images are saved remotely in Docker registries (default being DockerHub). You can download and store images on DockerHub in a similar fashion to GitHub</p><p>Using the docker pull command, one can download an image from the registry:</p><pre>docker pull node:16</pre><p>This will download the latest version of Node16 onto our local machines</p><p>Using the <strong>docker images</strong> command will reveal the Image downloaded.</p><p>Suppose we have created our own images, say using a Dockerfile, and we would like to upload it to our pre-created DockerHub account, we can use the <strong>docker push</strong> command.</p><pre>docker push &lt;hub-user&gt;/&lt;repo-name&gt;:&lt;tag&gt;</pre><p>But first, we have to log in to our user account using:</p><pre>docker login</pre><p>The command will prompt you to enter your username and password and once authenticated you can now push your image to the registry.</p><pre>docker push musinde/app:latest</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/549/1*wmQPLUXTXhZUVszmKYo9ZA.png" /></figure><p><strong>Removing Images</strong></p><p>If a container is using an image, you will need to first delete the container before the image using the <strong>docker container rm ‘container_names”</strong> or <strong>docker container prune</strong> to remove all containers.</p><p>Afterwards, simply run the command <strong>docker rmi “image_name”</strong> to remove the image. Alternatively, you can use the -f flag to forcefully remove an image even when the containers are running, <strong>docker rmi -f “image_name”</strong>.</p><h4><strong>Docker Containers</strong></h4><p><strong>docker run “image:tag”</strong> — creates a containers from the specified image . It also executes the command specified by the CMD command in the Dockerfile. Below example will spin up a container named app_c1 from the image app:v1.</p><pre>docker run --name app_c1 app:v1</pre><p><strong>docker ps — verifies running containers</strong></p><p><strong>docker ps -a</strong> — lists all containers</p><p>docker stop “container_name” — terminates the processes of a container gracefully</p><p>docker kill “container_name” — forcefully terminates a container</p><p><strong>Detached Mode</strong></p><pre>docker run -d --name app_c1 app:v1 </pre><p>Docker run command will, by default, attach the process output of a container to the console in which the docker run command was executed. All the logs in the container, therefore, appear in the console in real-time. What if we would also like to interact with the console while the container runs? The detach flag (-d) allows us to do this by running in <em>detached mode.</em></p><h3>DOCKER VOLUMES</h3><p>These are used to persist data in Docker containers. The storage options are:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/502/0*-BrghdR7KqXHNab6.png" /></figure><ol><li>Bind Mounts</li></ol><p>Mounts a file or directory that resides on the host inside the container. This is an effective mechanism that allows you to access files from the host inside the container. Once the container is stopped, the data remains because it lives on the host. On the downside, bind mounts are not decoupled from the host as you might like. You have to know the exact path on the host that you want to mount in the container.</p><p>2. Volumes</p><p>Preferred over bind mounts and can be described in a similar way except that Docker manages the storage on the host. There is no need to know the path to the file making it easier when working cross platform as Docker handles the volume. Volumes are not limited to the local host file system either, they allow the use of different drivers (which supports external mechanisms such as Amazon S3).</p><p>3. TMPFS Mounts</p><p>Not persistent. Allows file system access for the life of the running container, making it useful to store sensitive data such as access tokens.</p><p>A volume can be mounted to a container by passing the -v ( — volume) option to the docker run command when starting a container.</p><pre>docker run -d --name app_c1 -v volume_c1:/usr/src/app app:v1 </pre><p>The above command will map a volume named volume_c1 to the code found in the /usr/src/app directory in the Docker container.</p><p>docker volume ls — list the available volumes</p><p>We do not have to specify the name of the volume while running the docker run command. We can find out the name of the name assigned by Docker by running the <strong>docker inspect app_c1.</strong></p><p>Removing all images, containers and volumes</p><pre>docker system prune  # cleans images, containers, volumes, and networks that are not associated with a container<br>docker system prune -a  # same as above, but includes stopped containers and unused images<br><br>docker volume prune  # removes volumes that are not connected to containers (aka &quot;dangling&quot; volumes)<br><br>docker rmi $(docker images -a -q)  # removes all images that are not associated with existing containers<br>docker image prune -a # same as the above command<br>docker rmi $(docker images -a -q) -f  # same as above, but forces the images associated with existing containers (running or stopped) to also be removed<br><br>docker rm $(docker ps -a -q)  # removes all containers<br>docker rm $(docker ps -a -q) -f  # same as above, but forces running containers to also be removed</pre><h3><strong>DOCKER COMPOSE</strong></h3><p>Docker Compose allows for the definition of containers to be done in docker-compose.yml files with the use of the docker-compose CLI commands to interact with the Docker engine. Compose uses a YAML file to define the services. A service describes what image should be used for a running container and any other configuration associated with a container, such as environment variables, ports to be exposed, volumes to be used, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/0*8d020wVMuOFUr4xJ.png" /></figure><p>For us to use Docker Compose, we need to install Docker-Compose in the server. We can use this <a href="https://kenfavors.com/code/how-to-install-docker-compose-on-linux-systems/">tutorial</a> for this.</p><p>Now, let us create a docker-compose.yml file:</p><pre>version: &quot;3.8&quot;<br>services:<br>  web-app:<br>    container_name: web-app_c1<br>    build: ./web-app<br>    ports:<br>      - &quot;3000:80&quot;<br>  api:<br>    container_name: api_c1<br>    build: ./api<br>    ports:<br>      - &quot;5000:5000&quot;<br>  db:<br>    container_name: db_c1<br>    image: postgres<br>    ports:<br>      - &quot;5432:5432&quot;<br>    environment:<br>      POSTGRES_PASSWORD: password<br>    volumes:<br>      - db-volume:/var/lib/postgresql/data<br>volumes:<br>  db-volume:</pre><p>The above YAML file starts with defining the version of Docker Compose to be used (3.8). We then define three services: web-app, api and db. Under each service we have a container_name and other specifications that will be necessary while creating the containers. The build specifies the location of the Dockerfile. The <strong>. represents the directory where the docker-compose.yml file is present (current directory). The image keyword is used to specify the image from docker hub for MYSQL containers. W</strong>e also specify the environment variables for <strong>MYSQL</strong> which are required to run <strong>MYSQL</strong>. Finally, the port parameter specifies the port mapping from which to expose the containers.</p><pre>docker-compose up api  # Will only start the api service<br>docker-compose up  # Will start all the services<br>docker-compose down #Stops containers and removes containers, images, and networks associated with a service</pre><p>There are more commands that have not been mentioned in this article. We have however handled a substantial amount to help us get started with Docker.</p><p>Feel free to leave a comment or a question below and I will be sure to respond.</p><p>Until next article,</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b856f24aeb1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GIT FUNDAMENTALS]]></title>
            <link>https://medium.com/@musinde/git-fundamentals-cd9a16463b00?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd9a16463b00</guid>
            <category><![CDATA[github]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Thu, 30 Mar 2023 20:08:02 GMT</pubDate>
            <atom:updated>2024-04-16T11:08:53.873Z</atom:updated>
            <content:encoded><![CDATA[<p>The exercises in this article are based on the <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704"><strong>#90DaysofDevops</strong></a><strong> challenge Day 8 to Day 12. The article, however, is not confined to the content of the challenges but digs deeper into the Git and GitHub basics.</strong></p><p>Happy Learning!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VBE-98PprUjPh7vY.png" /></figure><p>What is Git?</p><p>Git is a distributed version control system that helps in keeping track of code changes and collaborate on code. With Git, multiple people can work on a single version of a file by sharing changes and merging those changes. The edits are then transferred to files in a Git repository.</p><p>What is GitHub?</p><p>GitHub is a web-based platform that serves as a location for uploading copies of a Git repository.</p><h3><strong>Version Control</strong></h3><p>Refers to a system that is responsible for managing changes to software code. Version control tracks these changes and and logs them for a rollback, comparison, or to check who modified the files/project.</p><p>Version Control is divided into three types:</p><p>. Local version control system whereby the code is edited on the server where it resides and the edits are stored in a patch. Every patch set contains only the changes made to the file since its last version.</p><ul><li>Centralized Version Control System (CVCS) uses a client-server model. Developers download the latest version from the server, make changes then upload back to the server. This system is suitable for small teams that can ensure no two developers work on the same piece of code simultaneously. Examples include CVS, Perforce and SVN.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dbSiejpgwmYvMY1I.png" /></figure><ul><li>Distributed Version Control System (DVCS) allows every single developer to have their own server and have a copy of the entire history of the code and all its branches in their local machine. Examples include Git and Mercurial.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZUSURrLHajvPpNQa.png" /></figure><p>Advantages of DVCS over CVCS</p><ul><li>Except for pushing and pulling the code, the user can work offline in DVCS, due to this the full history is always available.</li><li>DVCS is faster than CVCS because you don’t need to communicate with the remote server for each and every command.</li><li>Merging and branching the changes in DVCS is very easy.</li><li>No need to access a remote server therefore the performance of DVCS is better.</li><li>If the main server goes down or it crashes in DVCS, you can still get the backup or entire history of the code from your local repository or server where the full revision of the code is already saved.</li><li>Because of local commits, the full history is always available.</li><li>Ability to push your changes continuously</li><li>Good for projects with off-shore developers</li></ul><p><strong>Difference Between Main Branch and Master Branch</strong></p><p>Both Main and Master refer to the primary line of development in a Git repository. Due to cultural differences, the term Master is avoided and Main is more recently adopted.</p><p><strong>Differences between Git and GitHub</strong></p><ul><li>Git is a distributed version control software, whereas GitHub is a service built around git.</li><li>Git is an easy-to-use tool that a developer can install on the local system. On the other hand, GitHub is a cloud-based service that allows for remote access from anywhere in the world.</li><li>Git is managed by Linux, whereas GitHub is maintained by Microsoft.</li><li>Git is open source tool for developers which is available for free. Github is an online service that offers some of its features to its users for free.</li><li>Git is primarily used to manage source code, whereas GitHub is used for connecting, uploading, or downloading resources.</li></ul><p><strong>GIT COMMANDS</strong></p><p>First things first, let us make sure that we have git installed by running “git -v” in the terminal. If not, refer to this <a href="https://git-scm.com/book/en/v2/Getting-Started-Installing-Git">documentation</a> for instructions on how to install it.</p><p>To start, we shall create a new directory from which we shall host the files of our project and jump into the directory.</p><pre>mkdir -p DevOps/Git<br>cd DevOps/Git</pre><p>Now let us initialize the git repo by running:</p><pre>git init</pre><p>This command creates a <strong><em>.git</em></strong> hidden folder from which version control runs.</p><p><strong>git status </strong>command is used to check the status of the repository, including any modifications made.</p><p>Let us now create a new file “hello.txt” in the repo and run the git status command again.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/961/1*lahgAkUGWTXSAydy5hzB-A.png" /></figure><p>Now before we proceed let us clear the air. What does “untracked” mean? Well, an untracked file is a file that has been created in the repo’s working directory but not yet added to the repo’s tracking index via the <strong>git add</strong> command. Let us run the git add hello.txt and check the status.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*FlVYGZ3wCptp8HcKTQHOXA.png" /></figure><p>The <strong>git add</strong> command puts the file in a <strong>staging area (</strong>a middle ground between the changes made to the working directory and the last commit<strong>) </strong>which means it is now available for a commit command.</p><p>In case there is a file we do not want to include in the next commit we can “demote” the file back to unstaged and untracked state usnig the command:</p><pre>git rm --cached hello.txt</pre><p>Next, we commit the file to make it tracked (under version control).</p><pre>git commit -m &quot;My first Git commit&quot;</pre><p>we use the “m” flag to write a description for the commit. Git status presents:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/468/1*oVQCSsJeOPsBR9KoSp3crA.png" /></figure><p>Our hello.txt file is now under version control in Git. We can make changes to it by adding content in the file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*SemOv6UZHcTJVNkguPCqtg.png" /></figure><p>We have modified and because the file is tracked we can go further to know how much modification has been made from the last commit:</p><pre>git diff</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/467/1*OL3r1G-8JAWtOsEK-lM6zA.png" /></figure><p>Here we see that some text was added as identified by the + sign, and none was deleted ( — sign).</p><p>Let us add the file and commit.</p><p>Note:</p><p>For multiple files, we can simple add all of them to the staging area by:</p><pre>git add . </pre><pre>git log</pre><p>Git log command outputs a list of commits including the author and time of the commit, and any branches that are currently pointed to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/934/1*Cf0LdlNsa9KgVE7YIFc91g.png" /></figure><p>We have some important flags to note for the <strong>git log</strong> command:</p><pre>git log --oneline</pre><p>Displays output in a summarized, easy-to-read format</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*dYqaYQUdc0LrNwfDtEkemQ.png" /></figure><pre>git log --graph</pre><p>Shows a graphical representation of the course of your branches.</p><p>Now, what if we want to undo the second commit we made?</p><pre>git reset</pre><p>This command takes other powerful arguments:</p><p>git reset — soft : The soft flag undoes a commit by returning the file to the staging area waiting for commit</p><p>git reset — hard: This is a dangerous command as it discards all the changes made between commit. It is permanent.</p><p>You can further specify the commit to be undone using the HEAD command:</p><pre>git reset --soft HEAD~1</pre><p>This command will reset the branch to 1 commit before the current one and we can commit the same file with a different message.</p><p>Learn more about git reset form this <a href="https://www.atlassian.com/git/tutorials/undoing-changes/git-reset#:~:text=To%20review%2C%20git%20reset%20is,correspond%20to%20the%20three%20trees.">post</a>.</p><p><strong>git stash </strong>command is useful in a case where we want to “hide” a modified file and commit other files. The file is removed from the staging area until the command <strong>git stash pop </strong>is used<strong>. </strong>If you want to make sweeping changes to a codebase while preserving the state of some of your changes then these two commands come in handy.</p><p><strong>Adding GitHub</strong></p><p>To create a new repository on GitHub, follow these steps:</p><ol><li>Go to the GitHub website and log in to your account.</li><li>Click the “+” icon in the top right corner of the screen and select “New repository”.</li><li>Enter a name for your repository in the “Repository name” field.</li><li>Optionally, you can add a description of your repository in the “Description” field.</li><li>Select whether you want your repository to be public or private. Public repositories are visible to anyone and can be cloned or forked by anyone, while private repositories are only accessible to you and other people you specifically invite.</li><li>Click the “Create repository” button.</li></ol><p>We have created our GitHub repository, which is our remote repo. Unlike a local repo that is stored on our local machines, a remote repo is a version of your project that is stored on a server and is accessible from anywhere with an internet connection. Remote repos are essential for collaboration.</p><p>For us to connect our local repo to the remote repo to enhance collaboration:</p><ul><li>Initialize the Git repo in our local repo by running git init</li><li>Create a GitHub repo and copy the URL.</li><li>In your local repo, run the below command to add the remote repo</li></ul><pre>git remote add origin &lt;remote repo URL&gt;</pre><ul><li>Using the “git remote” command, we can list all the remotes, which are repos sharing the same codebase.</li><li>To push our local repo to the remote repo, run the following command:</li></ul><pre>git push -u origin master</pre><p>origin being the remote server name and master is the branch.</p><p>The -u flag creates a reference for every branch that you successfully push onto the remote repo. In the above command, our local branch is automatically lined to the master branch and therefore we can pull changes from the master branch using <strong>git pull</strong> command with no further arguments.</p><p><strong>Cloning a remote repo</strong></p><p>git clone command is primarily used to point to an existing repo and make a copy of that repo in a new directory, at another location.</p><pre>git clone &lt;remote repo URL&gt; &lt;local directory&gt;</pre><p>In this case, git will copy all the contents in the repo specified by the URL to a directory chosen on the local device.</p><p>Once the local and remote repos are connected, we use the git push and git pull commands to update a remote repo and a local repo respectively.</p><p><strong>Git Branching</strong></p><p><strong>As our team gets bigger during collaboration, we need a way for all developer groups to work independently without affecting the main codebase. Git introduces a version control system called branching that caters for this. A git branch enables groups to work on different features or fixes simultaneously.</strong></p><p>Once the changes on the branches are completed and reviewed, we can merge them to the main branch.</p><p>We create a new branch by using the command:</p><pre>git checkout -b &lt;branch name&gt;</pre><p>This command doubles to switch us to the new branch.</p><p><strong>Git Merge</strong></p><p>Once the branch is created, and changes in the branch are ready to be integrated into another branch (say main), we use the git merge command. Git will create a new commit that includes all the changes made in the source branch to the target branch IF there are no conflicts.</p><p>In case of conflicts, Git will prompt you to resolve the conflicts before trying to merge again</p><p><strong>Git Rebase</strong></p><p>This command, does a similar job to git merge, only in a linear manner. In case of rebase, the entire history of the repo is modified as the source branch is moved to a new location at the head of another branch.</p><p><strong>Cherry-pick:</strong></p><p>Git cherry-pick is a command that allows you to select specific commits from one branch and apply them to another. This can be useful when you want to selectively apply changes that were made in one branch to another.</p><h3>Day 10 and 11 of the <strong>#90DaysofDevops.</strong></h3><p>To wrap this up, let us get our hands dirty with tasks form Day 10 and 11 of the <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704"><strong>#90DaysofDevops</strong></a><strong>.</strong></p><h3>TASK 1 :</h3><p>Add a text file called version01.txt inside the Devops/Git/ with “This is first feature of our application” written inside. This should be in a branch coming from master, [hint try git checkout -b dev], swithch to dev branch ( Make sure your commit message will reflect as &quot;Added new feature&quot;). [Hint use your knowledge of creating branches and Git commit command]</p><ul><li>version01.txt should reflect at local repo first followed by Remote repo for review. [Hint use your knowledge of Git push and git pull commands here]</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JEsBC4Fl4_wP4kzudb0E5g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/569/1*lyZa9Dg8ESY5G64yEZcLNw.png" /></figure><p>Add new commit in dev branch after adding below mentioned content in Devops/Git/version01.txt: While writing the file make sure you write these lines</p><ul><li>1st line&gt;&gt; This is the bug fix in development branch</li><li>Commit this with message “ Added feature2 in development branch”</li><li>2nd line&gt;&gt; This is gadbad code</li><li>Commit this with message “ Added feature3 in development branch</li><li>3rd line&gt;&gt; This feature will gadbad everything from now.</li><li>Commit with message “ Added feature4 in development branch</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KFmsNU5jZEGLAPHmArIZCg.png" /></figure><p>Here, we used the git checkout -b dev to create a new branch called dev and switch to it.</p><p>Note: When we try pushing the branch dev to the remote repo an error occurs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/819/1*Gdbu5pQ4L2AthPcFDNbvfw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/907/1*nDTkSz44dBhybSxFuy_PTQ.png" /></figure><p>Now, if we check our GitHub repo, we have the dev branch created and the changes to version01.txt applied there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/309/1*bF8dYnEcOQeujPwAlWXfOw.png" /></figure><ul><li>Restore the file to a previous version where the content should be “This is the bug fix in development branch”</li></ul><p>Here, we need to restore to the first commit, which is three commits behind the current commit.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*TfIMIknGk0ZFNUfgP39xQg.png" /></figure><p>Using git reset with the soft flag and specifying the HEAD to be 3 does just that. We can see from the status command that the version.txt is in the unstaged state meaning we can still add it to return it to the “feature4” commit.</p><p>If we used the <strong>hard </strong>flag, we would permanently delete the commits and this cannot be undone.</p><p>Now, let us switch back to our main branch and merge the dev branch to it.</p><pre>git merge dev<br>git log --oneline</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*hCr3shNDA2x2xytRuQPsjw.png" /></figure><p>Finally, push to GitHub</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*DF-NTenCWY-kcONAB6lrrg.png" /></figure><p>We have not yet covered all the commands but these are sufficient to kick-start our curiosity in Git.</p><p>Kindly feel free to drop any questions regarding what we have covered in this article and I will surely answer you.</p><p>Until next article</p><p>Peace ✌️</p><p>Moses Musinde</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd9a16463b00" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[BASIC LINUX COMMANDS]]></title>
            <link>https://medium.com/@musinde/basic-linux-commands-3ea96b0a4916?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ea96b0a4916</guid>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[shell-script]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Mon, 27 Mar 2023 18:59:54 GMT</pubDate>
            <atom:updated>2023-03-27T19:09:56.991Z</atom:updated>
            <content:encoded><![CDATA[<p>This article has been inspired by <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704"><strong>#90DaysofDevops</strong></a><strong> challenge by </strong><a href="https://www.linkedin.com/in/ACoAABhZ4kMBt55axHJpEnVRp0UOUl-_JwwmPwk"><strong>Shubham Londhe</strong></a><strong> and the </strong><a href="https://github.com/Moses-Mathias/alx-system_engineering-devops"><strong>ALX SE System Engineering and DevOps </strong></a><strong>project.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/970/0*ueJyzEG2fV8lwsR_.png" /></figure><h3>Navigation</h3><p>The ‘<em>pwd’</em> command (print working directory) is used to print the absolute path name of the current working directory.</p><p>To list the contents of the current directory:</p><p>#<em>ls </em>command lists files and directories in a directory</p><ul><li>The ls -l command displays the files and directories in a directory in a long format while using the -a command includes the hidden items (start with a .)in the output. A combination of -la prints out the contents of the directory in long format and includes hidden files.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/1*9FFp7oRh4ZA7-b2QSMhHxw.png" /></figure><p>#<em>mkdir </em>command is used to create a directory</p><ul><li>To create nested directories, append -p to the mkdir command.</li></ul><pre>mkdir -p A/B/C/D/E</pre><p>#<em>cp</em> command copies the source file specified by the SourceFile parameter to a the destination file specified by the TargetFile parameter. To copy Devops.txt to the C directory created above:</p><pre>cp Devops.txt ./C</pre><p>#<em>mv </em>command moves files and directories from one directory to another or renames a file or directory.</p><ul><li>To move a file from current directory to the D directory:</li></ul><pre>mv Devops.txt ./D</pre><ul><li>To rename the file:</li></ul><pre>mv Devops.txt Bash.txt</pre><p>#<em>rm</em> command is used to delete a file or directory. Append the -r parameter to delete nested directories and/or files in a directory.</p><pre>rm Devops.txt</pre><p>#<em>cat </em>command is used to view what is written is a file.</p><ul><li>Use the <strong><em>| head -’number’</em></strong> command to trim the output to the number of lines specified from the top. The <strong><em>| tail -’number’</em></strong> will trim from the bottom e.g. cat devops.txt | head -3</li></ul><pre>cat Devops.txt<br>cat Devops.txt | head -3 //output first three lines<br>head -3 Devops.txt | tail +3 //output the third line</pre><p>#<em>touch</em> command is used to create files</p><p>Different text editors can be used to make changes to a file. Vi and <a href="https://www.gnu.org/software/emacs/tour/">EMACS</a> are some options though the more commonly used is Vi which comes by default in UNIX-based OS.</p><ul><li>#vi “name of file” helps you enter into the vi editor. For more on the vim editor check out this <a href="https://www.cs.colostate.edu/helpdocs/vi.html">cheat sheet</a>.</li></ul><p>#<em>history</em> command is used to check all the commands run so far.</p><ul><li>The -c option causes the history list to be cleared by deleting all of the entries.</li><li>If you want to run any of the specific Line command from Linux command history then you need to use Line number with ! mark e.g. history !10</li></ul><h3><strong>Permissions</strong></h3><p>#<strong><em>su </em></strong>command is used to switch between users.</p><p>#<strong><em>whoami </em></strong>command prints the current user.</p><p>#<strong><em>group </em></strong>command lists all the groups the current user is a part of.</p><p>#<strong><em>c</em></strong><a href="https://www.ibm.com/docs/en/aix/7.2?topic=c-chown-command"><strong><em>hown</em></strong></a> command allows you to change the user and/or group ownership of a given file, directory, or symbolic link.</p><p>#<a href="https://www.javatpoint.com/linux-chmod-command#:~:text=Linux%20chmod%20command%20is%20used,come%20across%20recursive%20directory%20traversal."><strong><em>chmod</em></strong></a> command is used to change the access permissions of files and directories e.g. chmod u+x devops.txt to grant the owner of the file execute permissions. You can use the — reference option to mirror permissions from other files e.g. chmod — reference=devops.txt hello.txt</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/474/1*9JhruM1DyS75q9NehmDnYA.png" /></figure><p>The first character on each line represent the type of entry. A dash represents a file while the letter d represents a directory.</p><p>The characters that follow represent the settings of three sets of permissions:</p><ul><li>The first three characters show the owner permissions.</li><li>The next three characters show the group permissions.</li><li>The last three characters show the other permissions (not included in the first two).</li></ul><p>The letters represent:</p><ul><li><em>r</em>: Read permissions. The file can be opened, and its content viewed.</li><li><em>w</em>: Write permissions. The file can be edited, modified, and deleted.</li><li><em>x</em>: Execute permissions. If the file is a script or a program, it can be run (executed).</li></ul><p>For example:</p><p>--- means no permissions have been granted at all.</p><p>rwx means full permissions have been granted. The read, write, and execute indicators are all present.</p><p>To use chmod to set permissions, we need to tell it:</p><ul><li><em>Who:</em> Who we are setting permissions for.</li><li><em>What</em>: What change are we making? Are we adding or removing the permission?</li><li><em>Which</em>: Which of the permissions are we setting?</li></ul><p>We use indicators to represent these values, and form short “permissions statements” such as u+x, where “u” means ” user” (who), “+” means add (what), and “x” means the execute permission (which).</p><p>The “who” values we can use are:</p><ul><li><em>u</em>: User, meaning the owner of the file.</li><li><em>g</em>: Group, meaning members of the group the file belongs to.</li><li><em>o</em>: Others, meaning people not governed by the u and g permissions.</li><li><em>a</em>: All, meaning all of the above.</li></ul><p>The “what” values we can use are:</p><ul><li><em>–</em>: Minus sign. Removes the permission.</li><li><em>+</em>: Plus sign. Grants the permission. The permission is added to the existing permissions. If you want to have this permission and only this permission set, use the = option, described below.</li><li><em>=</em>: Equals sign. Set a permission and remove others.</li></ul><p>The “which ” values we can use are:</p><ul><li><em>r</em>: The read permission.</li><li><em>w</em>: The write permission.</li><li><em>x</em>: The execute permission.</li></ul><p><strong>Numerical Shorthand</strong></p><p>Another way to use chmod is to provide the permissions you wish to give to the owner, group, and others as a three-digit number. The leftmost digit represents the permissions for the owner. The middle digit represents the permissions for the group members. The rightmost digit represents the permissions for the others.</p><p>The digits you can use and what they represent are listed here:</p><ul><li>0: (000) No permission.</li><li>1: (001) Execute permission.</li><li>2: (010) Write permission.</li><li>3: (011) Write and execute permissions.</li><li>4: (100) Read permission.</li><li>5: (101) Read and execute permissions.</li><li>6: (110) Read and write permissions.</li><li>7: (111) Read, write, and execute permissions.</li></ul><p>Each of the three permissions is represented by one of the bits in the binary equivalent of the decimal number. So 5, which is 101 in binary, means read and execute. 2, which is 010 in binary, would mean the write permission.</p><pre>chmod 764 devops.txt</pre><p>In the above snippet, we set the permissions for user 7 (111), group 6 (110) and others 4 (100) to the devops.txt file.</p><p>This will produce the same result as the below snippet.</p><pre>chmod u=rwx,g=rw,o=r devops.txt</pre><h3><strong>Shell Scripting</strong></h3><p>A shell script is a file containing a sequence of commands that are executed by the bash program line by line. In DevOps, the commands are normally used while automating repetitive tasks such as provisioning and configuring new infrastructure.</p><p><strong>The Shebang</strong></p><p>The “#!/bin/bash” is called a shebang and is a comment line (usually first line) that specifies which intepreter to execute a shell script. The interpreter in our case will be /bin/bash. You can also have “#!/bin/sh” in which the <strong>sh</strong> shell will execute the script.</p><p>Let us have examples of shell scripts in action:</p><ul><li>Write a Shell Script which prints I will complete #90DaysOofDevOps challenge</li></ul><pre>#!/bin/bash<br>echo &quot;I will complete #90DaysOofDevOps challenge&quot;</pre><ul><li>Write a Shell Script to take user input, input from arguments and print the variables.</li></ul><pre>#!/bin/bash<br>read -p &quot;What is your name? &quot; name<br>echo &quot;Hello, $name&quot;</pre><ul><li>Write a Bash script that displays Hello 10 times, but for the 9th iteration, displays Hello <em>and then</em> Hi on a new line.</li></ul><pre>#!/usr/bin/env bash<br>#Creating a script that uses an if statement<br>x=1<br>while [ $x -le 10 ]<br>do<br>    echo &quot;Hello&quot;<br>    if [ $x -eq 9 ]<br>    then<br> echo &quot;Hi&quot;<br>    fi<br>    ((x++))<br>done</pre><ol><li>On day 5 of <a href="https://www.linkedin.com/feed/hashtag/?keywords=90daysofdevops&amp;highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7015419431497416704"><strong>#90DaysofDevops</strong></a><strong> the following tasks are assigned:</strong></li></ol><p>Write a bash script createDirectories.sh that when the script is executed with three given arguments (one is directory name and second is start number of directories and third is the end number of directories ) it creates specified number of directories with a dynamic directory name.</p><p>Example 1: When the script is executed as./createDirectories.sh day 1 90then it creates 90 directories as day1 day2 day3 .... day90</p><p>$1 — First argument</p><p>$2 — Second argument</p><p>$3 — Third argument</p><pre>#!/bin/bash<br>path=~<br>for (( i=$2; i&lt;=$3; i++))<br>do<br> mkdir $path/DevOps/$1$i<br>done</pre><p>2. Create a Script to backup all your work done till now.</p><pre>#!/bin/bash<br>#Folder to backup<br>Backup_folder = &quot;~/Devops/*&quot;<br>#Where to back up<br>Backups = &quot;~/BackupFolder&quot;<br>#Archive filename created<br>day = $(date + &quot;%d-%b-%y&quot;)<br><br>mkdir $Backups/$day<br>cp -r $Backup_folder $Backups/$day<br><br>echo &quot;Backup created in $Backups/$day&quot;</pre><p>3. Read About Cron and Crontab, to automate the backup Script</p><p>Cron is a service utility for Unix-based systems that allows the user to run commands, programs and tasks at predetermined times. You can therefore use cron to schedule tasks such as regular backups.</p><p>Crontab (cron table) is a file that specifies the commands to run the cron utility. The file does not exist by default but can be created in the /var/spool/cron directory.</p><ul><li><strong><em>crontab -e</em></strong> — this command is used to create and edit the cron file.</li><li>crontab -l — this command is used to check the crontab list</li><li>crontab-r — this command is used to remove the current crontab file</li><li>crontab -d — this ommand deletes a specific user crontab</li></ul><p>A crontab file is composed of six fields, separated by spaces (use commas to indicate same field), that specify the schedule for a task to run. The fields, in order, are:</p><ol><li>Minute (0–59)</li><li>Hour (0–23)</li><li>Day of the month (1–31)</li><li>Month (1–12) OR jan, feb, mar, apr …</li><li>Day of the week (0–7, where both 0 and 7 represent Sunday) OR sun, om, tue, wed, thu, fri, sat</li><li>Command to be executed</li></ol><p>Example</p><p>We use the crontab -e command to create a crontab file. We intend to back up our folder DevOps every Tuesday at 0300 hours.</p><p>00 03 * * 2 ~/DevOps/backup.sh.</p><p>The “*” in position 3 and 4 represent a wildcard. It basically means “every day of the month” and “every month” respectively. This cron file will run the backup.sh every Tuesday at 3 a.m.</p><p>4. Read about User Management</p><p>User management refers to the process of creating, modifying, and deleting user accounts on a computer system or network. It also includes managing user permissions and access to resources.</p><p>On Linux systems, user management can be done using the command line tools such as useradd, usermod, userdel, passwd, and groupadd, groupmod, groupdel.</p><p>The post has covered some basics of Linux as guided by the #90DaysofDevops and ALX-SE. Feel free to leave your questions below and I will be more than happy to engage with you.</p><p>Until next time,</p><p>Peace ✌️</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ea96b0a4916" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Provisioning a Highly-Available VPC Architecture using CloudFormation]]></title>
            <link>https://medium.com/@musinde/provisioning-a-highly-available-vpc-architecture-using-cloudformation-2e30b2e3cd49?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e30b2e3cd49</guid>
            <category><![CDATA[aws-cloudformation]]></category>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Sun, 15 Jan 2023 08:29:35 GMT</pubDate>
            <atom:updated>2023-01-15T08:33:51.490Z</atom:updated>
            <content:encoded><![CDATA[<p>In this guide , we will learn How to Provision a Highly-Available VPC Architecture using CloudFormation. No prior experience is needed prior to this lab.</p><p><strong>Network Diagram</strong></p><p>First, we develop a diagram to visualize the design of our deployment and act as a visual aid to understand the CloudFormation script.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0sHDlgWUp76fVamG6EHTaA.png" /></figure><h3>What is AWS CloudFormation?</h3><p>AWS <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html">CloudFormation</a> is an AWS Solution for deploying infrastructure as code. Resources are grouped as stacks for ease of management giving us ample time to focus more on applications.</p><p>We just create a template with the details of AWS resources such as EC2 , RDS , SNS etc. that need to be launched to setup applications.</p><p>Once the template is created , we can import it to CloudFormation and AWS CloudFormation will take care of provisioning, configuring and mapping the resources.</p><p>AWS CloudFormation helps us to:</p><ul><li>Quickly replicate the exiting Infrastructure.</li><li>Simplify infrastructure management.</li><li>Easily control and track changes to the infrastructure.</li></ul><h3>Core Concepts of CloudFormation</h3><p>In CloudFormation , we mostly work with these components.</p><ul><li><strong>Templates</strong></li></ul><p>Templates in CloudFormation are written in JSON or YAML format. In this template , you can describe the resources with their properties that need to be created. <em>For example : EC2 Instance with t2.medium</em> . Then, using this template the CloudFormation will create those resources.</p><ul><li><strong>Stacks</strong></li></ul><p>A stack is a collection of AWS resources which we can manage as a single unit. All the resources in a stack are defined by CloudFormation template.</p><ul><li><strong>Change Sets</strong></li></ul><p>If any changes required in the existing infrastructure , we can update the stack. Before making changes we can create a change set (a summary of proposed changes) and the change set will let us know how the changes will impact the existing resources.</p><h3>What is VPC?</h3><p>Amazon Virtual Private Cloud is the AWS service where you can create resources such as databases, servers etc. in the virtual network.</p><p>For each Account and Each Region, a VPC is created by default.</p><p>Each VPC is logically isolated from other VPCs in the Cloud.</p><h3>Understanding Subnets</h3><p>We are going to Create Four subnets :</p><ul><li>Two subnets across Availability Zones as Public subnets where we create our Web Servers.</li><li>Another two subnets across Availability Zones as Private subnets where we will create the Databases.</li></ul><h3>Points to Remember</h3><ul><li>A Public subnet is a subnet associated with a Route Table having Internet Gateway attached to it.</li><li>A Private subnet is a subnet associated with a Route Table having NAT Gateway attached to it.</li></ul><h3>Provisioning VPC using CloudFormation template</h3><p>As per the diagram above, we will create the following Resources.</p><ul><li><strong>VPC</strong> with CIDR : 10.0.0.0/16</li><li><strong>2 Public Subnets</strong> in Each AZ.</li></ul><p>PublicSubnet1 : 10.0.10.0/24</p><p>PublicSubnet2 : 10.0.11.0/24</p><ul><li><strong>2 Private Subnets</strong> in Each AZ.</li></ul><p>PrivateSubnet1 : 10.0.20.0/24</p><p>PrivateSubnet2 : 10.0.21.0/24</p><ul><li><strong>1 Route Table</strong> for Public Subnets</li><li><strong>2 Route Tables</strong> , One for each Private Subnet</li><li><strong>Internet Gateway</strong> — Will be attached with Public Route Table</li><li><strong>2 NAT Gateway</strong> in each AZ and will be attached with each private Route Tables.</li></ul><h3>CloudFormation Template</h3><p>Let us create a file named <strong>VPC.yaml</strong> and add the below attached template to the file.</p><pre>Description: Template creates VPC with Public &amp; Private Subnets across AvailabilityZone. Also creates Internet Gateway, with a default route on the public subnets and deploys a pair of NAT gateways (one in each AZ),and default routes for them in the private subnets.<br><br>Parameters:<br>   EnvironmentName:<br>     Description: Each Resource name will be prefixed with Environment Name<br>     Type: String<br><br>   VpcCIDR:<br>     Description: Eenter CIDR notation VPC<br>     Type: String<br>     Default: 10.0.0.0/16<br><br>   PublicSubnet1CIDR:<br>     Description: Enter CIDR notation for Public subnet in the First Availability Zone<br>     Type: String<br>     Default: 10.0.10.0/24<br>   <br>   PublicSubnet2CIDR:<br>     Description: Enter CIDR notation for Public subnet in the Second Availability Zone<br>     Type: String<br>     Default: 10.0.11.0/24<br><br>   PrivateSubnet1CIDR:<br>     Description: Enter CIDR notation for Private subnet in the First Availability Zone<br>     Type: String<br>     Default: 10.0.20.0/24<br><br>   PrivateSubnet2CIDR:<br>     Description: Enter CIDR notation for Private subnet in the Second Availability Zone<br>     Type: String<br>     Default: 10.0.21.0/24<br><br>Resources:<br>   VPC:<br>     Type: AWS::EC2::VPC<br>     Properties:<br>        CidrBlock: !Ref VpcCIDR<br>        EnableDnsSupport: true<br>        EnableDnsHostnames: true<br>        Tags:<br>          - Key: Name<br>            Value: !Ref EnvironmentName<br><br>   InternetGateway:<br>     Type: AWS::EC2::InternetGateway<br>     Properties:<br>        Tags:<br>          - Key: Name<br>            Value: !Ref EnvironmentName<br><br>   InternetGatewayAttachment:<br>     Type: AWS::EC2::VPCGatewayAttachment<br>     Properties:<br>        InternetGatewayId: !Ref InternetGateway<br>        VpcId: !Ref VPC<br><br>   PublicSubnet1:<br>     Type: AWS::EC2::Subnet<br>     Properties:<br>        VpcId: !Ref VPC<br>        AvailabilityZone: !Select [ 0, !GetAZs &#39;&#39; ]<br>        CidrBlock: !Ref PublicSubnet1CIDR<br>        MapPublicIpOnLaunch: true<br>        Tags:<br>          - Key: Name<br>            Value: !Sub ${EnvironmentName} Public Subnet (AZ1)<br><br>   PublicSubnet2:<br>     Type: AWS::EC2::Subnet<br>     Properties:<br>        VpcId: !Ref VPC<br>        AvailabilityZone: !Select [ 1, !GetAZs &#39;&#39; ]<br>        CidrBlock: !Ref PublicSubnet2CIDR<br>        MapPublicIpOnLaunch: true<br>        Tags:<br>          - Key: Name<br>            Value: !Sub ${EnvironmentName} Public Subnet (AZ2)<br><br>   PrivateSubnet1:<br>     Type: AWS::EC2::Subnet<br>     Properties:<br>        VpcId: !Ref VPC<br>        AvailabilityZone: !Select [ 0, !GetAZs &#39;&#39; ]<br>        CidrBlock: !Ref PrivateSubnet1CIDR<br>        MapPublicIpOnLaunch: false<br>        Tags:<br>          - Key: Name<br>            Value: !Sub ${EnvironmentName} Private Subnet (AZ1)<br><br>   PrivateSubnet2:<br>     Type: AWS::EC2::Subnet<br>     Properties:<br>        VpcId: !Ref VPC<br>        AvailabilityZone: !Select [ 1, !GetAZs &#39;&#39; ]<br>        CidrBlock: !Ref PrivateSubnet2CIDR<br>        MapPublicIpOnLaunch: false<br>        Tags:<br>          - Key: Name<br>            Value: !Sub ${EnvironmentName} Private Subnet (AZ2)<br><br>   NatGateway1EIP:<br>     Type: AWS::EC2::EIP<br>     DependsOn: InternetGatewayAttachment<br>     Properties:<br>       Domain: vpc<br><br>   NatGateway2EIP:<br>     Type: AWS::EC2::EIP<br>     DependsOn: InternetGatewayAttachment<br>     Properties:<br>       Domain: vpc<br><br>   NatGateway1:<br>     Type: AWS::EC2::NatGateway<br>     Properties:<br>       AllocationId: !GetAtt NatGateway1EIP.AllocationId<br>       SubnetId: !Ref PublicSubnet1<br><br>   NatGateway2:<br>     Type: AWS::EC2::NatGateway<br>     Properties:<br>       AllocationId: !GetAtt NatGateway2EIP.AllocationId<br>       SubnetId: !Ref PublicSubnet2<br><br>   PublicRouteTable:<br>     Type: AWS::EC2::RouteTable<br>     Properties:<br>       VpcId: !Ref VPC<br>       Tags:<br>         - Key: Name<br>           Value: !Sub ${EnvironmentName} Public Routes<br><br>   DefaultPublicRoute:<br>     Type: AWS::EC2::Route<br>     DependsOn: InternetGatewayAttachment<br>     Properties:<br>       RouteTableId: !Ref PublicRouteTable<br>       DestinationCidrBlock: 0.0.0.0/0<br>       GatewayId: !Ref InternetGateway<br><br>   PublicSubnet1RouteTableAssociation:<br>     Type: AWS::EC2::SubnetRouteTableAssociation<br>     Properties:<br>       RouteTableId: !Ref PublicRouteTable<br>       SubnetId: !Ref PublicSubnet1<br><br>   PublicSubnet2RouteTableAssociation:<br>     Type: AWS::EC2::SubnetRouteTableAssociation<br>     Properties:<br>       RouteTableId: !Ref PublicRouteTable<br>       SubnetId: !Ref PublicSubnet2<br><br>   PrivateRouteTable1:<br>     Type: AWS::EC2::RouteTable<br>     Properties:<br>       VpcId: !Ref VPC<br>       Tags:<br>         - Key: Name<br>           Value: !Sub ${EnvironmentName} Private Routes (AZ1)<br><br>   DefaultPrivateRoute1:<br>     Type: AWS::EC2::Route<br>     Properties:<br>       RouteTableId: !Ref PrivateRouteTable1<br>       DestinationCidrBlock: 0.0.0.0/0<br>       NatGatewayId: !Ref NatGateway1<br><br>   PrivateSubnet1RouteTableAssociation:<br>     Type: AWS::EC2::SubnetRouteTableAssociation<br>     Properties:<br>       RouteTableId: !Ref PrivateRouteTable1<br>       SubnetId: !Ref PrivateSubnet1<br><br>   PrivateRouteTable2:<br>     Type: AWS::EC2::RouteTable<br>     Properties:<br>       VpcId: !Ref VPC<br>       Tags:<br>         - Key: Name<br>           Value: !Sub ${EnvironmentName} Private Routes (AZ2)<br><br>   DefaultPrivateRoute2:<br>     Type: AWS::EC2::Route<br>     Properties:<br>       RouteTableId: !Ref PrivateRouteTable2<br>       DestinationCidrBlock: 0.0.0.0/0<br>       NatGatewayId: !Ref NatGateway2<br><br>   PrivateSubnet2RouteTableAssociation:<br>     Type: AWS::EC2::SubnetRouteTableAssociation<br>     Properties:<br>       RouteTableId: !Ref PrivateRouteTable2<br>       SubnetId: !Ref PrivateSubnet2<br><br>   NoIngressSecurityGroup:<br>     Type: AWS::EC2::SecurityGroup<br>     Properties:<br>       GroupName: &quot;no-ingress-sg&quot;<br>       GroupDescription: &quot;Security group with no ingress rule&quot;<br>       VpcId: !Ref VPC<br><br>Outputs:<br>   VPC:<br>    Description: A reference to the created VPC<br>    Value: !Ref VPC<br><br>   PublicSubnets:<br>     Description: A list of the public subnets<br>     Value: !Join [ &quot;,&quot;, [ !Ref PublicSubnet1, !Ref PublicSubnet2 ]]<br><br>   PrivateSubnets:<br>     Description: A list of the private subnets<br>     Value: !Join [ &quot;,&quot;, [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]]<br><br>   PublicSubnet1:<br>     Description: A reference to the public subnet in the 1st Availability Zone<br>     Value: !Ref PublicSubnet1<br><br>   PublicSubnet2:<br>     Description: A reference to the public subnet in the 2nd Availability Zone<br>     Value: !Ref PublicSubnet2<br><br>   PrivateSubnet1:<br>     Description: A reference to the private subnet in the 1st Availability Zone<br>     Value: !Ref PrivateSubnet1<br><br>   PrivateSubnet2:<br>     Description: A reference to the private subnet in the 2nd Availability Zone<br>     Value: !Ref PrivateSubnet2<br><br>   NoIngressSecurityGroup:<br>     Description: Security group with no ingress rule<br>     Value: !Ref NoIngressSecurityGroup</pre><p>Let us try to understand the above template.</p><ul><li>First is the <strong>Parameters Section</strong></li></ul><p>We have used parameters to pass the values to the template when creating or updating a stack.</p><p>Parameters will help us to provide a custom value while creating or updating a stack.</p><p>First we are passing parameters and the default value for resources such as VPC , Public &amp; Private Subnets.</p><ul><li>Then comes <strong>Resources Section</strong></li></ul><p>Here we refer the Type of resources we’re going to create. The resources such as VPC , Internet Gateway , NAT gateway , Route Tables , Public and Private Subnets.</p><p><strong>!Ref</strong> — it returns the value of the specified resource from the parameters.</p><p><strong>!Select </strong>and <strong>!GetAZs</strong> — are the functions used by the AvailabilityZone to retrieve the list of availability zones for a Specified region and reference first element (0) from the list.</p><p><strong>EnableDnsHostnames</strong> — The instances launched in the public subnet will get Public DNS Hostnames if the <strong>EnableDnsHostnames</strong> is set to true.</p><ul><li>Finally the <strong>Output Section</strong></li></ul><p>To retrieve the attribute of the resources, <strong>!GetAtt</strong> function is used.</p><p>This function calls all the resource for their attributes and store them as an output with actual values.</p><p>Now Its time to provison the VPC using the template that we built.</p><p>Login to CloudFormation Console. Click <strong>Create stack</strong></p><p>On the <strong>Create stack</strong> page, Choose <strong>Template is ready</strong> and then</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/0*vbwFGhSu1RBVlseg.png" /></figure><p>For <strong>Specify template</strong> , Choose <strong>Upload a template file</strong> (VPC.yaml file) and then click <strong>Choose file</strong> and upload it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/0*lcBc0XP8SIkCHWie.png" /></figure><p>and click <strong>Next</strong>, Enter the Name for the Stack</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/433/0*7_1BykHK9tVPqKMH.png" /></figure><p>Under <strong>Parameters</strong>, provide the <strong>EnvironmentName</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/639/0*l2fbx0F0EUjvj4MZ.png" /></figure><p>And the Values for the <strong>VPC</strong> and <strong>Subnets</strong> will be automatically Populated from the template.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/0*tBu6DcFA61YNj2CX.png" /></figure><p>and click <strong>Next</strong>, on Configure stack options page, leave it to default.</p><p>and Click <strong>Next</strong> , review the configurations and click <strong>Create stack</strong></p><p>The Stack creation should be initiated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/636/0*RsO6otd_MDUL_8N4.png" /></figure><p>We can track all the events such as resources creation from the <strong>Events</strong> tab.</p><p>After the resources are created, we can check the lists of resources created by the stack from <strong>Resources</strong> tab.</p><p>The resources creation is completed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/967/0*MD0KA9mylIr5Rz4f.png" /></figure><p>From the <strong>Output</strong> tab , We can check all the resources and the values for the same.</p><p>Lets go to VPC Console and verify the Stack creation.</p><p>Choose <strong>Your VPCs</strong> , you can find the VPC created with the Environment Name.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/0*hZcsYSmH4gOkwLcf.png" /></figure><p>Choose <strong>Subnets</strong> and verify Whether 2 Public and Private Subnets are created across Availability Zones.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/0*Ab0NtjNSfVqB0k6c.png" /></figure><p>Choose <strong>Route Tables</strong> , you can find 1 Public Route table and 2 Private Route Tables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/0*IcwIgPM5Cl2vFbkp.png" /></figure><p>You can find 2 public subnets associated with a Public route Table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/0*b6ugGmF2yAO84GLx.png" /></figure><p>If you click Routes , you can find the Internet gateway attached with the Public Route Table.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/1*b31hiO4ynbDLRrXWEFF4aQ.png" /></figure><p>Same way , If we check the Private Route Tables , One subnet and One NAT Gateway should be attached with each Private Route Tables.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/701/0*aojB4Ksxwk8s-mht.png" /></figure><p>Choose <strong>Internet gateways</strong>, an Internet gateway should be created and attached with the VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/0*WfPB7c30gHnUDVC5.png" /></figure><p>Choose <strong>NAT Gateways</strong> , you can find two NAT Gateways with Elastic IP addresses attached with it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/420/0*ubPvUBJJ_nlEqN__.png" /></figure><p>As we have confirmed that all the VPC resources required to be created is successfully created by the CloudFormation template.</p><h3>Updating or Modifying Existing Infra</h3><p>Let’s say you want to change or add to the existing infrastructure, in this case we can create a <strong>Change set</strong> and Execute the changes.</p><p>First we need to modify the VPC.yaml file.</p><p>I am adding a third Public subnet to the VPC. For that, add the below template to VPC.yaml file.</p><p>Under <strong>Parameters</strong> Section:</p><pre>PublicSubnet3CIDR:<br>  Description: Enter CIDR notation for Public subnet in the Second Availability Zone<br>  Type: String<br>  Default: 10.0.12.0/24</pre><p>Under <strong>Resources</strong> Section:</p><pre>PublicSubnet3:<br>  Type: AWS::EC2::Subnet<br>  Properties:<br>    VpcId: !Ref VPC<br>    AvailabilityZone: !Select [ 0, !GetAZs &#39;&#39; ]<br>    CidrBlock: !Ref PublicSubnet3CIDR<br>    MapPublicIpOnLaunch: true<br>    Tags:<br>      - Key: Name<br>        Value: !Sub ${EnvironmentName} Public Subnet (AZ3)</pre><p>After adding the above template to the VPC.yaml file.</p><p>Go to CloudFormation Console , Click the Stack which we have already created.</p><p>Click <strong>Change sets</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/725/0*oRXqujttrlD2jkTj.png" /></figure><p>And click <strong>Create change set</strong> , click <strong>Replace current template</strong> , then choose upload a template file, here we need to upload the updated VPC.yaml file.</p><p>and then click <strong>Next</strong>, here you can find a New PublicSubnet Added.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/417/0*puqzBH5SvXsnj9d-.png" /></figure><p>Review the values and then click Next and finally choose <strong>Create change set</strong></p><p>Provide a name for the Change set and then click <strong>Create change set</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/0*SjYQxbvizvlTJW8m.png" /></figure><p>And finally click <strong>Execute</strong> for the changes to take effect.</p><p>you can track the process from the Events section.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/0*TSnQ_UJkEWZO4mUQ.png" /></figure><p>You will get the final response as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/627/0*nIPt2l91Hct92TvQ.png" /></figure><p>Let’s go to VPC console and verify whether the new PublicSubnet is created.</p><p>You can see that the new public subnet is added to the VPC successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/0*MzDeAdnrmC8fOTk7.png" /></figure><p>Lets clean up the stack we have created.</p><h3>Deleting Stack</h3><p>If you wish to vanish the entire stack that is created using the CloudFormation template, go to CloudFormation console.</p><p>Choose the Stack and then click <strong>Delete</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/520/0*Euqb1605bvtqI1c-.png" /></figure><p>and then Choose <strong>Delete stack</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/0*UBNHXp0WkayhgPUJ.png" /></figure><p>We have successully provisioned a highly-available VPC with it’s subnets, made changes to it and eventally deleted it.</p><p>Feel free to leave your queries in the comments and I will be sure to assist!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e30b2e3cd49" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[HOW TO CREATE AN ELASTIC LOAD BALANCER IN AWS]]></title>
            <link>https://medium.com/@musinde/how-to-create-an-elastic-load-balancer-in-aws-f504aeb552a3?source=rss-79430f618a44------2</link>
            <guid isPermaLink="false">https://medium.com/p/f504aeb552a3</guid>
            <dc:creator><![CDATA[Moses Musinde]]></dc:creator>
            <pubDate>Sat, 14 Jan 2023 07:41:48 GMT</pubDate>
            <atom:updated>2023-01-14T07:41:48.974Z</atom:updated>
            <content:encoded><![CDATA[<p>HOW TO CREATE AN ELASTIC LOAD BALANCER IN AWS</p><p>In this article we aim to take a step by step journey into creating an ELB. But first, what is a Load Balancer? Simply put, an ELB is a AWS cloud service that automatically distributes incoming traffic from applications or other cloud services across multiple cloud resources in different Availability Zones.<br>The importance of Load Balancers is that they improve the availability and scalability of applications running on EC2 instances.</p><p>Now that we are familiar with what an ELB, lets dig deep, shall we?</p><p>Create your Application running on two EC2 instances</p><p>First of all, create an EC2 instance with a web server running a Hello World application <a href="https://www.code4projects.net/definitive-amazon-ec2-tutorial-step-by-step-guide-beginners/">as shown in this tutorial</a>. If everything works properly you should enter the EC2 public IP address in the browser address bar and show a “Hello World” message with the private IP of your instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*p0vgGKA4Y3hRFihy.png" /></figure><p>AWS platform allows you to easily duplicate your EC2 instance by simply right-clicking the EC2 instance in the EC2 dashboard and selecting the menu <strong>Image and template</strong> -&gt; <strong>Launch more like this</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*Vz8zvJBe-19ZxKe0.png" /></figure><p>You should see two EC2 instances running in the EC2 dashboard. If you enter the IP of the second EC2 instance you should see a second “Hello World” message with a different IP. This IP later will help you to understand which EC2 instance will reply to your request.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*Z7YxyQJvi7DSrJqs.png" /></figure><h3>Create the Elastic Load Balancer</h3><p>Now you can create the Load Balancer that will live in front of your EC2 instances and will forward the requests to one or another to split the workload across them.</p><h3>Create the Application Load Balancer</h3><p>In the EC2 dashboard select the <strong>Load Balancers</strong> item from the main menu. Then select the <strong>Create Load Balancer</strong> button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*xIJ8BRWW-rjpqaM6.png" /></figure><p>You can choose to create one of the three load balancer types available: classic, application, or network. Select the <strong>Create</strong> button for <strong>Application Load Balancer</strong>. This load balancer works fine with HTTP/HTTPS applications like our “Hello World”. For more details about the features of these Load-Balancers <a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-application-load-balancer.html">read the AWS documentation</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*cwdbY8LH1EEsWgSV.png" /></figure><h3>Configure the Elastic Load Balancer</h3><p>Once you have decided on the Elastic Load Balancer to use you need to configure it. Insert the name of the load balancer (i.e. Demo-LB) and leave the default Scheme <strong>internet-facing</strong> in order to make it accessible from the Internet. Leave also the default IP address type <strong>ipv4</strong>. As you can see, this load balancer, by default, uses an HTTP listener. When it receives traffic on 80 port it automatically redirects the request to one of the EC2 instances registered in the Target Group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*fdOYY3vScO8X_EUn.png" /></figure><p>You can choose to add the fault tolerance to the load balancer itself. If you work with a single load balancer you’ll be affected by a single point of failure and this compromises the availability of your application. Amazon AWS allows you to replicate the load balancer across multiple AZs to improve its availability. Leave the default of 3 AZs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*bnxqGqRvbPuuEtCN.png" /></figure><p>In this <a href="https://www.code4projects.net/amazon-ec2-for-beginners/">following article</a>, we learn about Security Group as a way to protect traffic incoming or outgoing to an EC2 instance. In <a href="https://www.code4projects.net/definitive-amazon-ec2-tutorial-step-by-step-guide-beginners/">this other article</a>, we see how to use them in practice. Security Group should be configured on Load Balancer as well in order to allow the incoming or outgoing traffic. For example, in this tutorial, in order to allow HTTP traffic, it is fundamental to allow incoming traffic on port 80. It is not necessary to create a new Security Group, you can use the same used for EC2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*5GC2c4LyuHenz-0w.png" /></figure><h3>Create the ELB Target Group and register the two EC2 instances</h3><p>The application load balancer listens on port 80 and redirects the traffic to the EC2 belonging to its Target Group. You need to create now the Target Group and insert the two Ec2 instances in it. Insert a name for the Target Group and leave all the default settings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*yM2l9Xd1Uwmfkxp0.png" /></figure><p>Add the two EC2 instances to the Target Group.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*mQfsdmjjUNNzJNwl.png" /></figure><p>Review all the settings and click the <strong>Create</strong> button. You need to wait a while to let AWS provision the Load Balancer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*TAYeEfZsHSOg6T6h.png" /></figure><h3>How to test the Application Scalability</h3><p>Once the Application Load Balancer is created you can select it and in the <strong>DSN name</strong> field, you can find the hostname to insert in the browser address bar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*C6y72zHcM-zd26RC.png" /></figure><p>If you click repeatedly the browser <strong>Reload</strong> button you’ll find that the two EC2 instances will reply alternatively. You just tested that the workload is split equally on the two EC2 instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*Ho8JHEzMUxxqYK4z.png" /></figure><h3>How to test the Application Availability</h3><p>In order to test the availability of the application, you should verify what happens when an EC2 instance crash. You can simulate it by stopping manually an EC2 instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*jXG7dq0tY1UOYjpc.png" /></figure><p>If you go on the load balancer Target Group you’ll notice that the load balancer health check detects that one instance is down and it redirects all the traffic to the other EC2 instance. In fact, you’ll see always the same “Hello World” message coming from the same IP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/450/0*JdxF_2bzQY3LuYrs.png" /></figure><h3>Conclusion</h3><p>In this article, we talked about Elastic Load Balancers and how to use them in practice. In particular, this tutorial shows how to use an Application Load Balancer. However, things are not really different if you use a Network Load Balancer that is working at layer 4 of the IP stack and offers better performance. This is a good starting point to start using this tool on the Amazon platform. If you want more information refers to the <a href="https://aws.amazon.com/it/elasticloadbalancing/?whats-new-cards-elb.sort-by=item.additionalFields.postDateTime&amp;whats-new-cards-elb.sort-order=desc">official documentation</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f504aeb552a3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>