<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Gaurav kachariya on Medium]]></title>
        <description><![CDATA[Stories by Gaurav kachariya on Medium]]></description>
        <link>https://medium.com/@gauravkachariya?source=rss-23b7436770a9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*XC9-i2oErlItBvbSZjCFVg.jpeg</url>
            <title>Stories by Gaurav kachariya on Medium</title>
            <link>https://medium.com/@gauravkachariya?source=rss-23b7436770a9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 07:29:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@gauravkachariya/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[GitHub Self-Hosted Runner on EC2: A Step-by-Step Configuration Guide]]></title>
            <link>https://medium.com/@gauravkachariya/configuring-github-self-hosted-runner-on-ec2-a-step-by-step-guide-2449326c0f7b?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/2449326c0f7b</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[gi̇thub]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[cicd-tools]]></category>
            <category><![CDATA[ec2]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Fri, 01 Dec 2023 10:03:03 GMT</pubDate>
            <atom:updated>2024-04-02T14:04:12.832Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_qGPssASni-oJeIh6Asb1A.jpeg" /></figure><p>Continuous Integration (CI) and Continuous Deployment (CD) have become integral components of modern software development workflows. GitHub, being a leading platform for version control and collaboration, offers a powerful feature known as self-hosted runners to enhance CI/CD capabilities. In this blog post, we’ll delve into what self-hosted runners are, why they are crucial, and provide a step-by-step guide on adding an EC2 Ubuntu instance as a self-hosted runner.</p><h4>Understanding Self-Hosted Runners:</h4><p>Self-hosted runners in GitHub Actions are customizable execution environments that allow you to run your workflows on the infrastructure you control. Unlike GitHub-hosted runners, which GitHub provides, self-hosted runners offer more flexibility, allowing you to use your own hardware or cloud resources.</p><h4>Why Self-Hosted Runners?</h4><ul><li><strong>Increased Control:</strong> Self-hosted runners enable you to have fine-grained control over the environment in which your workflows run. This is particularly beneficial when your projects have specific dependencies or configurations.</li><li><strong>Cost Optimization:</strong> Utilizing your infrastructure, such as EC2 instances on AWS, can lead to cost savings compared to using GitHub-hosted runners, especially for larger projects with high CI/CD demands.</li><li><strong>Security and Compliance:</strong> In some cases, due to security and compliance requirements, organizations may prefer to run CI/CD workflows on their infrastructure. Self-hosted runners provide a solution that aligns with these concerns.</li></ul><h4>Adding an EC2 Ubuntu Instance as a Self-Hosted Runner:</h4><p>Now, let’s walk through the steps to add an EC2 Ubuntu instance as a self-hosted runner on GitHub:</p><p><strong>Step 1: Set Up an EC2 Instance on AWS</strong></p><ul><li>Log in to the AWS Management Console.</li><li>Navigate to the EC2 dashboard and launch a new Ubuntu instance.</li><li>Ensure that the instance has the necessary network configurations, security groups, and key pairs.</li></ul><p><strong>Step 2: Install GitHub Runner on the EC2 Instance</strong></p><ul><li>Connect to your EC2 instance using SSH.</li><li>Download the GitHub Actions self-hosted runner package from the GitHub repository.</li></ul><pre># Create a folder<br>mkdir actions-runner &amp;&amp; cd actions-runner<br><br># Download the latest runner package<br>curl -o actions-runner-linux-x64-2.311.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-linux-x64-2.311.0.tar.gz</pre><ul><li>Extract the downloaded package.</li></ul><pre>tar xzf ./actions-runner-linux-x64-2.311.0.tar.gz</pre><p>Run the configuration script and follow the prompts.</p><pre>./config.sh --url https://github.com/your-username/your-repository --token YOUR-PERSONAL-ACCESS-TOKEN</pre><p>Replace your-username and your-repository with your GitHub username and repository name.</p><p>To get a token, On GitHub, navigate to “Settings” &gt; “Actions” &gt; “Runners” and click on “Add self-hosted runner”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hvLsttd6qQhF9ICzPiqwwA.png" /></figure><p><strong>Step 3: Start the Self-Hosted Runner Service</strong></p><pre>./run.sh</pre><p><strong>Step 4: Verify and Monitor the Runner</strong></p><ul><li>Go to your GitHub repository, navigate to “Settings,” and click on “Actions.”</li><li>You should see your self-hosted runner listed under the “Runners” section.</li></ul><p><strong>Step 5: Use Self-Hosted Runners in Your Workflow:</strong></p><p>Once your self-hosted runner is set up, integrating it into your GitHub Actions workflow is a breeze. Simply add the following line to your workflow YAML file:</p><pre>runs-on: self-hosted</pre><p>For example:</p><pre>name: CI/CD with Self-Hosted Runner<br><br>on: [push, pull_request]<br><br>jobs:<br>  build:<br>    runs-on: self-hosted<br>    steps:<br>    - name: Checkout Repository<br>      uses: actions/checkout@v2<br>    # Add your build and test steps here</pre><p><strong>Conclusion:</strong></p><p>By adding a self-hosted runner to your GitHub repository, you empower your CI/CD workflows with increased flexibility, control, and potentially reduced costs. This guide provides a comprehensive walkthrough for setting up a self-hosted runner on an EC2 Ubuntu instance, paving the way for a more tailored and efficient development process.</p><blockquote>Thank you for taking the time to read this article. I hope you found it informative and useful. If you have any questions or comments, please feel free to leave them below. Also, if you enjoyed this article, I would appreciate it if you could give it a clap or share it with your network. Finally, if you would like to connect with me on <a href="https://www.linkedin.com/in/gaurav-kachariya/">LinkedIn</a>, I would be more than happy to connect with you and continue the conversation there. Thank you again for reading!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2449326c0f7b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automated EC2 Healing: Rebooting and Notifying with AWS Lambda and Slack]]></title>
            <link>https://medium.com/@gauravkachariya/automated-ec2-healing-rebooting-and-notifying-with-aws-lambda-and-slack-3e1b74d98557?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/3e1b74d98557</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[slack]]></category>
            <category><![CDATA[lambda-function]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Thu, 30 Nov 2023 06:01:51 GMT</pubDate>
            <atom:updated>2023-12-02T05:58:16.885Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/594/1*o_oxM1ahoyPdNmqfll8tGQ.jpeg" /></figure><p>In the dynamic landscape of cloud computing, maintaining the optimal health and functionality of EC2 instances is paramount. As organizations embrace the scalability and flexibility offered by AWS, the need for proactive and automated solutions becomes increasingly apparent. In this guide, we will explore a strategic approach to ensure the continuous well-being of your EC2 infrastructure. Specifically, we will delve into the implementation of an automated system utilizing AWS Lambda, designed to identify and reboot unhealthy EC2 instances seamlessly. Furthermore, our journey extends to the integration of Slack notifications, enabling real-time communication and alerting within your operational ecosystem. Join us as we navigate the realms of cloud automation, empowering your infrastructure to thrive with resilience and efficiency.</p><h4>Introducing Slack — Your Communication Hub</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*72Z_Cn4PdXjO3w_-RdG_XQ.jpeg" /></figure><p>Slack, an indispensable linchpin in modern collaboration, redefines how teams communicate and share information. As we delve into the orchestration of automated notifications, it’s essential to establish a robust connection between your AWS Lambda functions and the communication hub that is Slack. In this section, we’ll guide you through the straightforward process of creating an incoming webhook on Slack — a gateway to seamless integration and real-time updates within your chosen channel. Let’s embark on this journey, laying the foundation for efficient and automated communication in your workspace.</p><p><strong>Navigate to slack.com:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/1*WHuzoCW7jVttliqu8UHnWQ.png" /></figure><p>Begin by visiting the Slack website and creating an account if you haven’t already. Once logged in, establish a channel where you wish to receive notifications.</p><p><strong>Adding an Incoming Webhook</strong></p><p>After creating your channel, navigate to “Browse Slack” and click on “Apps” to explore the extensive range of integrations available.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/788/1*XASsp5Txd5PQ2kxBbSrGrQ.png" /></figure><p>Look for “Incoming Webhook” among the apps and proceed to add it to your workspace.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*102P1XWgpuxeimASXjipmg.png" /></figure><p>Click “Add to Slack” to initiate the integration process. Select the specific channel where you desire notifications and proceed by clicking “Add.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/1*wwC8Od867C-0tyIku4SRgg.png" /></figure><p><strong>Webhook Details</strong></p><p>Following the channel selection, you will receive comprehensive details about the incoming webhook, including a URL that serves as a unique identifier for your integration. This URL is crucial for connecting external services, such as AWS Lambda, to your Slack channel.</p><p>Here’s an example of what your webhook URL might resemble:</p><pre>https://hooks.slack.com/services/T04N9AUHXRQ/Bo4MSQZV9D5/UTvj8UwsT0p7zbpJpaqoMMf</pre><p>This webhook URL establishes a secure connection between your AWS Lambda function and the designated Slack channel, paving the way for automated notifications and seamless communication within your team.</p><p>Now that we’ve set up an incoming webhook on Slack, equipped with a unique URL, we’re ready to seamlessly integrate it with AWS. In the next steps, we’ll walk through the straightforward process of implementing AWS Lambda.</p><p><strong>Navigating AWS for IAM and Lambda Setup</strong></p><p>Let’s seamlessly transition to AWS for the next phase of our setup. First, we’ll create an IAM role to provide the necessary permissions:</p><p><strong>Navigate to AWS IAM Roles</strong></p><p>Head to the IAM Roles section on AWS and initiate the creation of a new role. Keep the default settings selected, choose Lambda as the use case, and proceed to the next step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*w6VbllAPKhocIdPSjVQ0LA.png" /></figure><p>Attach the following permission policies: AmazonEC2FullAccess and AWSLambdaBasicExecutionRole. Move on to the next step.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*fSrGO0MI-VKhlpHDs_G63w.png" /></figure><p>Assign an appropriate name to your role and finalize the creation by clicking on “Create Role.” With this, we’ve successfully established a role with the required permissions.</p><p>Next, let’s create a Lambda function:</p><p><strong>Navigate to AWS Lambda:</strong></p><p>Head to AWS Lambda and initiate the creation of a new function. Fill in the basic details</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/885/1*tXCCj7V8iNuOOBqRHnqkPw.png" /></figure><p>Below the permissions section, click on “Change default execution role.” Opt to “Use an Existing role” and select the role created earlier. Alternatively, you can create a new role following the same steps. Confirm the function creation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1004/1*AkwGskqgKWwN-aEGigmnww.png" /></figure><p>Now, armed with the Lambda function and its associated role, we are ready to implement the code. The provided script detects the health status of EC2 instances and sends a notification to Slack if any impairments are identified. Let’s proceed with this crucial step in automating our infrastructure:</p><pre>import boto3<br>import requests<br>import json<br><br>webhook_url = &quot;https://hooks.slack.com/services/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&quot;<br><br>def lambda_handler(event, context):<br>    ec2 = boto3.client(&quot;ec2&quot;)<br>    status = ec2.describe_instance_status(IncludeAllInstances=True)<br>    <br>    for i in status[&quot;InstanceStatuses&quot;]:<br>        instanceId = i[&quot;InstanceId&quot;]<br>        InstanceState = i[&quot;InstanceState&quot;][&#39;Name&#39;]<br>        status = i[&quot;InstanceStatus&quot;][&quot;Status&quot;]<br>        print(instanceId, &quot;: &quot;, status)<br>        <br>        if status == &#39;impaired&#39;:<br>            payloadText = &quot;EC2_ID-&quot; + instanceId + &quot; Is Stopped&quot;<br>            ec2.reboot_instances(InstanceIds=[instanceId])  #Reboot EC2<br>            payload = {&quot;text&quot;: payloadText}<br>            requests.post(webhook_url, json.dumps(payload)) #send notification to slack</pre><p>Deploy the code and click on “Test.” Upon execution, you may observe that it throws a module error.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*t07FlQae49t6o80C-yJevw.png" /></figure><p>To address this challenge and resolve the module error, You can access the step-by-step guide and valuable insights by following<strong> T</strong><a href="https://medium.com/@gauravkachariya/how-to-add-external-python-libraries-to-aws-lambda-499674113fb7"><strong>his Blog</strong></a><strong>.</strong></p><p>Upon implementing these changes, rerun your code to experience a smooth execution devoid of errors. Observe the effectiveness of the refined setup as you receive pertinent notifications on Slack. For a visual walkthrough, refer to the image below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/513/1*FyqdJQLxEOwYvg3wfOtUqw.png" /></figure><p>In summary, we’ve shown a simple and effective way to send Slack notifications for EC2 impairments, complete with an automatic restart using Boto3 and Python. This approach enhances AWS monitoring and responsiveness.</p><p>Indeed, the provided solution triggers when the Lambda function is executed. However, for continuous monitoring of your EC2 instances, you can leverage AWS EventBridge. This service allows you to set up recurring events at specific intervals, such as every minute, daily, or at predefined times. By integrating EventBridge, you establish a robust and automated monitoring system that ensures the health of your EC2 instances is regularly assessed, providing real-time insights into the status of your infrastructure. This additional layer of automation ensures a proactive approach to any potential issues, fostering a resilient and responsive AWS environment.</p><blockquote>Thank you for taking the time to read this article. I hope you found it informative and useful. If you have any questions or comments, please feel free to leave them below. Also, if you enjoyed this article, I would appreciate it if you could give it a clap or share it with your network. Finally, if you would like to connect with me on <a href="https://www.linkedin.com/in/gaurav-kachariya/">LinkedIn</a>, I would be more than happy to connect with you and continue the conversation there. Thank you again for reading!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3e1b74d98557" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging Terraform and Ansible for Seamless Infrastructure Automation]]></title>
            <link>https://medium.com/@gauravkachariya/leveraging-terraform-and-ansible-for-seamless-infrastructure-automation-c01bc740bff0?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c01bc740bff0</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[devops-practice]]></category>
            <category><![CDATA[ansible]]></category>
            <category><![CDATA[devops-tool]]></category>
            <category><![CDATA[terraform]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Sun, 26 Nov 2023 04:51:58 GMT</pubDate>
            <atom:updated>2023-11-26T04:51:58.365Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*y-MtZxwrAYoOB4-b.png" /></figure><p>In the ever-evolving landscape of Infrastructure as Code (IaC) and configuration management, Terraform and Ansible stand out as powerhouse tools. While each shines in its domain, combining them unlocks a synergistic approach for comprehensive, end-to-end workflows.</p><h3>Terraform at a Glance</h3><p>Terraform, born from HashiCorp, excels in managing and provisioning infrastructure across diverse environments like cloud platforms, Kubernetes, and RabbitMQ. Driven by HashiCorp Configuration Language (HCL), it adopts a declarative style, emphasizing reusability through modular components and dynamic configurations. Terraform’s state file ensures precise tracking of infrastructure states, a crucial element for seamless management.</p><h3>A Quick Overview of Ansible</h3><p>Ansible, an agentless configuration management tool, relies on SSH or WinRM for direct communication with hosts. Utilizing YAML in playbooks, Ansible orchestrates automation jobs with a focus on idempotence — ensuring consistent outcomes regardless of the number of executions. While capable of infrastructure provisioning, Ansible complements Terraform’s prowess in this arena.</p><h3>Why Terraform and Ansible Together?</h3><p>While Terraform manages infrastructure, Ansible excels in software installation and configuration on compute instances. Combining both is a strategic move. While Terraform’s provisioners offer Ansible-like capabilities, they’re less reliable. Conversely, using Ansible for infrastructure provisioning requires extensive coding for customization. Recognizing their strengths and weaknesses is key.</p><h3>Example setup — How to use Ansible with Terraform</h3><p>A common strategy involves using Terraform to establish the foundational infrastructure, covering networking, VM instances, and essential resources. Once this foundation is in place, Ansible can be seamlessly incorporated, whether triggered manually or through Terraform, to handle the configuration of instances, install the necessary software, and deploy applications.</p><p>Let’s explore a straightforward example where Terraform is employed to provision EC2 instances on AWS, followed by Ansible stepping in to configure and optimize these instances to suit specific needs:</p><pre>provider &quot;aws&quot; {<br>  region = &quot;ap-south-1&quot;<br>}<br><br>resource &quot;aws_instance&quot; &quot;ec2_instance&quot; {<br>  count         = 5<br>  ami           = &quot;ami-0287a05f0ef0e9d9a&quot;<br>  instance_type = &quot;t3.medium&quot;<br><br>  key_name      = &quot;ssh_key_1&quot;<br><br>  tags = {<br>    Name = &quot;EC2-instance-${count.index + 1}&quot;<br>  }<br>}<br><br>output &quot;public_ip_addresses&quot; {<br>  value = aws_instance.ec2_instance[*].public_ip<br>}</pre><p>In the above example, five EC2 instances are instantiated, each associated with a designated SSH key. Public IPs are allocated to expedite the configuration process through Ansible. In practical scenarios, however, instances typically possess solely private IPs, with configuration orchestrated via a bastion host to enhance security and network architecture.</p><p>Furthermore, we’ve set up an output to gather all the public IP addresses of the instances:</p><pre>output &quot;instances_ip_addresses&quot; {<br>  value = aws_instance.ec2_instance[*].public_ip<br>}</pre><p>Following the execution of the apply command, here are our observations:</p><pre>Apply complete! Resources: 5 added, 0 changed, 0 destroyed.<br>Outputs:<br>instances_ip_addresses = [<br>  &quot;88.64.204.106&quot;,<br>  &quot;54.73.244.160&quot;,<br>  &quot;34.254.227.95&quot;,<br>  &quot;33.133.71.201&quot;,<br>  &quot;103.209.22.65&quot;,<br>]</pre><p>Now, we can generate an Ansible inventory file based on the aforementioned servers. In Ansible, the inventory file serves as a configuration file for hosts, groups, and their corresponding tasks. It guides Ansible on where to execute tasks, enabling users to specify distinct variables and configurations for individual hosts.</p><p>Here is the method I employed to craft my inventory file:</p><pre>inventory.ini<br><br>[webservers]<br>88.64.204.106<br>54.73.244.160<br>34.254.227.95<br>33.133.71.201<br>103.209.22.65</pre><p>Let’s create a basic Ansible playbook to install Nginx:</p><pre>---<br>- name: Install Nginx<br>  hosts: all<br>  become: true  <br><br>  tasks:<br>    - name: Update apt package cache<br>      apt:<br>        update_cache: yes<br>      <br>    - name: Install Nginx<br>      apt:<br>        name: nginx<br>        state: present<br> <br>    - name: Start Nginx service<br>      service:<br>        name: nginx<br>        state: started</pre><p>We are now ready to execute Ansible to install Nginx on all the servers:</p><pre>ansible-playbook -i inventory install_nginx.yaml<br>TASK [Gathering Facts] *******************************************************************************************<br>ok: [88.64.204.106]<br>ok: [54.73.244.160]<br>ok: [34.254.227.95]<br>ok: [33.133.71.201]<br>ok: [103.209.22.65]<br>TASK [Update apt package cache] ***********************************************************************************<br>changed: [88.64.204.106]<br>changed: [54.73.244.160]<br>changed: [34.254.227.95]<br>changed: [33.133.71.201]<br>changed: [103.209.22.65]<br>TASK [Install Nginx] **********************************************************************************************<br>ok: [88.64.204.106]<br>ok: [54.73.244.160]<br>ok: [34.254.227.95]<br>ok: [33.133.71.201]<br>ok: [103.209.22.65]<br>PLAY RECAP ********************************************************************************************************<br>88.64.204.106             : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   <br>54.73.244.160             : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   <br>34.254.227.95             : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0<br>33.133.71.201             : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0<br>103.209.22.65             : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0</pre><p>In conclusion, this blog has guided you through the fundamentals of infrastructure automation using Terraform and Ansible. From EC2 instance creation and the deployment of an Ansible playbook for Nginx installation, you now possess a foundational understanding of these powerful tools. With the ability to streamline and automate various aspects of infrastructure management, Terraform and Ansible offer a potent combination for enhancing efficiency in diverse IT environments.</p><blockquote><em>Thank you for taking the time to read my article! If you found this article helpful, please feel free to share it with your network. And if you’d like to stay connected and be the first to know when I publish new content, don’t hesitate to connect with me on </em><a href="https://www.linkedin.com/in/gaurav-kachariya/"><em>LinkedIn</em></a><em>. Let’s continue the conversation there!</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c01bc740bff0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Continuous Deployment with Argo CD]]></title>
            <link>https://medium.com/@gauravkachariya/exploring-continuous-deployment-with-argo-cd-5aecd78dd396?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5aecd78dd396</guid>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[devops-practice]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[devops-tool]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Sat, 25 Nov 2023 07:57:14 GMT</pubDate>
            <atom:updated>2023-11-25T07:57:14.228Z</atom:updated>
            <content:encoded><![CDATA[<p>As DevOps practices become increasingly important for software delivery, tools that support continuous deployment (CD) are vital for automating the process of updating applications in production. In this article, we will take a look at Argo CD — an open-source CD tool designed specifically for Kubernetes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wa9ouqmnV58M6oHMaUq25g.png" /><figcaption>Argo CD</figcaption></figure><p>Argo CD allows developers to declaratively describe how applications should be deployed. It makes use of Git repositories to track desired application states and Kubernetes APIs to reconcile what is deployed against what is desired. By enabling automated deployment of configuration changes, Argo CD helps ensure production environments are running the desired code at all times.</p><p>To deploy Argo CD, it is essential to first install Docker and Kubernetes. Let us proceed with the installation process step by step.</p><p>To initiate the installation of Docker, execute the following command:</p><pre>sudo apt-get install docker.io</pre><p>Subsequently, validate the successful installation of Docker by running the command: docker version</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*HAVTO1WkNo6-deOTDEN9rA.png" /></figure><p>Following the installation of Docker, the next steps involve installing Minikube and kubectl, essential tools for Kubernetes.</p><p>Begin by downloading Minikube using the following command:</p><pre>curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64</pre><p>Proceed to install Minikube by executing:</p><pre>sudo install minikube-linux-amd64 /usr/local/bin/minikube</pre><p>Once Minikube is successfully installed, initiate a cluster to facilitate the installation of Argo CD. Execute the following command to commence a new cluster:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*xFVjuWyncivxdunxdc62Pg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*KUL_uWO5eKHUHVQ3Oy5XxA.png" /></figure><p>Various methods exist for installing Argo CD on a Kubernetes cluster. In this instance, we will opt for the HELM installation approach. Follow the subsequent commands to install HELM on your system:</p><pre>curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3</pre><p>Ensure the script is executable:</p><pre>chmod 700 get_helm.sh</pre><p>Execute the script to install HELM:</p><pre>./get_helm.sh</pre><p>These commands will facilitate the installation of HELM on your system.</p><h4>Installing Argo CD</h4><p>Execute the following commands to download and install Argo CD on your system:</p><p>Clone the Argo CD HELM repository:</p><pre>helm repo add argo https://argoproj.github.io/argo-helm</pre><p>Next, we can install Argo CD using the Helm command:</p><pre>helm install argocd argo/argo-cd</pre><p>This will install Argo CD in the default namespace. If you want to install it in a different namespace, you can use the --namespace flag.</p><h4>Access the Argo CD dashboard</h4><p>To access Argo CD, we need to forward the port. use following command for that:</p><pre>kubectl port-forward service/{YOUR_SERVICE_NAME} 8090:80 -n {YOUR_NAMESPACE_NAME} --address 0.0.0.0</pre><p>Replace {YOUR_SERVICE_NAME} with the actual name of the Argo CD service, and {YOUR_NAMESPACE_NAME} with the relevant namespace where Argo CD is deployed. This command will adapt to your specific configuration.</p><p>Now, navigate to either localhost:8080 or remoteip:8080 to access the Argo CD UI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*gmvYZGXtw4YDm2yKWOEiLg.png" /></figure><p>Upon accessing the Argo CD UI, you will be prompted to enter your credentials. The default username and password are admin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*0X6adqC_OMT0jeU2Opc34g.png" /></figure><p>Congratulations! Argo CD has been successfully configured on your system. If you encounter any issues or have further customization needs, feel free to explore the extensive capabilities of Argo CD to enhance your Kubernetes deployment and application management experience.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5aecd78dd396" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Containerized CI/CD: Running Docker Workloads Inside Jenkins Container]]></title>
            <link>https://medium.com/@gauravkachariya/configuring-docker-integration-in-your-jenkins-container-6035034f164c?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/6035034f164c</guid>
            <category><![CDATA[devops-practice]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[devops-tool]]></category>
            <category><![CDATA[jenkins]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Tue, 21 Nov 2023 07:18:51 GMT</pubDate>
            <atom:updated>2023-11-30T09:18:13.682Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*lB4oIEWvt9IouFv6m8iDlQ.png" /></figure><p>As DevOps practices continue to evolve bringing development and operations teams closer together, the need for automation and infrastructure as code has never been greater. Docker and Jenkins are two powerful open-source tools that can be combined to implement continuous integration and continuous delivery (CI/CD) pipelines for applications packaged as Docker containers.</p><p>In this blog, we will demonstrate how to configure your Jenkins server running in a Docker container to be able to build, run, and interact with other Docker containers on the same Docker host. With Docker integration enabled in Jenkins, you can build test, and deploy containerized applications through an automated CI/CD workflow.</p><h4>Prerequisites:</h4><ul><li>Docker installed and running on your system</li><li>Basic understanding of Docker and Jenkins</li></ul><h4>The steps are:</h4><ul><li>Create a Dockerfile</li></ul><pre>FROM jenkins/jenkins:lts<br>USER root<br>RUN apt-get update -qq \<br>    &amp;&amp; apt-get install -qqy apt-transport-https ca-certificates curl gnupg2 software-properties-common<br>RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -<br>RUN add-apt-repository \<br>   &quot;deb [arch=amd64] https://download.docker.com/linux/debian \<br>   $(lsb_release -cs) \<br>   stable&quot;<br>RUN apt-get update  -qq \<br>    &amp;&amp; apt-get -y install docker-ce<br>EXPOSE 8080<br>RUN usermod -aG docker jenkins</pre><ul><li>Build the Jenkins Docker image</li></ul><pre>docker image build -t jenkins-img .</pre><ul><li>Execute the following commands to run a Docker image and create a Jenkins container:</li></ul><pre>docker container run -d -p 8088:8080 -p 50000:50000 -v /var/run/docker.sock:/var/run/docker.sock --name Jenkins-c jenkins-img</pre><ul><li>Check whether the container is running using the following command:</li></ul><pre>docker ps</pre><ul><li>Retrieve the auto-generated password for Jenkins by typing the following command:</li></ul><pre>docker logs Jenkins-c</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*CJQT4LcNdoGtJ3rkvlo_MA.png" /></figure><p>To access Jenkins, open your web browser and navigate to either `localhost:8088` (for local access) or the IP address of your machine (in the case of a remote machine). This will take you to the Jenkins login page where you can enter the auto-generated password obtained earlier to proceed with the setup process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*IMrQTLQBvDUwav2_q9kUog.png" /></figure><p>If you encounter a screen similar to the image above, congratulations on successfully installing Jenkins! Paste the auto-generated password in the administrator password field to gain access to Jenkins and proceed with the configuration. This marks the successful completion of the Jenkins installation process as a Docker container.</p><blockquote><em>Thank you for taking the time to read this article. I hope you found it informative and useful. If you have any questions or comments, please feel free to leave them below. Also, if you enjoyed this article, I would appreciate it if you could give it a clap or share it with your network. Finally, if you would like to connect with me on </em><a href="https://www.linkedin.com/in/gaurav-kachariya/"><em>LinkedIn</em></a><em>, I would be more than happy to connect with you and continue the conversation there. Thank you again for reading!</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6035034f164c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Download Google Drive Files on Linux via Command Line]]></title>
            <link>https://medium.com/@gauravkachariya/download-google-drive-files-on-linux-via-command-line-c0ce06b51dba?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0ce06b51dba</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[google]]></category>
            <category><![CDATA[cloud-storage]]></category>
            <category><![CDATA[linux]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Sun, 19 Nov 2023 18:09:25 GMT</pubDate>
            <atom:updated>2024-03-01T08:27:59.727Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7cAkQn4AXluJdw7akeE4Pg.jpeg" /></figure><p>Have you ever found yourself needing to download a file from Google Drive on your Linux system, and you prefer a command-line approach? gdown, a tool designed to streamline file downloads without the need for a graphical interface. In the Linux environment, where GUI interactions are often limited, gdown proves to be a valuable asset for effortlessly retrieving files from Google Drive using simple command-line commands. Let&#39;s explore how you can enhance your file-downloading experience on Linux through CLI.</p><h3>Prerequisites</h3><p>Before we begin, ensure you have Python and pip3 installed on your system. While most Linux distributions come with Python pre-installed, you may need to install pip3 separately.</p><p>To install pip3 on Ubuntu, run the following commands:</p><pre>sudo apt update <br>sudo apt install python3-pip</pre><p>This will install pip3, the package installer for Python 3.</p><h3>Installing gdown</h3><p>Now that we have pip3 installed, we can proceed to install gdown. Open your terminal and use the following command:</p><pre>pip3 install gdown</pre><p>This command installs gdown and its dependencies, making it ready for use.</p><h3>Downloading a File from Google Drive</h3><p>Once you have gdown installed, you can use it to download a file from Google Drive by providing the file&#39;s Google Drive ID. Here&#39;s a step-by-step guide:</p><h4>1. Obtain the Google Drive File ID<strong>:</strong></h4><p>To download a file from Google Drive using gdown, you&#39;ll need the file&#39;s unique ID. Follow these steps to obtain the file ID:</p><h4>* Share the File:</h4><ul><li>Open your Google Drive and locate the file you want to download.</li><li>Right-click on the file and select “Share.”</li><li>In the sharing dialog, click on “Copy link.”</li><li>The link will look like <a href="https://drive.google.com/file/d/FILE_ID/view.">https://drive.google.com/file/d/FILE_ID/view</a></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DS6b_Ft3FGh9iyAPUGZfuA.gif" /></figure><h4>* Extract the File ID:</h4><ul><li>Paste the link into a text editor or directly into the terminal.</li><li>Extract the FILE_ID from the link. It is the long string of characters between &quot;<a href="https://drive.google.com/file/d/">https://drive.google.com/file/d/</a>&quot; and &quot;/view&quot;.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/831/1*YlE6x1oW6nbKPCUAsONYxA.png" /></figure><h4>2. Use gdown to Download the File:</h4><ul><li>Open your terminal and use the following command:</li></ul><pre>gdown https://drive.google.com/uc?id=YOUR_FILE_ID</pre><ul><li>Replace YOUR_FILE_ID with the actual Google Drive File ID you obtained in the previous step.</li><li>The file will be downloaded to your current working directory.</li></ul><h3>Additional Options</h3><p>gdown provides additional options to customize your download:</p><ul><li>Specifying Output File Name:</li></ul><p>You can use the -O option to specify the output file name. For example:</p><pre>gdown https://drive.google.com/uc?id=example_file_id -o output_file.txt</pre><ul><li>Resuming an Interrupted Download:</li></ul><p>If your download is interrupted, you can use the -c option to resume it:</p><pre>gdown https://drive.google.com/uc?id=example_file_id -c</pre><ul><li>Limiting Download Speed:</li></ul><p>You can limit the download speed using the --limit-rate option:</p><pre>gdown https://drive.google.com/uc?id=example_file_id --limit-rate 100k</pre><h3>Conclusion</h3><p>gdown streamlines the process of downloading files from Google Drive in a Linux environment. With just a few commands, you can effortlessly retrieve the files you need without the need for a graphical interface. Give it a try and enhance your file-downloading experience on Ubuntu!</p><p>Happy downloading!</p><blockquote>Thank you for taking the time to read my article! If you found this article helpful, please feel free to share it with your network. And if you’d like to stay connected and be the first to know when I publish new content, don’t hesitate to connect with me on <a href="https://www.linkedin.com/in/gaurav-kachariya/">LinkedIn</a>. Let’s continue the conversation there!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0ce06b51dba" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Dockerfile Composition: A Comprehensive Guide to Crafting Efficient Containerization…]]></title>
            <link>https://medium.com/@gauravkachariya/mastering-dockerfile-composition-a-comprehensive-guide-to-crafting-efficient-containerization-9cc967ad038c?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/9cc967ad038c</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Tue, 20 Jun 2023 12:53:38 GMT</pubDate>
            <atom:updated>2023-06-20T12:53:38.619Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mastering Dockerfile Composition: A Comprehensive Guide to Crafting Efficient Containerization Components</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*4Rka6hwNEdOBlk0A3vk_6w.png" /></figure><p>A Dockerfile is a script that contains commands and instructions used to build a Docker image. It specifies the desired environment, dependencies, configurations, and other settings required to run an application inside a container. The Dockerfile is used by the Docker engine to generate an image that can be deployed across various platforms and environments.</p><p>Before we start, let us take a momentary pause and familiarize ourselves with Docker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*ESmrGvFQ0sVmVd9RgyvXsw.jpeg" /></figure><h3>What is Docker?</h3><p>Docker is a platform that allows developers to create, deploy, and run applications in containers. Containers are lightweight images that contain all the necessary dependencies and configurations needed to run an application. Docker provides a consistent environment for developers, making it easier to develop and deploy applications. It also simplifies the deployment process by allowing applications to be deployed quickly across different environments, including local machines, servers, and cloud platforms. Docker has become popular due to its ability to improve efficiency, scalability, and security in software development. Overall, Docker’s containerization technology helps businesses save time, reduce costs, and streamline their development process.</p><h3>What is Dockerfile?</h3><p>A Dockerfile is a text file that contains a set of instructions for Docker to build an image. It serves as a recipe for creating a containerized application. Docker reads the instructions from the Dockerfile and executes them in order to create an image. To create a dockerfile, just create an empty file and save it as Dockerfile without any file extension. All of the instructions will be stored in this file.</p><h3>Components of Dockerfile:</h3><h4><strong>FROM:</strong></h4><p>Specify the base image that your Docker image will be built upon. It is the first instruction in a Dockerfile and is followed by the name of the base image.</p><p>When you build a Docker image, you start with a base image that already has an operating system and other dependencies installed. This base image serves as the foundation for your image, and you can then add your own application code and configuration on top of it.</p><p>For example, if you want to build an image for a Node.js application, you might use the official Node.js base image as follows:</p><pre>FROM node:14</pre><p>The “FROM” instruction is a fundamental part of Dockerfile syntax and is necessary for building Docker images. It allows you to specify the environment and dependencies for your application in a declarative way.</p><h4><strong>COPY:</strong></h4><p>The “COPY” instruction in a Dockerfile is used to copy files and directories from the host machine (where you are building the Docker image) to the image itself. It allows you to include necessary files and resources in your Docker image so that they are available when the image is run as a container.</p><p>The syntax for the “COPY” instruction is as follows:</p><pre>COPY &lt;src-path&gt; &lt;destination-path&gt;</pre><p>Here, &lt;src-path&gt; is the path to the file or directory on the host machine that you want to copy, and &lt;destination-path&gt; is the path inside the Docker image where you want to place the copied file or directory.</p><p>For example, if you have a file called “app.py” in the same directory as your Dockerfile, and you want to copy it to the “/app” directory inside the Docker image, you would use the following COPY instruction:</p><pre>COPY app.py /app/</pre><p>This will copy the “app.py” file from the host machine to the “/app” directory inside the Docker image.</p><p>It is important to note that the paths specified in the COPY instruction are relative to the build context, which is the directory that contains the Dockerfile. So make sure that the files and directories you want to copy are located within the build context or provide the absolute path to the files.</p><h4>ADD:</h4><p>The “ADD” instruction in a Dockerfile is similar to the “COPY” instruction, as both are used for copying files and directories from the host machine to the Docker image.</p><p>However, the “ADD” instruction can do more than just copying files. It allows using a URL as &lt;src&gt;, which means you can download files from remote locations during the build process. Nonetheless, it’s important to note that resources from remote URLs are not decompressed.</p><h4>WORKDIR:</h4><p>The “WORKDIR” instruction is used to set the working directory for any subsequent instructions like RUN, CMD, ENTRYPOINT, COPY, and ADD. It allows you to define the directory where these instructions will be executed within the Docker image.</p><p>The syntax for the “WORKDIR” instruction is as follows:</p><pre>WORKDIR /path/to/workdir/</pre><p>Here, “/path/to/workdir” is the path to the desired working directory inside the Docker image. If the specified directory does not exist, it will be created, even if it’s not used in any subsequent instructions.</p><p>The “WORKDIR” instruction can be used multiple times in a Dockerfile.</p><p>It is considered a best practice to set the “WORKDIR” explicitly in your Dockerfile to avoid unintended operations in unknown directories. This helps in maintaining clarity and predictability in your Docker image.</p><h4>RUN:</h4><p>The “RUN” instruction in a Dockerfile is used to execute commands during the build process of a Docker image. It allows you to run any command that you would normally run in a shell. The commands specified with the “RUN” instruction are executed in a new layer on top of the current image, and the changes made by these commands are committed to the image.</p><p>it can execute any valid command that you would run in a shell, such as installing packages, running scripts, or compiling code. For example, you can use the “RUN” instruction to install software packages using a package manager like apt-get or yum, or to download and extract files from the internet.</p><p>Here’s an example of a Dockerfile that demonstrates the use of the “RUN” instruction:</p><pre>FROM ubuntu:latest<br>RUN apt-get update &amp;&amp; apt-get install -y curl<br>RUN curl -o /tmp/run.sh https://raw.githubusercontent.com/user1/scripts/run.sh<br>RUN bash /tmp/run.sh</pre><p>It’s important to note that the “RUN” instructions are executed in the order they appear in the Dockerfile, and each instruction creates a new layer. Therefore, it’s a good practice to combine multiple commands into a single “RUN” instruction when possible to minimize the number of layers and reduce the final image size.</p><h4>ENTRYPOINT:</h4><p>The “ENTRYPOINT” instruction in a Dockerfile is used to configure the executable that will run as the main process inside the container when it is started. It allows you to define the command that will always run after the container is initiated, and it cannot be overridden or ignored, even when you run the container with command line arguments.</p><p>The syntax for the “ENTRYPOINT” instruction:</p><pre>ENTRYPOINT [&quot;node&quot;, &quot;app.js&quot;]</pre><p>You can pass command line arguments to the container usign ENTRYPOINT.</p><h4>CMD:</h4><p>The “CMD” instruction in a Dockerfile is used to specify the default command that will be executed when a Docker container starts. It defines the primary executable or process that should be run within the container.</p><p>The syntax for the “CMD” instruction is as follows:</p><pre>CMD [&quot;executable&quot;,&quot;param1&quot;,&quot;param2&quot;,...]</pre><p>The purpose of the “CMD” instruction is to define the default behavior of the container. It is used to specify the main process that should be run when the container starts. For example, if you have a container that runs a web server, the “CMD” instruction can be used to start the web server process.</p><p>If the Dockerfile contains multiple “CMD” instructions, only the last one will be executed. This means that the previous “CMD” instructions will be ignored. It’s important to note that the “CMD” instruction can be overridden when running the container with alternative command-line arguments.</p><p>The “CMD” instruction can also be used in conjunction with the “ENTRYPOINT” instruction. The “ENTRYPOINT” specifies the executable that will run when the container starts, and the “CMD” provides default arguments for that executable. This allows users to easily override the default behavior by providing their own command line arguments.</p><p>It’s worth mentioning that the “CMD” instruction is not necessary for a container to work. If you don’t specify a “CMD” instruction, the container will use the default behavior defined by the base image.</p><h4>EXPOSE:</h4><p>The “EXPOSE” instruction in a Dockerfile is used to indicate the ports on which a container listens for connections. It informs Docker about the network ports that the container is expected to use at runtime.</p><p>The purpose of the “EXPOSE” instruction is to document the ports that the container is listening on. It serves as a form of documentation for users who might want to connect to the application running inside the container. For example, if you have a web server running inside the container and it listens on port 80, you can use the “EXPOSE” instruction to indicate that the container exposes port 80.</p><p>It’s important to note that the “EXPOSE” instruction does not make the ports of the container accessible to the host or to other containers by default. It is purely for documentation purposes. To make the exposed ports accessible, you need to use the -p flag when running the container to publish the ports or use the -P flag to publish all the exposed ports.</p><p>Here’s an example of a Dockerfile that uses the “EXPOSE” instruction:</p><pre>FROM nginx:latest<br>EXPOSE 80</pre><p>In this example, the Docker image is based on the latest version of the NGINX image. The “EXPOSE” instruction indicates that the container will listen on port 80.</p><p>When running the container, you can use the -p flag to map the exposed port of the container to a port on the host machine. For example:</p><pre>docker run -p 8080:80 my-nginx-image</pre><p>This command maps port 80 of the container to port 8080 on the host machine, allowing access to the NGINX web server running inside the container.</p><h4>VOLUME:</h4><p>The “VOLUME” instruction in a Dockerfile is used to create a mount point or a named volume in a Docker container. It allows you to specify a directory or a path inside the container where data can be stored or accessed persistently, even after the container is stopped or deleted.</p><p>The “VOLUME” instruction has two forms:</p><p><strong>1.Anonymous volumes:</strong></p><pre>  VOLUME [&quot;/path/to/directory&quot;]</pre><p>In this form, you specify the path to the directory inside the container where the volume will be created. Docker will automatically create a new volume at the specified path when the container is created. The data stored in this volume will be preserved even if the container is removed.</p><p>When a container is created, a new volume will be created at “/path/to/directory”. Any data written to this directory will be stored persistently, and it can be accessed by other containers or the host machine.</p><p><strong>2. Named volumes:</strong></p><pre>   VOLUME [&quot;volume_name&quot;, &quot;/path/to/directory&quot;]</pre><p>In this form, you can specify a name for the volume, which allows you to reference it across multiple containers. Docker will create a named volume with the specified name and mount it at the specified path inside the container.</p><p>It’s important to note that the “VOLUME” instruction does not populate the volume with any existing data from the image. It simply creates a mount point or a named volume where data can be stored or accessed.</p><p>When running a container, you can use the -v flag to mount a host directory or specify a named volume to be mounted at the specified path inside the container. For example:</p><pre>docker run -v /path/on/host:/app/data my-image</pre><h4>LABEL:</h4><p>the “LABEL” instruction in a Dockerfile is used to add metadata to an image. It allows you to attach key-value pairs to the image, providing information about the image, such as its version, maintainer, description, or any other custom information you want to include.</p><p>The “LABEL” instruction has the following syntax:</p><pre>LABEL key1=value1 key2=value2 ...</pre><p>You can specify multiple key-value pairs by separating them with spaces. The keys and values can be any alphanumeric string, and the values can be enclosed in double quotes if needed.</p><p>Here’s an example of how the “LABEL” instruction can be used in a Dockerfile:</p><pre>FROM ubuntu:latest<br>LABEL maintainer=&quot;Gaurav Kachariya&quot;<br>LABEL version=&quot;1.0&quot;<br>LABEL description=&quot;This is a sample Docker image&quot;</pre><p>Labels can be used for various purposes, such as documenting the image, providing contact information for the maintainer, specifying licensing information, or adding any other relevant details. They can also be useful for organizing and searching for images in a Docker registry.</p><p>You can view the labels attached to an image by using the docker inspect command. For example:</p><pre>docker inspect my-image</pre><p>It’s important to follow best practices for writing efficient and maintainable Dockerfiles. Here are some additional tips:</p><ul><li><strong>Use a minimal base image:</strong> Choosing a lightweight base image, such as Alpine Linux, can help reduce the size of the final image and improve startup time.</li><li><strong>Leverage caching: </strong>Docker uses caching to speed up the build process. By ordering the instructions in your Dockerfile from least likely to change to most likely to change, you can take advantage of caching and avoid unnecessary rebuilds.</li><li><strong>Use multi-stage builds:</strong> Multi-stage builds allow you to create intermediate images during the build process, which can help reduce the size of the final image. This is especially useful when building applications with multiple build dependencies.</li><li><strong>Optimize layers:</strong> Each instruction in a Dockerfile creates a new layer in the image. It’s important to minimize the number of layers to reduce image size and improve performance.</li><li><strong>Use .dockerignore:</strong> By using the “.dockerignore” file, you can select which files and folders should be excluded from the build context, resulting in a shorter build time and avoiding the inclusion of any unnecessary files in the image.</li></ul><p>once you are ready with your dockerfile, you can build docker image with following command:</p><pre>docker image build -t image-name .</pre><p>Then use command below to create and run container from the image:</p><pre>docker run image-name</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iAiUaeuqr4fG9-LnZeHRHg.jpeg" /></figure><p>Up until now, we have covered the basic steps required for creating a Dockerfile, generating an image from the Dockerfile, and launching a container based on the image. These steps are essential for building and deploying applications with Docker.</p><blockquote>Thank you for taking the time to read this article. I hope you found it informative and useful. If you have any questions or comments, please feel free to leave them below. Also, if you enjoyed this article, I would appreciate it if you could give it a clap or share it with your network. Finally, if you would like to connect with me on <a href="https://www.linkedin.com/in/gaurav-kachariya/">LinkedIn</a>, I would be more than happy to connect with you and continue the conversation there. Thank you again for reading!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9cc967ad038c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Kubernetes Dashboard: The Ultimate Setup Guide]]></title>
            <link>https://medium.com/@gauravkachariya/demystifying-kubernetes-dashboard-the-ultimate-setup-guide-39f38d25a145?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/39f38d25a145</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Wed, 14 Jun 2023 06:36:59 GMT</pubDate>
            <atom:updated>2023-06-20T17:47:30.435Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UVyB9YNoU2W26MPallsf5Q.jpeg" /></figure><p>Kubernetes Dashboard is a web-based user interface for managing and interacting with Kubernetes clusters. It provides a visual representation of the Kubernetes resources running in your cluster, allowing you to easily manage them and troubleshoot issues when they arise.</p><p>The dashboard provides a number of useful features including:</p><ol><li><strong>Cluster Overview</strong>: Provides an overview of your entire cluster, including information about nodes and pods, as well as resource utilization metrics.</li><li><strong>Workload Management:</strong> Allows you to manage your workloads, such as deployments and daemonsets, from within the dashboard itself.</li><li><strong>Pod Management:</strong> This enables you to view and manage individual pods, including their logs and resource utilization.</li><li><strong>Resource Metrics:</strong> Displays real-time resource usages metrics, such as CPU and memory usage, for each pod and node in your cluster.</li><li><strong>Namespace Management:</strong> Enables you to manage namespaces and their associated resources, such as pods and services.</li><li><strong>Custom Resource Definition (CRD) Management: </strong>Allows you to create and manage custom resource definitions, which can be used to extend the functionality of Kubernetes.</li><li><strong>User Management:</strong> Provides RBAC-based access control, enabling you to grant specific users or groups of users access to specific resources within your cluster.</li></ol><p>The dashboard also supports a number of plugins and extensions, which can be used to extend its functionality even further. For example, you can integrate third-party monitoring tools like Prometheus, or add custom widgets to display additional data.</p><p>One of the main benefits of using Kubernetes Dashboard is that it simplifies the process of managing and troubleshooting Kubernetes clusters. By providing a user-friendly interface, it enables both developers and operators to quickly and easily interact with the cluster, without having to use complex command-line tools and scripts.</p><p>In order to set up the cluster, we must first install Docker and Minikube before proceeding to install the Kubernetes dashboard.</p><blockquote>For the purposes of this tutorial, I will be using an Ubuntu machine.</blockquote><h4>Installing Docker</h4><p>Docker is a platform that allows developers to easily create, deploy, and run applications in containers. Containers are lightweight, portable, and isolated environments that can run on any operating system or infrastructure. Docker simplifies the process of building, testing, and deploying applications by providing a standardized way of packaging software and its dependencies into a single package. With Docker, developers can quickly spin up new instances of their applications, scale them as needed, and ensure consistent performance across different environments. Docker has become an essential tool for modern software development, enabling agile workflows, continuous delivery, and DevOps practices.</p><p>To install Docker on an Ubuntu machine, execute the below command:</p><pre>sudo apt-get install docker.io –y</pre><p>Check the version of Docker to ensure that it is installed. If you get a permission error message, then run the next command.</p><pre>sudo chmod 777 /var/run/docker.sock</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*uaDsJoLypcfwWYujtH3cHA.png" /><figcaption>Docker installation</figcaption></figure><h4>Installing Minikube</h4><p>Minikube is a great tool that makes it easy to run Kubernetes locally. It runs a single-node Kubernetes cluster inside a virtual machine on your local machine, allowing developers to test and experiment with Kubernetes without the need for a separate cluster. It is a great tool for developers to learn and test Kubernetes without any real infrastructure.</p><p>Run the following commands to download and install Minikube:</p><pre>curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64</pre><pre>sudo install minikube-linux-amd64 /usr/local/bin/minikube</pre><p>Once it’s done, you need to start the Minikube cluster. Execute the below command. kindly be patient and allow some time for the tool to fully start up.</p><pre>minikube start </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*_xXx4-JkIhNAJIUKMyEE4A.png" /><figcaption>Minikube installation</figcaption></figure><h4>Installing kubectl</h4><p>kubectl is the CLI tool that can be used to interact with a Kubernetes cluster. It allows you to deploy, inspect, and manage the state of your applications and resources in the cluster. It can be used to create, update and delete resources such as pods, services, and deployments, and to view and troubleshoot the status of your applications. It is the main CLI tool to communicate with a Kubernetes cluster.</p><p>Run the following commands to download and install kubectl:</p><pre>curl -LO &quot;https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl&quot;</pre><pre>sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl</pre><h3>Kubernetes Dashboard</h3><p>We have already installed kubectl. Now, let’s install the Kubernetes dashboard using kubectl. Refer to the following command:</p><pre>kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml</pre><p>The next step is to Access Kubernetes Dashboard on Web Browser. For that, we need to forward the port as shown below:</p><pre>kubectl port-forward -n kubernetes-dashboard service/kubernetes-dashboard 10443:443 --address 0.0.0.0</pre><p>Visit port 10443 on your web browser. you will be able to get a screen as below image. For security, it will ask for a token to log in to your dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*s1qh0bQ3ZwLVg3Knluacgg.png" /></figure><p>To log in, we need to generate a token to access the dashboard.</p><p>There are two types of accounts in Kubernetes:</p><ul><li><strong>User Account:</strong> It is used to allow us, humans, to access the given Kubernetes cluster. Any user needs to get authenticated by the API server to do so. A user account can be an admin or a developer who is trying to access the cluster-level resources.</li><li><strong>Service Account:</strong> It is used to authenticate machine-level processes to get access to our Kubernetes cluster. The API server is responsible for such authentication to the processes running in the pod.</li></ul><p>For this demo, let’s create a service account. You can give any name according to you. I am using kubernetes-dashboard.</p><pre>Kubectl create serviceaccount kubernetes-dashboard</pre><p>Now, let’s generate the tokens for the service account that we have just created.</p><pre>kubectl create token --duration=48h kubernetes-dashboard</pre><p>Once you execute the above command, you’ll get a token that can be used to log in to Kubernetes Dashboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*-5lrBo8uyVjp9OzgOFPnvA.png" /></figure><p>Copy the token and paste it on the authentication page.</p><p>It will redirect to the Kubernetes dashboard, once you’ll be able to log in successfully.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*TITnu_NC22Vy0Sr9nHYBUg.png" /></figure><p>If you have successfully logged into the dashboard but still cannot access anything, click on the notification icon to find out the reason.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/526/1*fdrt9fj62T7QJIlOjyOdGA.png" /><figcaption>serviceaccount permission errors</figcaption></figure><p>It means by default our serviceaccount has no permissions to read, list, etc operations for pods, deployments, etc. Refer to the below YAML file. We are going to grant permissions to our serviceaccount.</p><pre>kind: ClusterRole<br>apiVersion: rbac.authorization.k8s.io/v1<br>metadata:<br>  namespace: default<br>  name: k8db<br>rules:<br>  - apiGroups: [&quot;apps&quot;,&quot;autoscaling&quot;,&quot;batch&quot;,&quot;extensions&quot;,&quot;policy&quot;] <br>    resources: [&quot;pods&quot;, &quot;replicasets&quot;, &quot;jobs&quot;, &quot;apps&quot;, &quot;deployments&quot;, &quot;nodes&quot;]<br>    verbs:     [&quot;get&quot;, &quot;watch&quot;, &quot;list&quot;]<br>---<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: ClusterRoleBinding<br>metadata:<br>  name: kubernetes-dashboard<br>subjects:<br>  - kind: ServiceAccount  <br>    name: k8db            # Reference to upper&#39;s `metadata.name`<br>    namespace: default    # Reference to upper&#39;s `metadata.namespace`<br>roleRef:<br>  kind: ClusterRole<br>  name: cluster-admin<br>  apiGroup: rbac.authorization.k8s.io</pre><p>Now, save the file with permission.yaml and let’s apply it using kubectl.</p><pre>kubectl apply –f permission.yaml </pre><p>Now, go to the dashboard again and just reload the web page. You’ll be able to access the Dashboard without any trouble.</p><blockquote><em>Thank you for taking the time to read my article! If you found this article helpful, please feel free to share it with your network. And if you’d like to stay connected and be the first to know when I publish new content, don’t hesitate to connect with me on </em><a href="https://www.linkedin.com/in/gaurav-kachariya/"><em>LinkedIn</em></a><em>. Let’s continue the conversation there!</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=39f38d25a145" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cron Job Mastery: A Comprehensive Guide to Task Scheduling with Cron Tab]]></title>
            <link>https://medium.com/@gauravkachariya/cron-jobs-the-ultimate-guide-ef56e7697fdb?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef56e7697fdb</guid>
            <category><![CDATA[li̇nux]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[script]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Fri, 09 Jun 2023 06:20:21 GMT</pubDate>
            <atom:updated>2024-01-29T16:21:06.500Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zxHZXfiSYsNYe-qrkeSGJg.jpeg" /></figure><p>Cron is a time-based job scheduler in Unix/Linux operating systems. It is a tool that allows users to schedule tasks or scripts to run automatically at specified intervals, such as daily, weekly, or monthly. Cron jobs can be used for a variety of purposes, such as running backups, sending automated emails, updating software, and more. A cron job is a command that runs automatically at specified intervals without user interaction.</p><p>Here are some additional information about cron jobs:</p><ul><li>The execution of cron jobs is commonly carried out by the cron daemon, a background process that operates on Unix-like operating systems.</li><li>Cron jobs can be used to run any type of command, including scripts, programs, and shell commands.</li><li>Cron jobs can be scheduled to run at specific times, dates, or intervals.</li><li>Cron jobs can be used to automate a wide variety of tasks, such as backups, file transfers, and email notifications.</li></ul><p>Cron jobs are defined in a configuration file called the crontab file. This file contains a list of commands that cron will run at a specified time. Each command in the crontab file is represented by a line that specifies the schedule and the terminal command to execute.</p><p>The syntax for crontab entries is as follows:</p><pre>*     *     *   *    *    command to be executed<br>-     -     -   -    -<br>|     |     |   |    |<br>|     |     |   |    +----- day of the week (0 - 6) (Sunday = 0)<br>|     |     |   |<br>|     |     |   +------- month (1 - 12)<br>|     |     |<br>|     |     +--------- day of the month (1 - 31)<br>|     |<br>|     +----------- hour (0 - 23)<br>|<br>+------------- min (0 - 59)<br></pre><p>Each asterisk represents a wildcard value, meaning that all possible values are accepted. For example, an entry of * * * * * /path/to/command would run the command every minute, every hour, every day of the month, every month, and every day of the week.</p><p>Type <strong>crontab -e</strong> and hit Enter. This will open the default text editor for your system and allow you to edit your cron jobs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*9KjXHj9Pkq7qbScXdTh19A.png" /><figcaption>Default crontab File</figcaption></figure><p>Let’s say you want to execute your backup script every day at 8:00 PM. The below cron job setting will help you.</p><pre>0 20 * * *  /home/ubuntu/gaurav/backup.sh</pre><p>let’s say you want to execute the script at 00:05 in August.</p><pre>5 0 * 8 * script.sh</pre><p>These configurations should be added in the cron file as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*UjbHjTTHHOTVQv3zRjOnnQ.png" /><figcaption>Adding cron jobs in crontab</figcaption></figure><h3>Let’s take a look at the operators that can be used to construct complex schedules.</h3><h4><strong>1. Comma (,)</strong></h4><p>In cron jobs, the comma is used to separate values in a field. When a comma-separated list is entered in a cron job’s field, the command will execute for each value specified.</p><ul><li>Let’s say you have a cron job that runs every day at 6:30 AM and 6:30 PM, you can use commas to separate the hours:</li></ul><pre>30 6,18 * * * script.sh</pre><p>The above command will run script.sh twice a day, once at 6:30 AM and a second time at 6:30 PM.</p><h4>2. Slash (/)</h4><p>In cron jobs, the forward slash (/) is used to define step values for a given field. It allows you to specify intervals at which a command is executed in the specified field.</p><ul><li>Suppose you want to run a command every 5 minutes. You can use the following cron job:</li></ul><pre>*/5 * * * * /path/to/command.</pre><p>The asterisk (*) means “any value”, so this command will run every minute, but the forward slash with the number 5 specifies that it should only run every five minutes. The result is that the command runs at the start of every 5th minute.</p><ul><li>Similarly, if you want to run a command every 2 hours, you can use the following cron job:</li></ul><pre>0 */2 * * * /path/to/command</pre><p>In this case, the first field is 0, meaning the command will run at the start of the hour. The forward slash and the number 2 in the second field specify that the command should run every two hours.</p><ul><li>You can use Slash with other fields as well. For instance, if you want to run a command every other day starting from the 1st of the month, you can use the following cron job:</li></ul><pre>0 0 1-31/2 * * cd /home/gaurav &amp;&amp; /bin/sh script.sh &gt;&gt; logout.data</pre><p>This command will run at midnight on every other day starting from the 1st of the month.</p><h4>3. Hyphen (-)</h4><p>In cron jobs, the hyphen (-) is used to specify ranges of values for a given field. It allows you to define a starting and ending value for the field.</p><ul><li>Suppose you want to run a command every day at 10:00 AM. You can use the following cron job:</li></ul><pre>0 10 * * * /path/to/command</pre><p>This command will execute the /path/to/command script at 10:00 AM every day. But what if you want to run the command only on weekdays? In this case, you can use a hyphen to specify a range of values for the fifth field, which represents the day of the week:</p><pre>0 10 * * 1-5 /path/to/command</pre><p>It’s important to note that you can also combine the use of hyphens and commas to specify more complex schedules. For instance, the following example runs a command on the first and last days of January, April, July, and October:</p><pre>0 0 1,31 1,4,7,10 * /path/to/command</pre><p>In this example, the commas separate different values, and the hyphens specify ranges. The command will run at midnight on the first and last days of every January, April, July, and October.</p><p>Overall, the hyphen is a powerful tool for specifying ranges of values in cron jobs, allowing you to create more complex and flexible schedules.</p><h4>Troubleshooting cron jobs</h4><p>If you are having problems with your cron jobs, there are a few things you can check:</p><ul><li>Make sure that you have entered the correct syntax in your crontab file.</li><li>Make sure that the cron daemon is running.</li><li>Check the permissions on your crontab file.</li><li>Check the logs for any errors.</li></ul><h4>Conclusion</h4><p>Cron jobs are a powerful tool that can be used to automate tasks on Unix-like operating systems. They can be used to run anything from backups to email notifications to software updates.</p><p>If you are looking for a way to automate tasks on your server, cron jobs are a great option.</p><blockquote>You can use <a href="https://crontab-generator.org/">CrontabGenerator</a> to create cron tabs.</blockquote><blockquote><em>Thank you for taking the time to read my article! If you found this article helpful, please feel free to share it with your network. And if you’d like to stay connected and be the first to know when I publish new content, don’t hesitate to connect with me on </em><a href="https://www.linkedin.com/in/gaurav-kachariya/"><em>LinkedIn</em></a><em>. Let’s continue the conversation there!</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef56e7697fdb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to add External Python Libraries to AWS Lambda]]></title>
            <link>https://medium.com/@gauravkachariya/how-to-add-external-python-libraries-to-aws-lambda-499674113fb7?source=rss-23b7436770a9------2</link>
            <guid isPermaLink="false">https://medium.com/p/499674113fb7</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Gaurav kachariya]]></dc:creator>
            <pubDate>Mon, 29 May 2023 05:33:41 GMT</pubDate>
            <atom:updated>2023-05-29T05:33:41.383Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*b986vGwuadFXeMsiVInBOg.png" /></figure><p>AWS Lambda is a computing service offered by Amazon Web Services (AWS) that operates in a serverless manner. This means that you can execute code without needing to allocate or handle servers. The charges are based on the duration of computation utilized by your code. AWS Lambda is equipped with powerful capabilities to interact with other AWS services and third-party libraries.</p><p>When developing a function using AWS Lambda, you have access to several pre-built modules and libraries. In some cases, you may require accessing Python libraries that are not available in the standard Lambda environment.</p><pre>import requests</pre><p>You might get an error like this:</p><pre>Unable to import module &#39;lambda_function&#39;: No module named &#39;requests&#39;</pre><p>To resolve this, we need to create our own libraries as custom packages and then attach the package to lambda.</p><p>There are several ways to add external Python libraries to AWS Lambda, depending on your specific requirements and preferences. One common approach is to use layers, which enables you to bundle and deploy your preferred libraries together with the Lambda function code. By using layers, you can more effectively manage dependencies and reuse code across multiple functions within your AWS environment.</p><h4><strong>Let’s Create an AWS lambda function and test</strong></h4><p>To get started, log in to your AWS account and navigate to the Lambda service. Once there, click on “Create Lambda” and proceed to fill out the details provided below. Choose the appropriate Python runtime version and click on create function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*IsA52WNNKGYOEngfKF9oug.png" /></figure><p>Once, lambda is created, you’ll get some default code. For this demo, I’ve imported python-requests.</p><p>Click on Deploy to save your code. Then click on Test to run the code. First time you’ll get a prompt to name the test event. Enter the Event name as “ pythonRequests “ and leave everything else as default.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*yf88jvAQOFhVx0hP8ZADBw.png" /></figure><p>Click “Test” again, and you should get a similar response below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*49XiC4YvLChpfKBmkoZGag.png" /></figure><p><strong>Let’s solve the problem in 3 easy steps:</strong></p><p>1. Install your packages on your local machine and zip</p><p>2. Create a lambda layer and upload packages</p><p>3. Attach layer to lambda and test</p><h3>1. Install your packages on AWS cloudshell and zip</h3><p>AWS CloudShell is a web-based shell environment that allows you to run commands against your AWS resources from anywhere, without the need for any client-side tools or installations.</p><p>Quickly open the AWS S3 console, create an S3 bucket, and give it a unique name. In this bucket, you will store the zip package that we will create soon.</p><p>Now open AWS Cloudshell. It can be found at the top right of the console. It looks like the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/944/1*HU4O_Cy_HRUdDqX4wkEDJA.png" /><figcaption>AWS CloudShell</figcaption></figure><p>Once The terminal is loaded, create a directory named “packages”, then change the directory. To perform this, run the below commands one after another.</p><pre>mkdir packages<br>cd packages</pre><p>To proceed, please create a virtual environment with the name “venv”. Then, activate the virtual environment by running the following commands in sequence.</p><pre>python3 -m venv venv<br>source venv/bin/activate</pre><p>Your virtual environment should be activated now. Next, create a folder named “python” and change to the python directory by running the commands below one after the other.</p><pre>mkdir python<br>cd python</pre><p>Keep in mind that it’s important to create a directory named python. While working with lambda layers, this is essential. Install Python packages in the python directory by running the commands.</p><pre>pip install requests -t . </pre><p><em>You can install packages as many as you want.</em></p><p>So far, we have executed the commands shown in the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*gNT-pWCqyEuetwQr2kn55g.png" /></figure><p>You can list the data in the current directory by typing “ls” in the terminal, and the output should resemble the image below image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*Cyw-WdM4nzOiXsBh95Uj6Q.png" /></figure><p>These are the packages that we’ll need to use for Lambda. In order to package them together, we’ll have to zip up the Python folder where they reside. However, before we do that, it’s a good idea to save some space by removing any objects with the “.dist-info” extension that aren’t necessary. You may do this by executing the next command:</p><pre>rm -rf *dist-info</pre><p>Once you’ve executed the above command, you should be left with only the relevant packages that are needed for Lambda.</p><pre>cd ..</pre><p>Now, let’s zip the python directory and name it “lambda-package.zip” using the following command:</p><pre>zip -r lambda-package.zip python</pre><p>After executing the previous command, you should see a new zip file named “lambda-package.zip” in your current directory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*ubgq2g168Ijkl-T3aNl8gg.png" /></figure><p>To upload the zipped package into the S3 bucket that we just created, you can use a command similar to the following one:</p><pre>aws s3 cp lambda-package.zip  s3://your-s3-bucket-name/</pre><p>Go to the S3 bucket you created. Copy the object URL by clicking the checkbox next to the object and clicking Copy S3 URI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*QKBK1sRNjQsF7-yOMaN0hw.png" /></figure><h3>2. Create lambda layer and upload packages</h3><p>Go back to your AWS lambda console and select layers as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*Eqm7UA6pnG1smJSJIx2fRg.png" /></figure><p>Click on “Create Layer” and fill in the required configuration. See the image below. select the “Upload file from Amazon S3” option and paste the URL we copied earlier into the text box below. Select the appropriate Python runtime and click Create.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*4oWj5h_RGxh8VDC8VvIcKg.png" /></figure><h3>3. Attach layer to lambda and test</h3><p>In this section, we will attach the lambda layer created above to our lambda function.</p><p>Go to the lambda function we created earlier and scroll to the very bottom. In Layers, you can see an option to add a layer, as shown in the image below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*8ETCUzjYHF1vFq8fyQe5VA.png" /></figure><p>Click “Add a Layer” and a page for configuration should appear. Select “Custom Layers” and from the dropdown, select the layer we created in the previous section. If it asks for a version, select version 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*jck_Gl2ZS8jYgyuKs3BWtw.png" /></figure><p>Once the layer is added, try running the lambda function again. It will run without any problem.</p><blockquote>Thank you for taking the time to read my article! If you found this article helpful, please feel free to share it with your network. And if you’d like to stay connected and be the first to know when I publish new content, don’t hesitate to connect with me on <a href="https://www.linkedin.com/in/gaurav-kachariya/">LinkedIn</a>. Let’s continue the conversation there!</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=499674113fb7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>