CI/CD Pipelines: Part 1
DevOps series:
Docker and Containers: Part1 Overview
Docker and Containers: Part 2 in practice
Docker and containers: Part3 Microservices and Docker compose
Orchestration of the containers: Part 1
Orchestration of the containers: Part 2 Deploying at scale a containerized 3-tier Web Application
Orchestration of the containers: Part3 Enhancing the Kubernetes cluster
CI/CD Pipelines: Part 2 (GitHub Actions in Practice)
CI/CD Pipelines: The main practice of DevOps
After delving into Docker as a tool for containerization and Kubernetes as a container orchestration solution, one might believe that the journey of developing and deploying applications is complete. With Docker, we possess a neatly encapsulated image of our application, ready to run in any environment. Additionally, Kubernetes stands by to elegantly manage and ensure the smooth operation of our application in production. However, what if we seek to introduce a new feature to our application? The process becomes manual: rebuilding the Docker image, pushing it to the Docker registry as a new version, and updating the Kubernetes configuration files. As previously emphasized in this book, DevOps revolves around automation, and here’s where we introduce the pivotal concept of CI/CD: Continuous Integration/Continuous Delivery or Continuous Deployment (we’ll explore the nuances between them shortly). So, what exactly is CI/CD?
Automating the SDLC using C/CD pipelines
The Continuous Integration and Continuous Deployment (CI/CD) pipeline consists of a series of steps required to deliver new software. This approach to automation aims to optimize software delivery throughout the software development lifecycle.
By implementing CI/CD methodologies that span the development, testing, manufacturing, and evaluation phases, we can accelerate code development to ensure quality and increased security Although the manual processing of each phase is still an option though, but the real value of CI/CD pipelines in their automated implementation.
Specifically, the CI/CD pipeline guides software development by automating development through steps such as build, test, and code implementation. Automation helps reduce human error and maintain accuracy in the release process. The pipeline may include code compilation, unit testing, code analysis, security testing, and binary builds. In containerized environments (which is our case), the pipeline includes packaging code into container images for deployment.
As a cornerstone of the DevOps approach, CI/CD enhances collaboration between developers and IT service teams to streamline software deployment. In today’s competitive environment where customized applications are the differentiator, the speed at which code can be released has emerged as an important competitive advantage
To summarize, CI/CD automates much, or all of the manual human intervention traditionally needed to get new code from a commit into production, encompassing the build, test (including integration tests, unit tests, and regression tests), and deploy phases, as well as infrastructure provisioning. With a CI/CD pipeline, development teams can make changes to code that are then automatically tested and pushed out for delivery and deployment.
As previously mentioned, A CI/CD pipeline is a series of steps that must be performed in order to deliver a new version of software. These steps are typically organized into four primary phases of release:
- Source: Check-in source code and peer review new code.
- Build: Compile code, manage dependencies, test code (Unit tests, etc.), package the App and create container images.
- Staging: Deploy to Staging environment and test the App: Load tests, UI tests, etc.
- Production: Deploy to production environment.
A typical example for a CI CD pipeline:
Now that we understand CI/CD pipelines, it’s time to break them down into their individual components: Continuous Integration (CI) and Continuous Delivery (CD). We’ll explore these concepts in more detail in the following sections.
Continuous Integration (CI)
Let’s look at Continuous Integration (CI) with an example. Imagine you work for a company that hasn’t started using the DevOps approach. In this situation, team members often work alone for a long time and then merge their changes into the code only when they finish their tasks. This way of working can cause problems during the merging process, leading to more chances of conflicts and using up a lot of time. Also, mistakes build up over time and are only dealt with later in the development process. But with CI, developers regularly save their code changes in a shared place using systems like Git. A CI system does different jobs like building the code, saving parts of it, doing tests, and helping with code reviews using tools like Sonar. By setting up the CI system to start when there’s a new save or merge in the code, developers make sure that integrating the code goes smoothly. This way of working encourages teamwork, reduces integration problems, and makes it easier to find and fix mistakes faster, making software development and release quicker and smoother.
Get Blogs4devs’s stories in your inbox
Join Medium for free to get updates from this writer.
In summary, Continuous Integration (CI) is a software development practice where developers regularly integrate their code changes into a central repository. This process is followed by automated builds and tests. This concept can be illustrated by the following diagram which shows a representation of a CI pipeline:
Once the developer commits their code to a version control system like Git, it triggers the CI pipeline which fetches the changes and runs automated build and unit tests. Based on the status of the step, the server then notifies the concerned developer whether the integration of the new code to the existing code base was a success or a failure.
Continuous Delivery vs Continuous Deployment:
Continuous Delivery:
Continuous delivery helps developers test their code in a production-similar environment, hence preventing any last-moment or post-production surprises. These tests may include UI testing, load testing, integration testing, etc. It helps developers discover and resolve bugs preemptively.
By automating the software release process, CD contributes to low-risk releases, lower costs, better software quality, improved productivity levels, and most importantly, it helps us deliver updates to customers faster and more frequently. If Continuous Delivery is implemented properly, we will always have a deployment-ready code that has passed through a standardized test process.
CD or Continuous Delivery is carried out after Continuous Integration to make sure that we can release new changes to our customers quickly in an error-free way. This includes running integration and regression tests in the staging area (similar to the production environment) so that the final release is not broken in production. It ensures automation of the release process so that we have a release-ready product at all times, and we can deploy our application at any point in time.
Continuous Delivery automates the entire software release process. The final decision to deploy to a live production environment can be triggered by the developer/project lead as required. Some popular CD tools are AWS CodeDeploy, Jenkins, and GitLab.
Continuous Deployment:
The final stage of CI and CD will be continuous deployment. It’s an extension of continuous delivery, which automates the proper code to the code repository, continuous deployment will automate the related app for production purpose because there is not having any manual gate at the stage of the pipeline before production, continuous deployment relies on high automation.
In simple language, it is a change of application that goes through the cloud which is carried by the developer, and it will live within a few minutes of writing pass with the automated testing.
The below image describes how Continuous Integration combined with Continuous Delivery helps quicken the software delivery process with lower risks and improved quality.
We have seen how Continuous Integration automates the process of building, testing, and packaging the source code as soon as it is committed to the code repository by the developers. Once the CI step is completed, the code is deployed to the staging environment where it undergoes further automated testing (like Acceptance testing, Regression testing, etc.). Finally, it is deployed to the production environment for the final release of the product.
If the deployment to production is a manual step. In that case, the process is called Continuous Delivery whereas if the deployment to the production environment is automated, it is referred to as Continuous Deployment.
Another way to see the CI CD pipeline with the four major phases of release is the following:
CI/CD Pipelines with GitHub Actions
There are many tools for CI/CD like Jenkins, GitLab CI/CD, CircleCI, Travis CI, and more, but for our purposes, we will use GitHub Actions due to its seamless integration with our existing GitHub repositories and its robust features for automating our CI/CD workflows.
GitHub Actions is a CI/CD (Continuous Integration/ Continuous Deployment) platform for automating the builds, test, and deployment process. Using GitHub actions, we can build and test every pull request (A pull request is a request made by a developer to merge their changes into the main codebase. For example, if a developer wants to add a new feature, they create a pull request for their changes to be reviewed and approved before merging them into the main branch.) in the repository using workflows (we will see what is a workflow soon) or push the merged pull requests to production with workflows. GitHub Actions allows you to conduct processes in response to the events in your repository. For example, anytime someone files a new issue in your repository, you may execute a process to automatically add the required labels or if someone sends a pull request to your repository, then the automated workflows built by you can check if the pull request is eligible for merging into the main code or not.
Components Of GitHub Actions:
When an event occurs in the repository, such as when a pull request is opened or an issue is raised, one can set up the GitHub Actions process to be triggered. The workflow includes one or more than one job that can be executed sequentially or concurrently. The components of GitHub Actions are listed below:
- Workflows: A workflow is a programmable automated procedure that executes one or more tasks. Workflows are written in a YAML (Yet Another Markup Language) file that is checked into your repository. They can be triggered by an event in your repository, manually, or on a set timetable. There can be a number of workflows that can be triggered at the desired time. For example, you may have a workflow that builds tests for every pull request that is sent to your repository or a workflow that pushes your code into production once the built test passes. So as soon as someone sends a pull request to your repository, the first workflow triggers and builds the code to check if it is working fine or not. All the workflows are written in a specific file and in a specific directory on GitHub that is, .github/workflows/file.yml. Here .github/workflows are the directory and file.yml is the file with the extension in which the script will be written.
- Events: Events are the states or activities at which the workflows will be triggered. When someone pushes the newly committed code to the repository or files an issue or creates a pull request, are some events that will ultimately trigger the GitHub actions to check for workflows, if any, and do the jobs as specified in the workflow. The job can also be started manually or on a specified timetable.
- Jobs: A job is like a collection of workflow stages that run on the same runner. Each step can either be a shell script or an action that will be executed. All the steps are carried out in a sequential manner and are interdependent. Being interdependent, the exchange of data between the steps can be done easily. Let us take the previous example only, which says to send the code to the production once the build passes. Now, as soon as the build passes the next step to send the code to the production will be triggered else it will fail.
- Actions: An action is a GitHub Actions platform-specific custom program that performs a sophisticated but regularly performed job. Use action to help you write less repeated code in your workflow files. An action can grab your git repository from GitHub, configure your build environment’s toolchain, or configure your cloud provider’s credentials.
- Runners: In simple terms, runners are the servers that execute the workflows built by you. Just like a C++ code needs to be compiled first using mingw and then run, similarly when the workflows are triggered, a runner executes them. Each runner is limited to one job at a time. To run the workflows, GitHub provides Ubuntu Linux, Microsoft Windows, and macOS runners, each workflow run takes place on a new, freshly provisioned virtual machine. You can host your own runners if you need a different operating system or a specific hardware setup.
Those components can be summarized in the figure below:
That concludes this blog. See you in the next one! 👍
