The Adventures of Blink S3e7: Automation with GitHub Actions

Hey everyone! Blink here, back for another adventure! You can always find me over on GitHub.

Before we dive in, a quick reminder: if you find value in today's exploration, please let me know with an emoji or a comment! Your feedback helps me understand what resonates and guides me in creating even better content for you in the future. And of course, don't forget to subscribe so you don't miss out on any future adventures, especially as we're in the midst of a season where each episode builds upon the last. Join the party and let's learn and grow together!

TL/DR: Youtube

Don't feel like reading, would you rather watch? Don't worry, I've got ya...

Automating all the things

Today, we're tackling a topic that's close to my heart: automation. I've often heard it said that the best software engineers are "productively lazy." Now, that might sound like an oxymoron, but it makes perfect sense when you reframe it as being automation obsessed.

Think about it: a great engineer doesn't want to waste time on repetitive, manual tasks. They want to get things done efficiently and effectively. So, if there's a way to automate those tedious tasks, they're all in! This frees up their mental energy and time to focus on more valuable and complex challenges.

This naturally leads to the question: what should we automate? In my experience, and what I see echoed throughout the DevOps community, the answer boils down to one word:

Toil

We define toil as repetitive, manual work that doesn't add lasting value and scales as a service grows. We all have these kinds of tasks in our lives – the things that are necessary to keep the wheels turning but don't actually improve our product or directly benefit our customers.

The repetitive nature of toil makes it the perfect candidate for automation. If a task is predictable and follows the same steps every time, we can script it out and let machines handle it. By minimizing the effort spent on toil, we maximize the time we can dedicate to activities that truly create value.

Let's look at some concrete examples of toil that are ripe for automation:

• Compiling Code: Manually building your code into an executable is time-consuming, requires focus, and is prone to errors if you're rushing or miss a step. Automating the build process ensures consistency and frees up your workstation for other tasks.
• Security Scans: Tools for scanning your code for supply chain vulnerabilities (like Black Duck) or code quality issues (like SonarQube) can take a significant amount of time. Automating these scans allows them to run without direct human intervention, providing consistent and timely feedback.
• API Documentation: Thankfully, tools have existed for years that can generate developer documentation directly from your API code. Integrating this into your workflow saves developers countless hours of writing and explaining their code.
• Other Documentation: With the rise of large language models, the possibility of automating even more documentation is becoming a reality. Imagine passing your code through an LLM to generate a first draft of your technical documentation!
• Deployment: Perhaps the most popular use case for developer automation. In the past, deploying code involved manually copying files to servers and painstakingly configuring them. Automating this entire process, making it touch-free, is a core competency of DevOps. It should be easy to get your code into production!

Now, you might be wondering, what does all of this have to do with Git? We're talking about various tools and processes that don't inherently live within Git itself.

Well, our code lives in Git, right? And all of these automation tasks often take code as their input. It makes sense that if we're going to automate this work, we'd want to start close to the source of truth – our code repository.

While Git itself doesn't have built-in automation protocols, the web-based tools built around Git, like GitHub and GitLab, offer powerful automation features. Over the next two weeks, we'll be exploring these capabilities. Today, we're starting with GitHub and its Actions feature.

Some of you who have been on this adventure with me since Season 2, Episode 5 might remember us touching on GitHub Actions. Back then, we focused on using it for unit testing – automatically running our tests every time we pushed code to ensure we weren't introducing regressions.

But today, we're going to take a broader look at GitHub Actions, as it can be used for so much more than just unit testing!

Diving into GitHub Actions

Let's see how to create a GitHub Action directly from the interface. In your repository, navigate to the "Actions" tab. You'll see a variety of pre-configured workflows, but we're going to start with a "Set up a workflow yourself" option to create a simple workflow from scratch.

GitHub will provide a basic template, which already has most of what we need to get started.

First, you'll need a name for your workflow. This is how it will be identified in the Actions tab when you're reviewing past runs.

Next, you'll see the on variable. This defines when your workflow will run. The common triggers are:

• push: Triggers the workflow when code is pushed to the repository.
• pull_request: Triggers the workflow when a pull request is created or updated.  
• workflow_dispatch: Allows you to manually trigger the workflow from the Actions tab.

You'll typically choose either push or pull_request depending on the automation task. For example, you might want to trigger a deployment workflow on a push to your main branch, while you'd want to run security scans on pull_request to provide feedback to the reviewer before merging. Failing workflows on a pull request can even prevent the pull request from being merged until the issues are resolved. Unit tests are another great example of something you'd want to run on a pull request.

Within a workflow, the next key concept is jobs. A workflow can consist of multiple jobs, which are the building blocks of your automation. Each job has a name.

Inside a job, the first crucial element is runs-on. This specifies the GitHub Runner that will execute the job. Think of runners as isolated build environments, often based on container technology (though the underlying implementation might vary). They come pre-equipped with certain software, allowing them to perform various tasks. In our example, we'll use ubuntu-latest, which means our job will run on a fresh Linux server.

Finally, within a job, we have steps. Steps are the individual actions or commands that are executed within the runner. To run a linter, you might have a step that executes the linter command against your code. Steps can also include simple commands like echo to provide documentation or output during the workflow execution.

Building a Simple Python Test Workflow

Let's imagine we have a Python application and we want to automate the process of installing dependencies and running unit tests. Here's how we might configure our GitHub Action:

name: Run PyTest

on:
  pull_request:
    branches: [ main ]

jobs:
  run-pytest:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4 # Using a specific version for stability

      - name: Set up Python 3.12
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install Dependencies
        run: pip install -r requirements.txt

      - name: Run Tests
        run: pytest

In this workflow:

• We give our workflow the name Run PyTest.
• We specify that it should run on every pull_request targeting the main branch.
• We define a single job named run-pytest that will run on an ubuntu-latest runner.

Our steps are:

• Checkout Repository: This step uses the official actions/checkout action to clone our repository onto the runner. Specifying a version ensures stability.
• Set up Python 3.12: This step uses the actions/setup-python action to set up the Python environment with version 3.12.
• Install Dependencies: This step executes the command pip install -r requirements.txt to install the project's dependencies.
• Run Tests: This step executes the pytest command to run our unit tests.

When we commit this workflow file (typically named something like main.yml) to the .github/workflows directory in our repository, GitHub Actions will automatically recognize it.

You'll notice that as soon as we commit this file, the workflow might even try to run! In our example, it likely failed because the repository doesn't actually have a requirements.txt file. This is perfectly normal and highlights how you can set up your automation even before your application code is fully complete. This allows you to define your desired workflow early in the development process.

Expanding Our Workflow: Pre-Deployment Checks

Let's consider a more complex scenario where we want to perform several checks before allowing code to be deployed. We can add more steps and even multiple jobs to our workflow. For instance, we might want to run a dependency check using OWASP Dependency Check:

name: Pre-Deployment Checks

on:
  pull_request:
    branches: [ main ]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
      - name: Set up Python 3.12
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install Dependencies
        run: pip install -r requirements.txt
      - name: Run Tests
        run: pytest

  dependency-check:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
      - name: Run OWASP Dependency Check
        uses: dependency-check/dependency-check-action@v3 # Example action
        with:
          project: 'My Project'
          scan_type: 'All'

Here, we've introduced a second job called dependency-check that uses a community action for OWASP Dependency Check. You can find a vast ecosystem of pre-built actions for various tasks, making it easy to integrate powerful functionality into your workflows.

You can even incorporate custom scripts for tasks like database configuration or interact with cloud services like AWS to deploy Lambda functions directly from your GitHub Actions.

Important Considerations: Secrets and Resource Management

As you build more sophisticated workflows, keep these crucial points in mind:

• Secrets Management: Never store sensitive information like API keys or SSH keys directly in your workflow files. GitHub provides a secure Secrets feature where you can store these values and reference them in your actions. Make sure to research and implement proper secrets management to protect your credentials.
• Resource Efficiency: While GitHub offers a generous amount of free usage for Actions, be mindful of your workflow execution. Only run workflows when absolutely necessary to avoid exceeding your free tier and incurring costs. Consider the frequency of your triggers (e.g., running tests on every commit vs. only on pull requests).

Wrapping Up

And just like that, you have the foundational knowledge to start automating your entire developer workflow with GitHub Actions! It's a powerful tool that can significantly reduce toil, improve consistency, and free up your time to focus on what truly matters.

Don't forget to subscribe so you don't miss next week's episode, where we'll be exploring the GitLab equivalent of GitHub Actions – their CI/CD pipelines. We'll compare the similarities and differences and see what each platform has to offer. It's going to be an interesting comparison!

I can't wait to continue this adventure with you. See you next time!