Building a Tiny CI Server with Pythonanywhere and GitHub Webhooks

Originally drafted in 2022. Revisited and updated in 2026.

Most developers encounter Continuous Integration through tools such as GitHub Actions, CircleCI, Jenkins, or Travis CI.

You push some code, something happens somewhere, your tests run, and a few seconds later GitHub tells you whether your commit passed or failed.

It is easy to use CI for years without ever thinking much about what is actually happening underneath.

So instead of configuring another CI platform, we’re going to build a tiny one.

Using only Python, Flask, GitHub Webhooks, and PythonAnywhere, we’ll build a server that receives an event whenever code is pushed to a GitHub repository and runs the project’s tests.

The finished system will look roughly like this:

The architecture of a basic CI Setup

This is intentionally a small educational implementation rather than something I would recommend replacing GitHub Actions with in production.

The interesting part is not PythonAnywhere itself. The goal is to understand the primitive behind a CI system:

an event happens → a server receives it → work is triggered → a result is produced.

Once you understand that loop, CI becomes much less mysterious.

What is Continuous Integration?

Continuous Integration, or CI, is a development practice in which changes from developers are integrated into a shared codebase frequently and automatically checked.

Those checks may include:

linting
        ↓
type checking
        ↓
unit tests
        ↓
integration tests
        ↓
build
        ↓
deployment

A modern CI platform can do considerably more than this, but at its core we only need a few pieces:

1. Something that detects a code change
2. Something that receives the event
3. Something that executes our checks
4. Some representation of the result

GitHub gives us the first part through webhooks.

What is a GitHub webhook?

A webhook is simply an HTTP request sent by one application to another when an event occurs.

Instead of our CI server repeatedly asking GitHub:

Has anyone pushed yet?
Has anyone pushed yet?
Has anyone pushed yet?

we give GitHub an HTTP endpoint and tell it:

When somebody pushes, send the event here.

When a push occurs, GitHub sends an HTTP POST request containing information about the repository, branch, commit and other details.

For example:

git push
   |
   v
GitHub
   |
   | POST /webhooks/github
   | X-GitHub-Event: push
   | X-Hub-Signature-256: ...
   |
   v
Our Flask application

Our Flask application then decides what to do with that event.

There is one important security consideration: we should not trust any random HTTP request claiming to be GitHub.

When configuring the webhook, we’ll create a secret shared between GitHub and our application. GitHub signs every request with that secret, and our server verifies the signature before doing anything.

GitHub currently sends that signature using the X-Hub-Signature-256 header and recommends HMAC-SHA256 verification.

Architecture

Our tiny CI system will consist of three main components:

That is essentially our entire CI server.

Real systems add job queues, isolated runners, containers, caching, parallel execution, secrets management, artefacts, logs, permissions and considerably more.

But the fundamental event loop remains surprisingly similar.

Before we start

You’ll need:

• a GitHub account;
• a PythonAnywhere account;
• a GitHub repository containing a Python project;
• at least one automated test;
• basic familiarity with Python and Flask.

For this tutorial, I’ll assume the project’s tests can be executed with:

pytest

You can substitute another command later.

Creating the PythonAnywhere application

PythonAnywhere hosts WSGI web applications and provides us with a public URL that GitHub can send webhook requests to.

From the Web tab in PythonAnywhere, create a new Flask web application using an available modern Python version.

PythonAnywhere’s current system image supports Python 3.11, 3.12 and 3.13, although the exact versions available depend on the system image attached to your account. PythonAnywhere Help

You can also use Manual configuration, which is what the original version of this tutorial used. PythonAnywhere still supports both Flask quick-start applications and manually configured WSGI applications. PythonAnywhere Help

Once your application is running, you should have a URL similar to:

https://yourusername.pythonanywhere.com

Opening that URL should display your Flask application.

Creating the webhook endpoint

Now we can create the endpoint GitHub will call.

from flask import Flask, request, abort

app = Flask(__name__)


@app.post("/webhooks/github")
def github_webhook():
    event = request.headers.get("X-GitHub-Event")

    if event != "push":
        return {"status": "ignored"}, 200

    payload = request.get_json()

    repository = payload["repository"]["full_name"]
    commit = payload["after"]

    print(f"Received push for {repository}")
    print(f"Commit: {commit}")

    return {"status": "received"}, 200

At this point, our application isn’t actually doing CI yet.

It is simply receiving a GitHub event.

And that’s useful because we can test one piece at a time.

Don’t trust the request yet

Before this endpoint can execute arbitrary commands on our server, we need to verify that the request genuinely came from GitHub.

Set a random webhook secret:

WEBHOOK_SECRET=<a-long-random-secret>

Then verify GitHub’s signature:

import hashlib
import hmac
import os

from flask import abort


def verify_github_signature(body: bytes, signature: str | None):
    if not signature:
        abort(403)

    secret = os.environ["WEBHOOK_SECRET"].encode()

    expected = "sha256=" + hmac.new(
        secret,
        body,
        hashlib.sha256,
    ).hexdigest()

    if not hmac.compare_digest(expected, signature):
        abort(403)

Now modify our endpoint:

@app.post("/webhooks/github")
def github_webhook():
    body = request.get_data()

    verify_github_signature(
        body,
        request.headers.get("X-Hub-Signature-256"),
    )

    if request.headers.get("X-GitHub-Event") != "push":
        return {"status": "ignored"}, 200

    payload = request.get_json()

    repository = payload["repository"]["full_name"]
    commit = payload["after"]

    print(f"Verified push for {repository}@{commit}")

    return {"status": "received"}, 200

Now we have something substantially more interesting:

GitHub
   │
   │ signed event
   ▼
Flask
   │
   ├── calculate expected signature
   │
   ├── compare signatures
   │
   ├── invalid ──────► 403
   │
   └── valid
          │
          ▼
     process event

Connecting GitHub

Open the repository you want to test and navigate to:

Settings → Webhooks → Add webhook

Set the payload URL to:

https://yourusername.pythonanywhere.com/webhooks/github

Content type: application/json

Enter the same secret you configured on PythonAnywhere.

For events, select:

Just the push event

Save the webhook.

GitHub should send a test delivery to your application.

At this point you’ve built the first half of the system:

git push
   ↓
GitHub
   ↓
signed webhook
   ↓
PythonAnywhere
   ↓
Flask
   ↓
signature verification
   ↓
push recognised

Preparing the repository on the CI server

Our CI server needs its own local copy of the repository.

Open a Bash console on PythonAnywhere and create a directory for the repository:

mkdir -p ~/ci-workspace
cd ~/ci-workspace

Clone the repository that you want the CI server to test:

git clone https://github.com/YOUR_USERNAME/YOUR_REPOSITORY.git project

Then enter the project directory:

cd project

Install the dependencies required by the project.

For example, if the project contains a requirements.txt file:

pip install -r requirements.txt

We will assume that the project’s tests can be executed using:

python -m pytest

Before continuing, run the tests manually once:

python -m pytest

If the test suite runs successfully here, we are ready to make the CI server run it automatically.

Why we should test the exact commit

The webhook payload contains an important field called after.

For a push event, this value contains the SHA of the commit that GitHub has just received.

A simplified payload might look something like this:

{
  "ref": "refs/heads/main",
  "after": "a918de82f85e...",
  "repository": {
    "full_name": "username/project"
  }
}

It might be tempting to make our CI server simply execute:

git pull

whenever a webhook arrives.

That would work in simple cases, but there is a subtle problem.

Suppose two commits are pushed in quick succession:

Commit A pushed
      |
      | webhook A
      v

Commit B pushed
      |
      | webhook B
      v

By the time the server handles webhook A, the repository’s latest commit may already be commit B.

If we simply run git pull, the build triggered by commit A could accidentally test commit B.

That is not what we want.

A CI build should be reproducible. If GitHub tells us that commit abc123 triggered a build, then abc123 is the commit we should test.

So instead of asking Git for “the latest version”, we will explicitly check out the commit SHA contained in the webhook.

Creating the test runner

Create a new file called runner.py.

1. Preparing the repository on the CI machine

from dataclasses import dataclass
from pathlib import Path
import subprocess
import sys


PROJECT_DIR = Path.home() / "ci-workspace" / "project"


@dataclass
class BuildResult:
    commit: str
    passed: bool
    return_code: int
    output: str


def run_command(command):
    return subprocess.run(
        command,
        cwd=PROJECT_DIR,
        capture_output=True,
        text=True,
        timeout=120,
    )


def checkout_commit(commit_sha):
    fetch = run_command([
        "git",
        "fetch",
        "origin",
    ])

    if fetch.returncode != 0:
        raise RuntimeError(
            f"Unable to fetch repository:\n{fetch.stderr}"
        )

    checkout = run_command([
        "git",
        "checkout",
        "--force",
        "--detach",
        commit_sha,
    ])

    if checkout.returncode != 0:
        raise RuntimeError(
            f"Unable to checkout commit {commit_sha}:\n"
            f"{checkout.stderr}"
        )


def run_tests():
    return run_command([
        sys.executable,
        "-m",
        "pytest",
    ])


def run_build(commit_sha):
    checkout_commit(commit_sha)

    result = run_tests()

    output = result.stdout + result.stderr

    return BuildResult(
        commit=commit_sha,
        passed=result.returncode == 0,
        return_code=result.returncode,
        output=output,
    )

There are two important pieces here.

First:

git fetch origin

updates Git’s knowledge of the remote repository.

Then:

git checkout --force --detach <commit>

checks out the exact commit that triggered the build.

The --detach flag means that we are not moving a local branch such as main. We simply tell Git:

Give me the repository exactly as it existed at this commit.

That is useful for CI because the build is associated with a commit rather than with whatever happens to be the latest branch state.

Running the tests

Our test runner eventually executes:

subprocess.run(
    [
        sys.executable,
        "-m",
        "pytest",
    ],
    ...
)

subprocess.run lets our Python application execute another program—in this case, pytest.

Three options are particularly useful here:

capture_output=True

captures what pytest writes to standard output and standard error.

text=True

returns that output to us as strings rather than raw bytes.

And:

timeout=120

prevents a broken test suite from running indefinitely.

The result returned by subprocess.run also contains a returncode.

Like most command-line programs, pytest returns:

0       tests passed successfully

non-zero
        something failed

This means our tiny CI server can determine whether a build passed with:

result.returncode == 0

That little expression is effectively the heart of our first CI implementation.

Connecting the runner to the webhook

We can now modify the webhook endpoint.

Get Junior Brown’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Assuming that the signature verification code from the previous section is already present, our route can look like this:

@app.post("/webhooks/github")
def github_webhook():
    body = request.get_data()

    verify_github_signature(
        body,
        request.headers.get("X-Hub-Signature-256"),
    )

    event = request.headers.get("X-GitHub-Event")

    if event != "push":
        return {
            "status": "ignored",
            "event": event,
        }, 200

    payload = request.get_json()

    commit_sha = payload["after"]
    repository = payload["repository"]["full_name"]

    result = run_build(commit_sha)

    return {
        "repository": repository,
        "commit": result.commit,
        "status": "passed" if result.passed else "failed",
        "return_code": result.return_code,
        "output": result.output,
    }, 200

Do not forget to import our runner:

from runner import run_build

Now the entire flow looks like this:

Developer
    |
    | git push
    v
 GitHub
    |
    | POST /webhooks/github
    v
Flask Server
    |
    | verify signature
    v
Read commit SHA
    |
    v
git fetch
    |
    v
checkout exact commit
    |
    v
python -m pytest
    |
    v
capture exit code + output
    |
    v
PASS / FAIL

We now have something that can legitimately be called a very small CI server.

Push a commit to the repository and GitHub sends the webhook.

The webhook reaches PythonAnywhere.

PythonAnywhere checks out the commit.

The tests run.

And the server determines whether the build passed.

Pretty cool for a relatively small amount of Python.

Testing it

Let us deliberately create a passing build first.

Suppose our project contains the following function:

def add(a, b):
    return a + b

And the corresponding test:

def test_add():
    assert add(2, 2) == 4

Commit and push the code:

git add .
git commit -m "Add passing test"
git push

GitHub should send the push event to our webhook.

Our server will check out the new commit and execute the test suite.

Because pytest exits with code 0, our CI server reports:

{
  "status": "passed"
}

Now deliberately break the test:

def test_add():
    assert add(2, 2) == 5

Commit and push again.

This time pytest returns a non-zero exit code and our server reports:

{
  "status": "failed"
}

We have now reproduced one of the fundamental behaviours of platforms such as GitHub Actions, CircleCI and Travis CI:

code change
    ↓
automated test run
    ↓
pass or fail

Of course, those platforms contain significantly more infrastructure around this loop, but the primitive is already here.

There is a problem with our implementation

There is, however, something wrong with the architecture we just built.

Our webhook request currently does this:

receive webhook
      ↓
fetch repository
      ↓
checkout commit
      ↓
run entire test suite
      ↓
return HTTP response

That means GitHub has to wait for our tests to finish before our application responds to the webhook.

That is not a good design.

GitHub expects webhook receivers to respond with a successful 2xx HTTP response within 10 seconds. If the response takes longer, GitHub considers the webhook delivery unsuccessful. GitHub Docs

A test suite could easily take:

30 seconds
5 minutes
20 minutes
or much longer

The webhook receiver therefore should not be responsible for performing the build itself.

Its job should be much smaller:

receive event
     ↓
verify event
     ↓
record build job
     ↓
return immediately

A separate process should execute the build.

In other words, what we actually want is this:

                 GitHub
                    |
                    | webhook
                    v
             Flask CI Server
                    |
                    | create job
                    v
               Build Queue
                    |
                    | next job
                    v
                Worker
                    |
             +------+------+
             |             |
             v             v
       checkout code    run tests
             |             |
             +------+------+
                    |
                    v
              save result

This separation between receiving work and executing work is a pattern that appears everywhere in distributed systems.

So let us improve our CI server.

Adding a tiny build queue

We could introduce Redis, RabbitMQ or another dedicated message queue here.

But that would introduce quite a lot of infrastructure for a tutorial and for PythonAnywhere, whose goal is understanding CI.

Instead, we are going to build an extremely small queue using SQLite.

SQLite ships with Python, so we do not need another database server.

Create a file called database.py.

from pathlib import Path
import sqlite3


DATABASE = Path.home() / "ci-server" / "builds.db"


def get_connection():
    connection = sqlite3.connect(DATABASE)
    connection.row_factory = sqlite3.Row
    return connection


def initialize_database():
    with get_connection() as connection:
        connection.execute("""
            CREATE TABLE IF NOT EXISTS builds (
                id INTEGER PRIMARY KEY AUTOINCREMENT,
                delivery_id TEXT UNIQUE,
                repository TEXT NOT NULL,
                commit_sha TEXT NOT NULL,
                status TEXT NOT NULL DEFAULT 'queued',
                output TEXT,
                return_code INTEGER,
                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
                started_at TIMESTAMP,
                finished_at TIMESTAMP
            )
        """)

Each row represents one CI build.

A build can move through several states:

queued
  ↓
running
  ↓
passed

or

queued
  ↓
running
  ↓
failed

Queueing builds

Add the following function:

def create_build(delivery_id, repository, commit_sha):
    with get_connection() as connection:
        cursor = connection.execute(
            """
            INSERT INTO builds (
                delivery_id,
                repository,
                commit_sha,
                status
            )
            VALUES (?, ?, ?, 'queued')
            """,
            (
                delivery_id,
                repository,
                commit_sha,
            ),
        )

        return cursor.lastrowid

Our Flask application no longer needs to run the tests.

Instead, it creates a build job.

The webhook endpoint becomes:

@app.post("/webhooks/github")
def github_webhook():
    body = request.get_data()

    verify_github_signature(
        body,
        request.headers.get("X-Hub-Signature-256"),
    )

    event = request.headers.get("X-GitHub-Event")

    if event != "push":
        return {
            "status": "ignored",
            "event": event,
        }, 200

    payload = request.get_json()

    repository = payload["repository"]["full_name"]
    commit_sha = payload["after"]
    delivery_id = request.headers.get("X-GitHub-Delivery")

    build_id = create_build(
        delivery_id=delivery_id,
        repository=repository,
        commit_sha=commit_sha,
    )

    return {
        "status": "queued",
        "build_id": build_id,
        "commit": commit_sha,
    }, 202

A 202 Accepted response is useful here because it describes exactly what has happened:

We accepted your request, but the work has not necessarily completed yet.

Our webhook now does very little work:

GitHub
   |
   v
verify request
   |
   v
create build record
   |
   v
202 Accepted

That is much better.

GitHub itself recommends processing webhook payloads asynchronously when work may take longer than the webhook response window.

Creating the worker

Now we need something that continuously looks for queued builds and executes them.

Create worker.py.

import time

from database import get_connection
from runner import run_build


def get_next_build():
    connection = get_connection()

    try:
        connection.execute("BEGIN IMMEDIATE")

        build = connection.execute(
            """
            SELECT *
            FROM builds
            WHERE status = 'queued'
            ORDER BY id
            LIMIT 1
            """
        ).fetchone()

        if build is None:
            connection.commit()
            return None

        connection.execute(
            """
            UPDATE builds
            SET status = 'running',
                started_at = CURRENT_TIMESTAMP
            WHERE id = ?
            """,
            (build["id"],),
        )

        connection.commit()

        return build

    finally:
        connection.close()


def complete_build(build_id, result):
    status = "passed" if result.passed else "failed"

    with get_connection() as connection:
        connection.execute(
            """
            UPDATE builds
            SET status = ?,
                output = ?,
                return_code = ?,
                finished_at = CURRENT_TIMESTAMP
            WHERE id = ?
            """,
            (
                status,
                result.output,
                result.return_code,
                build_id,
            ),
        )


def run_worker():
    print("CI worker started")

    while True:
        build = get_next_build()

        if build is None:
            time.sleep(2)
            continue

        print(
            f"Running build #{build['id']} "
            f"for commit {build['commit_sha']}"
        )

        try:
            result = run_build(
                build["commit_sha"]
            )

            complete_build(
                build["id"],
                result,
            )

        except Exception as error:
            with get_connection() as connection:
                connection.execute(
                    """
                    UPDATE builds
                    SET status = 'failed',
                        output = ?,
                        finished_at = CURRENT_TIMESTAMP
                    WHERE id = ?
                    """,
                    (
                        str(error),
                        build["id"],
                    ),
                )


if __name__ == "__main__":
    run_worker()

Run the worker from a Bash console:

python worker.py

You should see:

CI worker started

The worker waits until a build appears.

Once someone pushes code:

GitHub
    ↓
Webhook
    ↓
Build #12 queued

the worker discovers it:

Build #12
    ↓
running
    ↓
checkout commit
    ↓
pytest
    ↓
passed / failed

PythonAnywhere recommends moving long-running web work out of the request itself; persistent background workers can be run using its task facilities on accounts that support them. PythonAnywhere Help

For the purposes of this tutorial, keeping the worker running in a Bash console is enough to see the system working.

Viewing build results

One last thing would make our tiny CI system much more useful.

We need to be able to inspect our builds.

Add the following endpoint to the Flask application:

@app.get("/builds/<int:build_id>")
def get_build(build_id):
    with get_connection() as connection:
        build = connection.execute(
            """
            SELECT *
            FROM builds
            WHERE id = ?
            """,
            (build_id,),
        ).fetchone()

    if build is None:
        return {
            "error": "Build not found"
        }, 404

    return dict(build)

Now visiting:

https://YOUR_USERNAME.pythonanywhere.com/builds/12

might return:

{
  "id": 12,
  "repository": "username/project",
  "commit_sha": "a918de82f85e...",
  "status": "passed",
  "return_code": 0,
  "output": "3 passed in 0.12s",
  "created_at": "2026-09-28 04:20:10",
  "started_at": "2026-09-28 04:20:11",
  "finished_at": "2026-09-28 04:20:12"
}

Congratulations.

We have built a small but real event-driven CI system.

What did we actually build?

It is worth stepping back for a moment.

Our final architecture consists of five pieces.

┌─────────────────────┐
│ GitHub Repository   │
└──────────┬──────────┘
           │
           │ push
           ▼
┌─────────────────────┐
│ GitHub Webhook      │
└──────────┬──────────┘
           │
           │ signed HTTP request
           ▼
┌─────────────────────┐
│ Flask CI Server     │
└──────────┬──────────┘
           │
           │ queue build
           ▼
┌─────────────────────┐
│ SQLite Build Queue  │
└──────────┬──────────┘
           │
           │ claim job
           ▼
┌─────────────────────┐
│ CI Worker           │
│                     │
│ git checkout        │
│ pytest              │
└──────────┬──────────┘
           │
           │ result
           ▼
┌─────────────────────┐
│ Build Record        │
│                     │
│ passed / failed     │
│ logs                │
│ exit code           │
└─────────────────────┘

And if you look at that architecture carefully, the basic ideas should look familiar if you have ever used a modern CI service.

The implementations are obviously much more sophisticated, but the underlying loop is recognizable:

event
  ↓
job
  ↓
worker
  ↓
isolated execution
  ↓
result

That is the part I wanted to expose with this tutorial.

GitHub Actions can make CI look like this:

- run: pytest

And that abstraction is extremely useful.

But underneath it, something still has to notice that code changed, schedule some work, acquire the correct source code, execute a command, inspect its exit code and record the result.

We just built a tiny version of that something.

What our CI server is still missing

Our implementation is intentionally simple.

I would not expose this server to arbitrary repositories or use it as a production CI service in its current state.

A real CI platform would need to solve considerably harder problems, including:

• isolated build environments;
• concurrent builds;
• dependency caching;
• build cancellation;
• time and resource limits;
• secure secrets management;
• repository permissions;
• multiple branches;
• multiple test commands;
• build artifacts;
• log streaming;
• retries;
• worker failures;
• duplicate webhook deliveries;
• status reporting back to GitHub;
• and potentially hundreds or thousands of workers.

The biggest missing feature is isolation.

Right now every build runs on the same machine and uses the same local repository.

A more serious implementation would create a fresh environment for each build, perhaps using a container:

Build #41 ──► Container A
Build #42 ──► Container B
Build #43 ──► Container C

Each build could then execute independently without modifying the environment used by another build.

From there, we are beginning to wander into the architecture of actual CI platforms.

And that is probably a tutorial for another day.

Conclusion

We started with a surprisingly simple question:

What actually happens after I push code and a CI platform runs my tests?

To find out, we built our own small CI server using Python, Flask, PythonAnywhere and GitHub Webhooks.

Our final system can:

receive a GitHub push
        ↓
verify the webhook
        ↓
queue a build
        ↓
checkout the exact commit
        ↓
run automated tests
        ↓
capture the exit code and logs
        ↓
record PASS or FAIL

The resulting server is nowhere close to replacing GitHub Actions, CircleCI, Jenkins or any other production CI platform.

That was never the goal.

The goal was to remove some of the magic.

And once the magic disappears, Continuous Integration turns out to be built from a handful of concepts that show up throughout software engineering:

HTTP events, queues, workers, processes, exit codes and state.

Sometimes the best way to understand an abstraction is to build a tiny version of it yourself.