Setting up Uptime Kuma on an EC2 instance

TLDR: We needed a monitoring service to let us know when one of our applications went rogue, and Uptime Kuma fit the bill. Here I talk about how to self-host it on an AWS EC2 instance, and how to squeeze literally every aspect of that process into the “everything-as-code” paradigm. I share the templates, the config file and the scripts and won’t even ask you to buy me a coffee.

Uptime Kuma, for those who have not yet come across this open-source gem, is a self-hosted monitoring tool designed to track the availability of services, applications, and databases. It comes with a fairly straight-forward UX and graceful-looking charts. One can set it up to repeatedly hit endpoints and verify that the response is indeed what one expects.

Imagine a kid asking “are we there yet?” on an infinite loop, and you have the sense of what Kuma is doing. Of course, being non-sentient, neither Kuma nor the endpoint is going to get tired of the game, although given the pace of development in AI, I am not really sure how long that non-sentient premise will hold. To get a first-hand sense of what the tool does, you can visit their demo site.

As I might have mentioned in earlier blogs, at The Ninja Studio, we have many projects going simultaneously, and the services do have a way of surprising one every now and then. One key requirement then is that if any application or a component becomes unavailable, someone needs to raise the alarm, ring the bell, light up the bat-signal etc.

In the comics, it is Commissioner Jim Gordon’s job to light up the Bat Signal. On the web, however, it was decreed that Uptime Kuma should do the job. And to those of you who noticed that there is a web but no Spiderman, I salute your astute observation. I did some research, and a spider-signal does exist, but it is not nearly as well-respected by the fandom, so we’ll stick with the Bat.

If you head over to GitHub and visit the Uptime Kuma repo, you’ll find that Louis Lam has offered a no-fuss deployment protocol. You can set-up Kuma within a few minutes on your own machine, and then add the monitors as required.

So as part of the initial evaluation, that is what we did. We set up monitors and verified that they worked well for our use case. There is a wide variety of options, available through a drop-down menu, for the kind of monitors that can be set up. For instance, a simple monitor could just query an endpoint and be happy with a 200 status code response, or you could set up a keyword evaluation wherein Kuma looks for a keyword that is present in the response body. In any case, the actual set up of the monitor will depend upon what your own use-case looks like.

If the application endpoint behaves in a manner that is contrary to expectation, the monitors will spot it. But we do need the bat-signal to be actually sent out, so the next thing to do is to set up a notification channel. Once again, Kuma offers a wide range of integrations. I set up a slack channel and it was very simple to do this using the Kuma UI, which offers a testing button to make sure you are indeed getting the notifications.

So this quick setup of the monitor and notification channel completed proof-of-concept. However, as you can imagine, it is not desirable to keep Kuma running perpetually on my own machine. So I looked to the AWS EC2 instance, a natural choice as the compute engine since there is a lot of organizational experience with it. And the memes notwithstanding, a really tiny t4g.micro would not burn a hole in your pocket, especially amortized over all your projects.

In the spirit of everything-as-code, I wanted a CloudFormation template that would do everything. But before conjuring one up, a complete, grand vision was required. What were the requirements? Here, admittedly, I got a little ambitious.

My shopping list

consisted of the following elements.

1. An EC2 instance that would run the Uptime Kuma within a Docker container.
2. A Caddy container, so that Kuma dashboards and status pages could be viewed securely by team members on a dedicated URL.
3. An Elastic IP address, since I don’t want to setup DNS again, in case something bad happened and the EC2 instance required a restart.
4. A way to avoid the early setup steps. I mean, if I have to go to the URL and manually create my username, password and choose the Kuma database by, god forbid, clicking on things, then that’s not really everything as code now, is it?
5. A way to make sure that if my colleagues add monitors on their own, their hard work can be preserved and restored in case of any untoward events in Gotham City. That is, I need a disaster management plan.
6. Periodic evaluation of the performance of the EC2. I mean, I am going to go with perhaps the cheapest instance I can find, so I need to be sure that the poor thing is not overworked. Thankfully, I have a ready-to-use, drop-in solution for that. Its detailed in my excellent earlier blog, in case you are interested.

After a few days of hard work, debugging and intense, soulful conversations with Gemini and Claude, I was able to get what I wanted. Everything tied up neatly in folder that I called the uptime-kuma-goodie-bag. Here’s what it looks like on my system

As you can see, there are a mere six files that will do all the heavy lifting. Let’s get into each one for a high-level overview, strictly in order of ease of understanding.

1. uptime-kuma-secret-template.yaml

This CloudFormation template creates an AWS Secret. There are four key-value pairs in this secret, namely

1. webhook: placeholder
2. channel: placeholder
3. username: placeholder
4. password: placeholder

The point of the placeholders being, the secret is created with these and the user can visit the AWS UI and edit the values. One can always use the actual values instead of placeholders, but that exposes the password, which is generally a bad idea.

The webhook and channel name refer to Slack, as the scripts have been written specifically with Slack in mind. The username and password refer to the first user (the admin user) on Kuma, the person who can create new monitors etc, and without which Kuma will simply not move forward.

The scripts (kuma-setup.py and kuma-backup.py) can retrieve this information from the AWS SecretsManager and do their respective things.

2. kuma-config.json

This is a basic config file that contains information to set up a few, non-critical monitors. When I started on this project, my intention was to have information on all monitors stored as this “easy-to-read” json file. However, forays into the real world revealed that this would be problematic on two accounts.

First, I would need to figure out what to add to this json and then have my script read from there and create the desired monitor. This is easier said than done, and believe me, I tried. Almost like Batman trying to talk sense into Robin, and we all know how that ends. POWW !! KABOOM !!

Second, each monitor would need to be a complete, multi-element object within this config file. Imagine dealing with umpteen monitors, each of which is specific to the individual user. The lofty ideal of simplicity would be buried in a heap of json lego-bricks, if I were to take this proposal to prod.

Waste not, want not, they say, and so here I repurposed the config file and give it a new reason to live — its Ikigai.

Now, the config file is set up to create three basic monitors. The idea of doing that is, of course, to have someone at the door when we first knock on the Kuma UI. I mean, why walk into a barren landscape when we can have it pre-populated. Also, we can test the integrations really quickly, because one of the monitors is designed to fail and send that alert message to the designated Slack channel.

Of course, if you prefer, you can always pre-populate with your own monitors and that’ll be just fine.

3. uptime-kuma-s3.yaml

This is a CloudFormation template that performs the simple task of creating an S3 bucket. The bucket is where the backups will be stored. It is a versioned bucket with a lifecycle policy that will retain information for 30 days and no more.

4. kuma-setup.py

This is the engine of innovation, right here. This script, on first boot, pulls Slack webhook details and initial admin credentials from AWS Secrets Manager, injects configuration by directly writing to the SQLite database, integrates a global Slack notification channel that Kuma will use, and then loops through the kuma-config.json file to create all those monitors — all without a human ever touching the Kuma interface. I get breathless just saying all that!

Get Saurabh’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The script runs in an automated mode as part of the EC2 userdata, but also includes an interactive menu for manual execution and troubleshooting. In the manual mode, which can be run anytime you happen to be casually browsing your EC2 instance, the script offers multiple options.

Operators can manually trigger a configuration injection, switch between restoring from S3 or the config file, kick off an on-demand backup, or invoke a full “nuclear reset” that wipes Kuma’s data directory and recreates the container from zero. Yes, that great power is given to you, and I would definitely have put in the Spiderman quote here, except our theme today is The Bat.

The script is resilient by design. Before doing any fresh setup, it checks whether a valid database already exists locally on the EC2, and if not, it scans S3 version history for the most recent uncorrupted backup to restore from. This means that if the EC2 instance is replaced or re-provisioned, the monitors and any user-defined changes that were backed up will be restored automatically rather than rebuilt from scratch.

Backup validation isn’t merely a simple “does the file exist” test— it actually connects to each candidate database and checks for the presence of required tables, skipping over any corrupt versions. And you’ll surely agree, corruption is one of the bigger issues we face as a society.

5. kuma-backup.py

kuma-backup.py has a mandate to keep a recoverable copy of the Uptime Kuma database in S3. Rather than doing a naive file copy — which risks capturing the database mid-write — it uses SQLite's VACUUM INTO command to produce a clean, consistent snapshot to a temp file before uploading. This means the backup in S3 always represents a valid, uncorrupted state of the database, which is exactly what kuma-setup.py script relies upon when scanning version history for a viable restore point.

The script is designed to run on a daily schedule via a systemd timer, which it installs and configures itself when run without arguments. This script is also invoked by kuma-setup.py's manual backup option which you can run anytime. I figured that would be a good idea in the early, enthusiastic days of setting up Kuma when one can be expected to explore things and set up all sorts of new monitors/groups/notifications and a bunch of other stuff that Kuma offers.

So after having spent the better part of the day setting those up, if you are happy with the outcome, you can trigger a manual backup. Later on, when the novelty wears off and your colleagues holler for your attention on other, presumably more pressing concerns, you can leave your EC2 in peace and backups will still happen once a day.

If anything goes wrong — missing database file, failed S3 upload, or otherwise — the script will send up the Bat-Signal — a formatted Slack alert directly to the same webhook used by Kuma's monitoring notifications, so failures don't go unnoticed even if Kuma itself is the thing that's broken.

6. uptime-kuma-ec2-template.yaml

And finally, we arrive at the coordinator of everything, the Alfred Thaddeus Crane Pennyworth of our setup. If that long name took you by surprise, you are in good company. That is Batman’s butler’s complete name, as per Wikipedia. Who knew, huh.

Alfred is the primary CloudFormation template. We harness the power of userdata in the EC2 segment to create the Docker resources for both Uptime Kuma and Caddy. The Python scripts and the config file that are needed for Kuma setup and backup, are copied from the public GitHub repo to the EC2.

Well, actually, and I don’t mean to nitpick here, but actually the EC2 userdata uses the GitHub Contents API to fetch download URLs and then curls each file individually, so that’s not just a mere copying task. And we keep it dynamic, so should there be any updates or more files in the future, the userdata is robust to that change.

This template also creates and associates an elastic IP address with the EC2 instance. Users will need to specify some parameters, one of which is the domain name where one intends to access the Uptime Kuma dashboard. Once the template starts provisioning resources, one must visit the DNS service and set up an A record that points to the elastic IP address. The value of the IP address is available in the outputs section of this CloudFormation template.

The EC2 userdata runs a loop that waits for this DNS resolution to occur, before it brings up the containers. This is to avoid hitting Caddy’s request limits for LetsEncrypt certificates. I opted for the loop rather than the staging option provided by Caddy, as I intend this whole setup to be production-grade.

Alfred also quietly handles the house keys — a Lambda function generates the EC2 key pair and stores it safely in Secrets Manager, so you never have to manage a .pem file manually. Just in case you want to connect to the EC2 from your local CLI.

Never let it be said that Dr Saurabh doesn’t overengineer things.

Ok, I’m sold, what do I do now?

Good, glad to see that you’ve made it this far. Now is the time to reap some rewards. Here’s what you need to do.

Steps in sequence

1. Check that you have what it takes — a heart of gold, an AWS account where you can create resources, a domain name that you can use for the Uptime Kuma monitors and a Slack channel webhook where you can send messages. If you have actual projects to monitor, well, that’s the icing on the cake.
2. Download the Kuma Goodie Bag folder contents (actually, you only need to get the three CloudFormation templates) to your local machine — this lets you setup the parameters as default values, all ready to load into the AWS Cloud.
3. Supply your own parameters for each of the templates. Each parameter has a brief description that explains what it is for. However, there are three specific parameters that must retain the same value across the three templates, because they are used in naming resources. The values themselves don’t matter, just that they are the same for all three templates. These parameters are — ProjectName, Disambiguation and Environment. In case the S3 template fails because the S3 bucket name was not unique, just change the Disambiguation parameter and take another swipe at it.
4. In several places, I have provided what I believe to be sensible defaults, but hey, change whatever you need to, keeping the above guidelines in mind. As current world affairs show, what is sensible to one may be mind-bogglingly ridiculous to another. Anyway, social commentary aside, once you have made changes to the parameters and saved the files, you can move ahead to the next step.
5. Log in to AWS and head for the clouds. I mean, the CloudFormation segment. You need to create three stacks here, using your templates.
6. Use the sequence and wait for one stack to complete before launching the next.
   i. uptime-kuma-s3-template.yaml
   ii. uptime-kuma-secret-template.yaml — after this stack is completed, visit the AWS SecretsManager and update the secret values, in case you have used placeholders.
   iii. uptime-kuma-ec2-template.yaml. Remember, this stack involves an EC2 userdata loop. So it will not complete without your help. Once the Elastic IP address becomes available, and this will happen quite early in the stack events, you need to visit your DNS provider and point the domain name you have selected, to this elastic IP address. You need to set up a DNS-only A-record that points to the public IPv4 address. Once you do this, the EC2 userdata loop will break and your CloudFormation stack will be completed.

After all three stacks successfully created, you can fire up the batmobile and head to the domain, where the Kuma shall greet you. The initial set up will have been completed as per the config file. Your Slack channel should have received the message about the monitor that succeeded in its failure.

A small gotcha here — even after the final CloudFormation stack shows as complete, it takes a little bit of time before the Uptime Kuma dashboard actually becomes available. So if you don’t see it right away, perhaps just wait a minute or two.

Now you can configure other monitors, delete the default ones and frolic to your hearts content. You have the option of connecting with the EC2 instance and triggering a manual backup, or you can be lazy about it as it will happen after a while anyway.

As the sun sets over Gotham City, I am glad to see that you are still here, all the way at the end of this blog. As a token of my appreciation, here’s some references for you to dig into.

Thank you

— Dr Saurabh Sawhney

References and resources

1. The GitHub repo for uptime-kuma https://github.com/louislam/uptime-kuma
2. A demo to see if you like the monitors, before you commit to the tech-stack. https://demo.kuma.pet/start-demo
3. My public GitHub repo where you can find the code and possibly updates later on, though don’t count on it — https://github.com/DrSaurabh/code-blooded/tree/main/uptime-kuma-goodie-bag. How about giving me a star or two here?
4. CloudFormation docs: https://docs.aws.amazon.com/cloudformation/
5. EC2 UserData: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
6. IMDSv2 (used in both Python scripts for region detection): https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html
7. Caddy automatic HTTPS (explains why the DNS guard loop is necessary): https://caddyserver.com/docs/automatic-https
8. Let’s Encrypt- a free, automated, and open Certificate Authority: https://letsencrypt.org/
9. VACUUM INTO (the technique used in kuma-backup.py): https://www.sqlite.org/lang_vacuum.html
10. Systemd timers (used for daily scheduling): https://www.freedesktop.org/software/systemd/man/systemd.timer.html
11. The writing is mine, but I must acknowledge my AI buddies who have generously provided the artwork and more — Gemini and Claude.