Grafana, Prometheus & Node-Exporter | Setup Guide

Grafana open source software enables you to query, visualize, alert on, and explore your metrics, logs, and traces wherever they are stored with beautiful dashboards.

Prometheus is a core technology for monitoring and observability of systems. Understanding Prometheus basics, why it’s valuable for system observability, and how users use it in practice will both help you better understand it and help you use Grafana.

Node Exporter is a tool used with Prometheus to collect hardware and OS-level metrics from a Linux server, such as CPU usage, memory usage, disk I/O, and network statistics. It runs as an agent on the server and exposes these metrics on an HTTP endpoint, which Prometheus can scrape at regular intervals. These metrics are then visualized in Grafana, allowing for monitoring and alerting on the server’s health and performance.

Before interacting with any project, please DYOR about the project:
1) DYOR guide
2) DYOR practices

Server requirements

Such specs should be enough: 2CPU, 2RAM and 20GB SSD.

Suitable servers:
- Hetzner — CX22, CPX11, CAX11
- Cityhost — KVM35

Prerequisite

Node-exporter should be installed on servers that require monitoring.
Grafana & Prometheus will be installed on a separate server to run our monitoring system.

If you have only one server to monitor, you can install everything on that server and monitor it locally.

Update the local package repository database & upgrade installed packages to their latest versions based on the updated package database:

sudo apt-get update && sudo apt-get upgrade -y

Node-Exporter

1) Install Node-Exporter:

wget https://github.com/prometheus/node_exporter/releases/download/v1.8.2/node_exporter-1.8.2.linux-amd64.tar.gz && \
tar xvf node_exporter-1.8.2.linux-amd64.tar.gz && \
rm node_exporter-1.8.2.linux-amd64.tar.gz && \
sudo mv node_exporter-1.8.2.linux-amd64 node_exporter && \
chmod +x $HOME/node_exporter/node_exporter && \
mv $HOME/node_exporter/node_exporter /usr/bin && \
rm -Rvf $HOME/node_exporter/ && \
node_exporter --version

2) Create node_exporterd service file:

tee /etc/systemd/system/node_exporterd.service > /dev/null <<EOF
[Unit]
Description=Node Exporter
After=network.target

[Service]
User=$USER
ExecStart=/usr/bin/node_exporter
Restart=always
RestartSec=3
LimitNOFILE=65535

[Install]
WantedBy=multi-user.target
EOF

3) Reload daemon, enable node_exporterd, start it and check status:

systemctl daemon-reload && \
systemctl enable node_exporterd && \
systemctl start node_exporterd && \
systemctl status node_exporterd

If everything went ok, should look like this:

4) Check logs:

journalctl -u node_exporterd -f -o cat

5) Check whether metrics are provided:

curl 'localhost:9100/metrics'

Looks like this:

6) Go to a browser and check metrics there:

http://<YOUR_IP>:9100

7) Click on “Metrics”:

You should see a lot of metrics from your server:

Prometheus

1) Install Prometheus:

wget https://github.com/prometheus/prometheus/releases/download/v2.55.0/prometheus-2.55.0.linux-amd64.tar.gz && \
tar xvf prometheus-2.55.0.linux-amd64.tar.gz && \
chmod +x prometheus-2.55.0.linux-amd64/prometheus && \
chmod +x prometheus-2.55.0.linux-amd64/promtool && \
cp prometheus-2.55.0.linux-amd64/prometheus /usr/local/bin/ && \
cp prometheus-2.55.0.linux-amd64/promtool /usr/local/bin/ && \
cp -r prometheus-2.55.0.linux-amd64/consoles /etc/prometheus && \
cp -r prometheus-2.55.0.linux-amd64/console_libraries /etc/prometheus && \
cp prometheus-2.55.0.linux-amd64/prometheus.yml /etc/prometheus/ && \
rm -rf prometheus-2.55.0.linux-amd64.tar.gz prometheus-2.55.0.linux-amd64 && \
prometheus --version && \
promtool --version

2) Open prometheus.yml file:

nano /etc/prometheus/prometheus.yml

Here you can add as much servers as you want.
Mind, that this file is in YAML format and has strict rules to work properly.

3) Create prometheusd service file:

tee /etc/systemd/system/prometheusd.service > /dev/null <<EOF
[Unit]
Description=Prometheus
After=network-online.target

[Service]
User=$USER
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries \
    --web.listen-address=:9090

[Install]
WantedBy=multi-user.target
EOF

4) Reload daemon, enable prometheusd, start it and check status:

systemctl daemon-reload && \
systemctl enable prometheusd && \
systemctl start prometheusd && \
systemctl status prometheusd

If everything went ok, should look like this:

5) Check logs:

journalctl -u prometheusd -f -o cat

6) Check whether metrics are provided:

curl 'localhost:9090/metrics'

Looks like this:

6) Go to a browser and use IP of a server with Prometheus installed:

http://<YOUR_IP>:9090

7) Click on “Status” > “Targets”.

Prometheus displays statistics for your local server where it’s running, as well as for all additional servers you’ve added to the prometheus.yml file.

Grafana

Grafana should be installed on the same server as Prometheus. It will visualize the metrics.

Get crptcpchk’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

1) Install Grafana.
Latest release can be found on their site.

sudo apt-get install -y adduser libfontconfig1 musl && \
wget https://dl.grafana.com/enterprise/release/grafana-enterprise_11.3.0_amd64.deb && \
sudo dpkg -i grafana-enterprise_11.3.0_amd64.deb

2) Reload daemon, enable grafana-server, start it and check status:

systemctl daemon-reload && \
systemctl enable grafana-server && \
systemctl start grafana-server && \
systemctl status grafana-server

If everything went ok, should look like this:

5) Check logs:

journalctl -u grafana-server -f -o cat

6) Go to a browser and use IP of a server with Grafana installed:

http://<YOUR_IP>:3000

Login: admin
Password: admin

7) Change password for your own.
DON’T SKIP THIS STEP.

8) Click on “Configurations” > “Data sources”.

9) “Add data source”

10) Choose Prometheus.

11) Add your Prometheus IP.
If Prometheus is on the same server as Grafana, leave it as is.

12) “Save & test”

13) Click on “Dashboards” > “Import”.

14) Choose which dashboard you want to use by adding URL or ID.
Full list of dashboards can be found here.

Classic Node-Exporter dashboard ID 1860.

15) Choose name for a dashboard and add Prometheus.
“Import”.

Now you can monitor your servers.

16) Open your server with prometheus.
16.1) Open prometheus.yml file:

nano /etc/prometheus/prometheus.yml

16.2) Add you servers to monitors them.
Should look like this:

targets — IP addresses of your servers
label — name that will be displayed on Grafan.

17) Reload daemon and restart prometheusd:

systemctl daemon-reload && \
systemctl restart prometheusd

Delete

Delete Node-Exporter:

systemctl stop node_exporterd
systemctl disable node_exporterd
systemctl daemon-reload
rm /etc/systemd/system/node_exporterd.service

Delete Prometheus:

systemctl stop prometheusd
systemctl disable prometheusd
systemctl daemon-reload
rm /etc/systemd/system/prometheusd.service

Delete Grafana:

systemctl stop grafana-server
systemctl disable grafana-server
systemctl daemon-reload

Links

Follow for more guides on nodes, drops and other useful stuff.

psi.crypto Discord: https://discord.gg/9kvtEQwtqJ

X: https://twitter.com/the01100011

Telegram: https://t.me/psicrypto

Nostr/Farcaster: @crptcpchk

HackenDAO: https://discord.gg/hacken

RawBox: https://twitter.com/rawbox_tech