Three Ways to experiment eBPF on Raspberry Pi: bcc (Python), libbpf-rs (Rust), Aya (Rust)

Background

If you just wanted to have a quick ‘getting started’ knowledge about eBPF and you happened to have a spare Raspberry Pi (4 and above), exploring different ways of interacting with eBPF on the Pi is a perfect hands-on way of getting a preliminary understanding of eBPF. While there are so many online resource talking about what eBPF is and the benefits of utilizing eBPF[1], I will let those resources serve your curiosity. This article mainly tries to offer you the practical steps to setup your eBPF communication channels, so that you can further explore this new cool technology.

The Experiments

Initially from some online search and study, the bcc framework was the first one I was ‘recommended’, which is relatively straightforward. After getting the ‘hello-world-ebpf’ to run in the bcc approach, I wonder if there is a Rust binding to the bcc framework instead of writing Python. From some crate search/hunting, I found out that there are at least one (i.e., bcc-rs), but most of them are no longer active. Instead, I found another interesting one, libbpf-rs, and learned that the eBPF ecosystem is trending toward eBPF CO-RE (Compile Once — Run Everywhere), so libbpf-rs with no doubt became my second experiment target. However, libbpf-rs is still a binding crate that depends on libbpf, meaning that you will still have to write some C syntax like BPF code. Nowadays, even the Linux kernel already started using Rust to implement some kernel subsystem, and given the popularity of eBPF, it does not make sense that there is no Rust native solution/framework to eBPF. This questioning led me to find Aya framework. This is a pure Rust implementation that relies on Linux system calls that directly communicating to the kernel eBPF subsystem. It still relies on LLVM and its Clang compiler at compile time, but the Rust compiler can directly transform the Rust code to eBPF bytecodes via it BPF target support[2].

eBPF via bcc

As mentioned above, this approach is relatively easy to setup and run the hello-world-ebpf. After flashing the my Pi400 with the latest bookworm release (kernel version 6.1.x), I would only need to install a couple dependencies to bring up the hello-world-ebpf example provided by Liz Rice in her ‘Learning eBPF’ book [3]:

pip install bcc
pip install numba
pip install pytest
apt-get install bpfcc-tools
apt-get install raspberrypi-kernel-headers

eBPF via libbpf-rs

This approach requires us to recompile the kernel with CONFIG_DEBUG_INFO_BTF=y. While Raspberry Pi has some good official documentation of re-compiling the kernel locally on the Pi, from my experiment, I noticed that there were a few traps that we could fall into and scratched our heads with no direct answers from google search or stackoverflow (maybe ChatGPT has it? TL;DR), so I here document them for you and future myself.

The first tip is that how could we find out if the running OS kernel was compiled with BPF subsystem enabled?

sudo modprobe configs
zcat /proc/config.gz | less

After getting the current kernel configuration into the less reader, we can do /bpf pattern search (i.e., type /bpf in the less interface) to confirm the BPF subsystem was compiled and enabled. If configuration like CONFIG_BPF and CONFIG_HAVE_EBPF_JIT are ‘y’, then the running kernel has the BPF subsystem enabled. Then we only need to recompile the kernel with CONFIG_DEBUG_INFO_BTF=y . There is no direct answer on how to do it in Raspberry Pi OS, there are the key takeaways:

1. After getting the default .config for your Pi, you would need to manually edit the .config to add CONFIG_DEBUG_INFO_BTF=y.
2. Follow the Pi kernel re-compile doc to install menuconfig and enter its GUI, then search for DEBUG_INFO_BTF:

3. The search result will suggest you the exact path of the DEBUG_INFO_BTF configuration. Follow that to get to ‘Compile-time checks and compiler options’, and you won’t see the ‘Generate BTF type info’ until you unselect the ‘Disable debug information’:

4. Upon saving this .config, we can move forward with the official doc to re-compile the Pi kernel.

Get Riyao Lin’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

5. After rebooting, we should see the BTF vmlinux file in /sys/kernel/btf/vmlinux .

6. To verify you now can build eBPF app via libbpf-rs, you could checkout therunqslower example [4].

eBPF via Aya

Aya’s Development Environment doc[5] provides some decent guideline to setup, but for Raspberry Pi environment, we need to pay attention to the llvm installation and configuration:

1. if we use the https://apt.llvm.org/ to install llvm, the polly lib was not included, hence we need to install it separately via sudo apt install libpolly-17-dev (assuming you install the major version 17 llvm).
2. after installation llvm, we would need to add it to PATH (e.g., export PATH=/usr/lib/llvm-17/bin:$PATH), otherwise the Aya cannot locate llvm during compiling llvm-sys, and gives us the ‘error: no suitable version of llvm was found system-wide or pointed to by llvm_sys_*_prefix’.
3. Raspberry Pi by default did not install zstd lib, hence we would need to manually install it to make Aya work: sudo apt install libzstd-dev .
4. Run the Aya’s xdp-log example [6] to verify this approach works:

// upon building the ebpf code and the example crate

RUST_LOG=debug cargo xtask run

Reference

[1]. https://ebpf.io/what-is-ebpf/

[2]. https://github.com/rust-lang/rust/pull/79608

[3]. https://github.com/lizrice/learning-ebpf/tree/main/chapter2

[4]. https://github.com/libbpf/libbpf-rs/tree/master/examples/runqslower

[5]. https://aya-rs.dev/book/start/development/

[6]. https://github.com/aya-rs/book/tree/main/examples/xdp-log