Intro

I had always been curious about having a smart home, but I never really had a reason to build one. That changed when I unexpectedly got a Tapo smart socket and used it to control a standing lamp in the living room.
After that came a Dyson fan and a couple of Xiaomi devices. Before long, I had a small zoo of smart gadgets, each with its own app. Managing them all from separate apps felt unnecessarily complicated, so I started thinking that voice control might be a simpler way to bring everything together.

I had a Raspberry Pi 5 running Ubuntu but no Home Assistant yet, so I installed it and decided to see how far I could get with a local voice setup.

The target setup looked simple enough:

USB conference speakerphone
    ↓
Raspberry Pi 5
    ↓
local wake word
    ↓
local speech-to-text
    ↓
Home Assistant Assist
    ↓
local text-to-speech

I also had a Raspberry Pi AI HAT+ 2 installed, which made the idea even more tempting: maybe some of the speech workload could eventually move to the Hailo accelerator instead of burning Pi CPU. But more about it later.

For the voice hardware, I used an Anker PowerConf S330. It is a USB conference speakerphone with built-in microphones and a speaker, which sounds almost perfect for this kind of project. No battery, Bluetooth or proprietary smart-speaker firmware, just USB audio plugged into a Linux box.

On paper, this looked like a setup that should work. And in many ways, it did. Home Assistant managed the devices, TTS worked, the S330 showed up as normal Linux audio, and the whole thing could be wired into a local voice pipeline. But testing it from the couch, about 2.5 meters away, quickly showed the gap between a demo and something I would actually use every day. The wake word often worked, but the rest of the interaction was unreliable: commands were misheard, interpreted incorrectly, or answered much too late. A simple “turn the lights off” was enough to make the whole setup feel inconsistent rather than useful.

In this story, I’ll share what I tried, what worked, what failed, and why I eventually returned the speakerphone and parked the idea until the local voice ecosystem gets better. This is not a happy “here is how to build your own Alexa replacement” guide. I didn’t manage to build one.

The “voice pipeline” project

Once the devices were controllable from Home Assistant, the real project became the voice pipeline. I used OHF Linux Voice Assistant for the satellite side. It connects to Home Assistant through the ESPHome-compatible API and can use Home Assistant’s Assist pipeline without needing Home Assistant OS or the add-on store.

The basic audio path looked like this:

Anker PowerConf S330
  ↓
PipeWire
  ↓
Linux Voice Assistant
  ↓
Home Assistant Assist
  ↓
Speech-to-text (STT)
  ↓
Intent and conversation handling
  ↓
Piper text-to-speech (TTS)
  ↓
Linux Voice Assistant
  ↓
PipeWire
  ↓
Anker PowerConf S330

Because I was running Home Assistant Container on Ubuntu, I had to treat the Pi more like a regular Linux machine than a dedicated appliance. Audio was handled by PipeWire in my normal Linux user’s session, while Home Assistant and the Wyoming speech services ran independently in Docker containers. The S330 itself was not difficult to expose to Linux. It appeared as a normal USB audio device with both microphone input and speaker output.

The first pleasant surprise was the wake word. I used “Okay Nabu”, and it worked better than I expected. Not perfectly, but reliably enough. Once the assistant woke up, though, the whole experience depended on speech-to-text. That was where the project stopped being a setup exercise and became a debugging exercise.

My first general-purpose STT backend was Faster Whisper on the Pi 5. I started with base-int8, which was accurate enough to make the project feel possible. At the same time, however, it was frequently slow, so short commands could take up to a minute to receive a response. When that happened, the assistant just sat there processing, with no way to terminate it prematurely.

Yet I wasn’t ready to give up, so I tried the more obvious Home Assistant-native option: Speech-to-Phrase. Conceptually, it is exactly what a smart-home assistant should use. Most of the time, I was not dictating free-form text. I was saying short commands against a known set of entities:

turn the lights on
turn the lights off
turn on the white fan
turn off the Dyson

A constrained recognizer should have a much easier job than a general-purpose speech model. It knows the grammar, it knows the device names, and it does not need to understand arbitrary English.

In practice, Speech-to-Phrase was promising, but not enough by itself. It was fast, and when it worked, it felt much closer to the kind of response time I wanted. But around 50% of the time, it simply returned no transcript, which meant the command was not recognized. When speech-to-text returned no transcript, the Assist pipeline terminated with stt-no-text-recognized before reaching the TTS stage. Linux Voice Assistant exposed this as a pipeline_error, for which its peripheral API recommends a visual error indication, but there was no spoken “sorry, I didn’t understand that” response. In practice, a failed command could therefore end in silence unless the satellite hardware provided some other feedback. Of course, it might simply have been bad timing for this project, as I’m not alone experiencing this.

At this point, I could confidently say that the pipeline was technically correct and that all services were running properly: the wake word worked, audio flowed through the system, Home Assistant could receive the transcript, match an intent, run the action, and speak back through Piper.

In day-to-day usage, however, the solution’s reliability and speed were below the minimum usability threshold. From the couch, or simply while passing by, it managed fewer than 50% of my commands. A deeper debugging session was therefore needed to answer a simple question: why was this thing failing on commands that were only three or four words long?

Debugging the voice assistant

At first, I assumed the microphone was simply not good enough. The S330 was sitting on the TV stand roughly 2.5 meters away from the couch. Conference speakerphones are designed for meetings, not for replacing a Google Home in a living room, so poor microphone quality seemed like the obvious explanation. So, I ran a few tests standing right next to the speaker, yet it still struggled with commands that should have been trivial.
And the failures also weren’t even that random. For some reason, commands containing “off” consistently seemed harder than commands containing “on”. Turn the lights on would mostly work, but Turn the lights off, would frequently fail.

I tried changing the wording:

Turn off the lights
Lights off
Turn the white fan off
Turn off the white fan

No obvious pattern emerged, sometimes one variation worked while another didn’t, only for the result to flip a few minutes later.

At this point I stopped guessing and started collecting evidence. Home Assistant can save the audio captured by the Assist pipeline, so instead of repeatedly speaking new commands and trying to judge them by ear, I enabled pipeline debugging and began analyzing the recorded WAV files. That immediately weakened several of my earlier theories. The recordings were clean — very little background noise, healthy signal levels and, most importantly, the supposedly troublesome words were actually there: the final “off” was audible and the microphone wasn’t obviously cutting it out. To a human listener, the commands were completely understandable.
It made it increasingly difficult to blame the S330 mic quality.

Speech-to-Phrase, which had frequently returned no transcript during normal use, correctly recognized six of the seven test recordings and usually finished in less than two seconds. Whisper also recognized the recordings, giving me a useful comparison with a more general-purpose transcription engine.

At first that seemed almost contradictory. If the recorded audio was good enough for both Whisper and Speech-to-Phrase, why did the same system feel so unreliable when used normally?

I’m not an expert in either system — far from it, really — but as a technical person, I have a few guesses about what might be happening here.

• One possible explanation is that live recognition introduces timing and streaming variables that are not present when replaying a complete recording. The system has to decide when speech starts and stops, and a saved recording may give the recognizer a cleaner or more consistent input.
• Another possibility is that Speech-to-Phrase is less forgiving than Whisper. Whisper is a general transcription service that tries to reproduce whatever words it hears, while Speech-to-Phrase is designed to recognize commands that Home Assistant already knows. If the result is close enough to one of those commands, Speech-to-Phrase succeeds; if not, it returns nothing at all. Whisper may still produce a rough or imperfect transcript where Speech-to-Phrase rejects the same audio.
• That could explain why the system sometimes felt inconsistent. Small differences in timing, pronunciation or the way the audio was captured may have been enough to turn an otherwise understandable command into an empty response from Speech-to-Phrase, even though Whisper could still produce some text — although it would take considerably longer.

I couldn’t prove which of these factors mattered most, but the replay tests suggested that the live capture process was at least part of the problem. However, the tests with both Speech-to-Phrase and Whisper showed that the microphone was producing understandable recordings, so I could at least rule out hardware issues. Something about the complete live-recording process was making the results unreliable.

Before writing it all off I decided to explore all currently available models in more depth. That turned into an interesting comparison between Whisper, Speech-to-Phrase, Qwen3-ASR, NVIDIA Parakeet and even Hailo-accelerated Whisper.

Trying different STT models

The next step was to try different speech-to-text backends and see whether any of them made the system usable in real life.

Speech-to-Phrase

Speech-to-Phrase was the least interesting backend to benchmark because its strengths and limitations were already visible from the design. It was not trying to be a general dictation engine but rather a “mapper” of a small set of spoken phrases onto Home Assistant intents.

When I replayed the same seven recorded commands through it, it recognized six correctly and usually finished in roughly 1.5–2 seconds. That result was promising enough that I built a small STT fallback service around it.

The idea was simple:

Speech-to-Phrase
      ↓
transcript returned? → use it
      ↓ no
Faster Whisper

Speech-to-Phrase would handle the common Home Assistant commands quickly, while Whisper would act as a safety net whenever it failed to recognize anything.

The experiment worked technically. When Speech-to-Phrase succeeded, the fallback layer added almost no measurable overhead. When it returned an empty transcript, the same recording was automatically sent to Whisper instead.

But the design exposed another limitation: fallback could only help when Speech-to-Phrase returned nothing. If it confidently returned the wrong words, there was no reliable way to know that the result should be retried with another model. And more importantly, the good replay benchmark never materialized into equally reliable live use.

Faster Whisper

Faster Whisper was the first general-purpose STT backend I used. It was easy enough to run through Wyoming, and the quality was good enough to make the project feel realistic. However, latency was the problem.

With base-int8, the recognition was often acceptable, but some commands took more than 10 seconds to process. Occasionally, one bad input would send the decoder into a very long run, and the assistant would just sit there doing nothing useful. And for a voice assistant that makes a big difference.
Therefore I decided to try smaller Whisper models such as tiny-int8 and tiny.en, hoping that the latency would improve enough to make the trade-off worthwhile. And the smaller models were indeed faster, but the transcripts became too unreliable for device control. The recognized text was sometimes quite funny:

Dice them on
The reason of
Wear the white fan on
Turn the wet fan on
Turn the lights home

But that is the sort of failure that can make voice control dangerous rather than useful. If the model confuses a device name, drops the action, or changes “off” into something else, Home Assistant will end up doing the wrong thing.

So Whisper worked, but it did not feel appliance-grade on a Raspberry Pi 5.

Qwen3-ASR

Then I tried Qwen3-ASR through the newer Wyoming Faster Whisper support with rhasspy/qwen3-asr-0.6b-onnx-int4 model.

On the Pi 5, it was heavier than Speech-to-Phrase. During inference, it used roughly four CPU cores and around 2.5 GB of RAM. That is not outrageous on a 16 GB Pi 5, but it is still a noticeable amount of compute for a voice assistant.

And the fixed-WAV results were the best I had seen from a general STT model. For the same saved recordings, Qwen produced exactly the transcripts I would expect, so it was genuinely promising.
It also had another interesting feature: Home Assistant vocabulary biasing. In theory, that should help with exactly the kind of words that matter in a smart-home setup: entity names, room names, and aliases like white fan, Dyson, living room, and so on.

However, in live testing, it remained as unreliable as the previous two models. It could understand carefully separated speech, but natural commands were still hit or miss. For example, saying something like Turn lights on could fail, while a much more artificial version TURN … LIGHTS … ON was more likely to work.

At that point, the setup had crossed an important line for me. If I have to speak to my own voice assistant as though I am labelling a training dataset, the project has already failed as a living-room device.

Qwen3-ASR was still the best general-purpose CPU result I found, but it did not close the gap between “good benchmark” and “usable appliance”.

Parakeet through ONNX-ASR

The strangest result came from Parakeet.

I tested nemo-parakeet-tdt-0.6b-v2 through onnx-asr, and on the fixed WAV files it looked excellent. After warm-up, processing time was around 1.2 seconds, and most of the saved recordings were recognized correctly. For a moment, this looked like the best option: faster than Qwen, more accurate than tiny Whisper, and good enough on the exact command recordings I cared about. Until I tried it live. The transcripts immediately fell apart:

The lights of
Dice and wall.
Uh fan of
Oh
I smoke.

That was while I was saying variations of commands like:

Turn the lights off
Turn the fan off.

In real use, Parakeet was probably the worst backend I tried.

That result was useful, though, because it made the benchmark problem impossible to ignore. The same model could look excellent on a tiny set of saved WAV files and completely unusable during normal interaction.

After trying all of these, the results were pretty clear:

Backend              | Good part                                        | Bad part
---------------------|--------------------------------------------------|-----------------------------------------
Speech-to-Phrase     | Fast and well matched to Home Assistant commands | Constrained, and still unreliable live
Faster Whisper tiny  | Lightweight                                      | Too inaccurate
Faster Whisper base  | Better transcription                             | Too slow and occasionally gets stuck
Qwen3-ASR 0.6B INT4  | Best general CPU result on saved recordings      | Still unreliable with natural live speech
Parakeet / ONNX-ASR  | Very fast and excellent on saved WAVs            | Terrible live behaviour in my setup

The frustrating part was that nothing failed cleanly: the microphone, pipeline, wake word, Piper, and Home Assistant all worked well enough individually, and some models performed well in controlled tests, but the complete interaction still had too many edge cases. With voice control, “mostly works” is not good enough — a light switch that responds 70% of the time is just a worse light switch. Local STT on the Pi was technically impressive and occasionally useful, but too brittle for everyday use.

Two side quests

Completely optional, but still very interesting side projects I decided to explore while building everything.

AI HAT+ 2

While testing different CPU-based STT models, I also had one obvious question in the back of my mind: what about the AI HAT+ 2?

My Raspberry Pi 5 had a Hailo-10H accelerator attached, and speech-to-text looked like exactly the kind of workload that should benefit from dedicated inference hardware. If the Hailo accelerator could handle STT quickly and leave the CPU mostly free, that would be a much nicer architecture.

Running the same WAV file tests as before made the speed result very clear: Hailo Whisper inference took around 0.8 seconds. But the actual recognition was disastrous. For the phrase:

Turn off the white fan

it produced something very funny:

Turn of the white pen
Turn of the White Benton

This made the AI HAT+ 2 result more interesting than simply saying “Hailo Whisper was not accurate enough”. The accelerator itself was clearly capable of running speech models quickly. The missing piece was the software interface around the model: contextual vocabulary, prompting, or some other reliable way to bias recognition toward my actual Home Assistant entities and commands.

For smart-home control, that context matters a lot. The STT engine does not need to understand the entire English language equally well. It needs to distinguish:

lights
white fan
Dyson
turn on
turn off

and it needs to do that reliably.

Without a good way to feed that context into the Hailo STT stack, the fast inference result was not enough. I could probably have spent more time building a custom Wyoming service around it, but if the available model still confused white fan with white pen, that would not solve the problem I actually had.

S330 LEDs

There was, however, one hardware side quest that worked much better than expected: the S330 LEDs. The Anker speakerphone exposes a HID interface, and after poking at it for a while, I found that some LED output reports were writable:

Report 0x03 → call button blinks green
Report 0x04 → mute button turns steady red

That was perfect for Linux Voice Assistant’s peripheral API.

I wrote a small service that listened to the assistant state and drove the LEDs accordingly:

assistant active / processing → green call LED blinks
pipeline finished             → LED off
pipeline error                → red LED briefly lights

This sounds like a tiny detail, but it made the system feel much more understandable. When the assistant was processing speech, the green LED kept blinking. When something failed, the red LED flashed. When the pipeline finished, the LEDs turned off.
That kind of feedback matters more than I expected. A spoken “sorry, I didn’t understand” response is nice, but it depends on the pipeline reaching the TTS stage. If STT fails early or times out, there may be no spoken response at all. A visual indicator can still work even when the voice pipeline itself is stuck or confused.

Ironically, this small HID detour felt more polished than several much larger parts of the project.

It did not fix recognition, of course. But it did make failures easier to understand, which is already a big improvement over a silent smart speaker that simply ignores you.

Conclusion

So in the end, the project failed.

Not completely, and not in an uninteresting way. The local stack worked well enough to prove that the idea is real: Home Assistant could control the devices, Linux Voice Assistant could act as a satellite, local STT and TTS could run on the Pi, and the whole thing could technically process voice commands without Google or Amazon involved.

But it failed at the thing that actually mattered: being usable every day.

A smart speaker is not judged by whether it works once in a demo. It is judged by whether it works when I walk past it, speak normally, and expect the light to turn off without thinking about microphone distance, model latency, grammar constraints, wake-word state, or whether the STT backend is about to hallucinate something ridiculous.

The individual parts were impressive, but the complete interaction was still too fragile. So I returned the Anker PowerConf S330 and parked the whole idea for now.

I still think local Home Assistant voice control is worth watching, the ecosystem is clearly moving in the right direction. The progress is real, and the building blocks are getting better. But at least for my setup, in 2026, “technically works” and “replaces a normal smart speaker” were still very different things.

Intro

I recently finished rewriting a production service from Go to Rust.
It was not a small experiment or a side project. The service handles a high volume of requests, runs across several regions, and sits on a critical path for the business. A failed rewrite would have caused real problems.

Once the Rust version reached production, CPU usage dropped by roughly 2–3×. We were able to reduce the number of Kubernetes pods by more than half while serving the same traffic with the same latency.

The “problem” here is that it was a much larger improvement than I anticipated.

The obvious conclusion would be that Rust is simply two or three times faster than Go. But after benchmarking both implementations and examining the main request paths, the real explanation turned out to be more interesting. Some of the improvement came from Rust itself while a significant part came from problems in the original Go implementation that the rewrite forced me to notice and fix.

This article covers how I approached the rewrite, how I used coding agents without handing over control of the architecture, what happened when the service reached production, and where the performance improvement actually came from.

The Approach

The original Go service contained around 9,000 lines of production code. That is small enough to rewrite manually, but large enough for the work to take a meaningful amount of time. At the same time, I also wanted to test how useful modern coding agents could be for a real production rewrite.

I did not want to give an agent the repository and ask it to “rewrite this in Rust.” The service was too important for a one-shot conversion, and matching the old behavior was only part of the job. The new implementation also needed a clean architecture, predictable performance, useful tests, and code that the team could maintain after the rewrite was finished.

So I treated the agents as implementation tools rather than architects. I started by designing the Rust service end to end. I decided how configuration, HTTP handling, shared state, database access, metrics, and error handling should work before asking an agent to implement the larger parts. I then wrote several smaller components and tests myself. These served as reference implementations: the agents could follow existing patterns instead of inventing a new structure for every module.
After that, the rewrite was split into a sequence of small stages, each covering a specific part of the existing service. At the end of every stage, I reviewed the code and corrected the direction where necessary before moving on.
The existing integration-test suite made this approach much safer. It covered around 90% of the service’s behavior and all of its critical paths. A stage was not considered complete until the expected tests passed.
Once the Rust implementation passed the full suite, the existing deployment pipeline allowed me to deploy it to a single production node first. That gave us a controlled way to compare both versions under real traffic before expanding the rollout.

Under these conditions, coding agents reduced the amount of implementation work considerably. But they were effective because the difficult decisions had already been made: the architecture was defined, the patterns were visible, the work was divided into small steps, and every step had a clear testable outcome.

Both Claude Opus 4.6–4.8 and Codex 5.4–5.5 were used throughout the rewrite, and under those constraints they proved genuinely useful implementation partners.

What the Service Does

The service in question is a cookie-matching microservice used in an advertising platform. In AdTech, different platforms often assign their own identifiers to the same browser. Cookie matching allows them to associate those external identifiers with an internal user ID. That mapping can later be used to enrich bid requests and make advertising decisions.

The request path is fairly simple:

1. The service receives an HTTP GET request containing a cookie
2. It validates and maps that value to an internal user ID
3. It stores the match in ScyllaDB
4. The ad server reads the mapping later when processing a bid request

The service is stateless and deployed on Kubernetes across the US, Europe, and APAC. Although the business logic is not especially complicated, the endpoint handles a high volume of traffic, so small costs on the request path add up quickly. Each pod requested 8 vCPUs and 2 GiB of memory, with average request latency staying below approximately 20 ms.

Here is a high-level comparison of the two implementations:

The Rust implementation ended up with less than half the production code and considerably less test code, while increasing the number of test cases.
That does not mean Rust automatically produces smaller programs. Some of the reduction came from removing duplication, simplifying abstractions, and redesigning parts of the service during the rewrite rather than translating the Go code line by line.
Still, it was an early sign that the new implementation was not merely a port. It was a chance to reconsider how the service worked internally.

What Happened in Production

Once the Rust service passed the full integration-test suite, I deployed it to a single production node. The initial rollout was intentionally small. I wanted to observe the new service under real traffic, compare its behavior with the Go version, and make sure there were no differences that our tests had missed. The service remained stable, so I gradually expanded the rollout.
The effect on resource usage was visible almost immediately. Under comparable traffic, the Rust implementation consumed roughly two to three times less CPU. Because the Kubernetes deployment scaled according to CPU usage, the lower consumption translated directly into fewer running pods.
The exact reduction varied with regional traffic, but the pattern was consistent. In absolute terms, each regional deployment typically went from three or four running pods to one or two while processing the same workload. All graphs are for the past 30 days from 21st June 2026:

That was far beyond what I had expected.

I had assumed Rust would perform somewhat better. The service is request-heavy, does a fair amount of parsing, and sits on top of an asynchronous HTTP stack, so a reduction in CPU usage would not have been surprising. But a two-to-three-times improvement raised a more important question:

Was Rust really that much faster, or had the rewrite also fixed problems in the original Go implementation?

Production metrics could show the size of the improvement, but they could not explain where it came from. To answer that, I needed to benchmark both versions and break the request path into its main components.

Finding the Source of the Improvement

To investigate, I ran both implementations locally under the same conditions. Both services were compiled as optimized builds, pinned to the same four CPU cores, and connected to the same local ScyllaDB instance and Google Cloud Storage emulator.
Rather than comparing request latency alone, I measured the CPU time consumed by each process per request. This is close to the signal used by our production autoscaling setup and gives a better view of how much compute each implementation needs to process the same workload.

The load generator replayed requests using a dataset of 15,000 real requests. One of the goals was to exercise the service’s most expensive code paths, including browser and device identification (aka User-Agent), which was expected to be a significant source of CPU usage. I tested two traffic distributions:

• Uniform: every user agent had the same probability of appearing. This creates an unrealistic worst case with a very low cache hit rate.
• Zipfian: a small number of common user agents appeared frequently, followed by a long tail of less common ones. This is much closer to real web traffic.

I also benchmarked the main request paths individually and ran the Go service with garbage collection disabled to isolate how much of the difference came from the GC.
The realistic Zipfian replay produced a larger gap than we saw in production: approximately 119 µs of CPU time per request in Go versus 33 µs in Rust, or around 3.6×.

That did not mean every part of the Rust service was 3.6 times faster. The total was the result of several separate differences, and they did not all have the same explanation. Some were genuine advantages of Rust and its ecosystem. Others were inefficiencies in the original Go implementation that could have been fixed even before.

Reason 1: User-Agent Parsing: The Biggest Contributor

Every incoming request contains a User-Agent header. The service parses it to identify the browser and device, mainly so it can skip browsers where third-party cookie matching is unlikely to work, which turned out to be more expensive than it may sound.
The parser checks the raw header against the shared uap-core rule set, which contains hundreds of regular expressions. Running that full set on every request would be far too costly, so both implementations cache previously parsed user agents.

The Go code itself was only a thin wrapper around uap-go:

func (p *parser) GetBrowser(ua string) string {
    return p.uap.ParseUserAgent(ua).Family
}

The library handled caching internally. The Rust version did the same job through uaparser, with a custom cache added around it by me. So, both version cached the results, but the gap in performance came from two things:

1. how effective the cache was
2. how expensive a cache miss was

The Go Cache Was Too Small and Shared

uap-go uses a fixed 1,024-entry ARC cache shared across all goroutines and protected by a mutex:

const cacheSize = 1024
c.userAgent, _ = lru.NewARC(cacheSize)

My own Rust implementation used an 8,192-entry LRU cache per worker thread:

thread_local! {
    static FAMILY_CACHE: RefCell<LruCache<String, String>> =
        RefCell::new(
            LruCache::new(NonZeroUsize::new(8192).unwrap())
        );
}

That difference matters under realistic traffic. User-agent strings repeat, but there is also a long tail of less common values. A 1,024-entry shared cache fills quickly, and every access also has to pass through the same lock. The larger per-thread Rust cache held more useful entries and avoided cross-thread contention.
This was not an inherent Go limitation. The service could have placed a larger sharded cache in front of uap-go, or switched to another parser. But neither option was a one-line fix. The cache size inside uap-go is hardcoded and not configurable. Replacing the library would also require careful validation because other parsers may classify browsers differently, while the service relies on exact browser-family names.

So this part was technically fixable in Go, but it required a real refactor.

Cache Misses Were Also Cheaper in Rust

The second difference appeared whenever the requested user agent was not already cached. Both libraries use the same underlying uap-core rules, but the regex engines are different. In the benchmark, parsing an uncached user agent took approximately:

• Go: 305 µs
• Rust: 131 µs

That made the Rust parser about 2.3 times faster on a cache miss. Unlike the cache design, this was a more direct ecosystem advantage. Rust’s regex engine handled this workload considerably faster than Go’s standard regexp package. Because a single miss costs hundreds of microseconds, the overall service performance closely followed the cache hit rate.

The realistic middle row is the most relevant. Under that workload, user-agent parsing accounted for roughly three-quarters of the total CPU gap between the two services. Rust had a genuine advantage when the parser had to execute the regex rules. But the original Go service also left a large amount of performance on the table through a small, shared, lock-protected cache driven by the choice of UA parsing library.

Reason 2: Metric Label Sanitization: A Pure Go Mistake

The service emits several Prometheus metrics for each request. Before a value could be used as a metric label, it passed through a small sanitization function. In the Go implementation, that function compiled its regular expression every time it was called:

func sanitizePrometheusLabelValue(value string) string {
    re := regexp.MustCompile(`[^\w]+`)
    return strings.Trim(re.ReplaceAllString(value, "_"), "_")
}

That meant every request could trigger the same regular expression compilation several times. The cost of one call was not enormous — about 1.3 µs — but it also created around 18 allocations. On a high-volume request path, repeated small costs like this add up quickly.

The Rust version compiled the pattern once and reused it:

static LABEL_SANITIZER: LazyLock<Regex> =
    LazyLock::new(|| Regex::new(r"[^\w]+").unwrap());

...

pub fn sanitize_prometheus_label_value(value: &str) -> String {
    LABEL_SANITIZER
        .replace_all(value, "_")
        .trim_matches('_')
        .to_string()
}

That reduced the cost to roughly 146 ns per call, almost nine times faster. But this was not a Rust advantage. The Go implementation could achieve nearly the same result by moving the compiled expression to package scope:

var labelSanitizer = regexp.MustCompile(`[^\w]+`)

...

func sanitizePrometheusLabelValue(value string) string {
    return strings.Trim(
        labelSanitizer.ReplaceAllString(value, "_"),
        "_",
    )
}

This was simply a hot-path mistake in the original service — one that the rewrite happened to expose.

Reason 3: GeoIP Decoding: Reading Far More Than We Needed

The service performs a GeoIP lookup for every request to determine the client’s country. The Go implementation used the high-level Country() method from geoip2-golang:

record, _ := d.reader.Country(parsedIP)
isoCode := record.Country.IsoCode

The service only needed the two-letter ISO country code. However, the library decoded the full country record, including continent data, localized names, registered-country information, and other fields that the request path never used.

The Rust implementation took a narrower approach. It declared a small structure containing only the required fields:

#[derive(Deserialize)]
struct CountryRecord<'a> {
    #[serde(borrow)]
    country: Option<CountryIso<'a>>,
}

#[derive(Deserialize)]
struct CountryIso<'a> {
    iso_code: Option<&'a str>,
}

The lookup then decoded only that projection:

let iso = reader
    .lookup(ip)?
    .decode::<CountryRecord>()?
    .country
    .and_then(|country| country.iso_code);

This reduced the lookup from approximately 3.3 µs and 64 allocations in Go to around 428 ns in Rust. At first glance, that looks like another large Rust win. But Go could do the same thing.

geoip2-golang is built on top of maxminddb-golang, and the lower-level decoder supports partial records. By using it directly, the Go implementation could declare only the field it needed, which would avoid decoding the unused sections of the database record:

type countryRecord struct {
    Country struct {
        ISOCode string `maxminddb:"iso_code"`
    } `maxminddb:"country"`
}

var record countryRecord
_ = reader.Lookup(parsedIP, &record)
isoCode := record.Country.ISOCode

So, once again, the improvement did not come from something Go was incapable of doing. The original implementation had chosen a convenient high-level API, and the rewrite replaced it with a more focused hot-path implementation.

Reason 4: The Runtime and Framework

The rest of improvements was the general cost of processing a request: HTTP handling, allocations, middleware, shared state, and runtime overhead.
The Rust service used actix-web on Tokio, while the Go service used net/http. In this workload, the Rust stack required fewer allocations and had a lower baseline cost per request. This is also where the lack of a garbage collector helped — but much less than I initially expected.

Garbage collection is often the first explanation people reach for when comparing Go with Rust. To measure its actual impact, I ran the Go service with GOGC=off, disabling automatic collection during the benchmark.

CPU usage improved by only around 8% on the base request path, equivalent to roughly 4% of the total gap under realistic traffic. Go was still close to twice as expensive per request even with the collector disabled. And Rust does not have to pay that cost. But it was not the main reason for the overall result. The larger difference came from the combined cost of framework behavior and per-request allocations. Go could reduce some of that through further profiling and optimization, but Rust started from a lower floor.

This was the clearest part of the benchmark where the language and runtime choice itself made a lasting difference.

The Results in One View

Here are the main hot paths side by side:

These numbers are useful, but they do not contribute equally to the final result. A user-agent cache miss costs hundreds of microseconds but happens only on some requests. Metric sanitization is much cheaper, but it runs several times per request. Garbage collection is spread across many requests rather than appearing as one explicit function call.

After weighting each cost by how often it occurred in the realistic Zipfian workload, the approximate breakdown of the 86 µs saved per request looked like this:

The user-agent number and the garbage-collection number came directly from end-to-end measurements and the GOGC=off test. The split between sanitization, GeoIP, and the remaining framework overhead is an estimate based on their individual costs and how often they run.
The proportions also depend on traffic. With a higher user-agent cache hit rate, parsing matters less and the base request overhead matters more. With a lower hit rate, parsing dominates even further.

The Fair Takeaway

The production result was real: the Rust service used two to three times less CPU and allowed each regional deployment to run on one or two pods instead of three or four. But the honest conclusion is not that Rust is automatically two or three times faster than Go. A large part of the improvement came from issues that could also have been fixed in the original service. Had we profiled and optimized the Go implementation first, the gap would have been considerably smaller.

Rust still had real advantages. Its regex engine handled cache misses much faster, its HTTP stack had a lower per-request cost in this workload, and its ownership model allowed the service to avoid much of the allocation and garbage-collection overhead present in Go. The rewrite also acted as a forcing function. Instead of carrying every old abstraction and implementation decision forward, I had to examine each part of the request path again. Some of the largest gains came not from translating the code, but from asking why the old code worked the way it did. Yet the language still determines the performance envelope available to the system.

And just as importantly, Rust turned out to be far less intimidating in practice than its reputation suggests, and the resulting codebase was generally smaller and easier to reason about than the one it replaced.

The lesson I take away is not merely that rewrites can be valuable. It is that a well-designed systems language like Rust gives you a stronger foundation to build high-throughput services on, and a rewrite is often the moment when those advantages become impossible to ignore.

I bought a Raspberry Pi 5 with a fairly ambitious idea in mind: use it as a small local AI box for coding tasks.

Part of the motivation was practical. I wanted something that could take a bit of pressure off Claude API limits. Part of it was also simpler than that: I already had this surprisingly expensive little board, sold with plenty of “AI experiment” energy around it, and I wanted to find a real use for it.
Running a local coding model on a Pi sounded perfect on paper. Low power, always on, sitting on the desk, ready to help with tests, scripts, and small coding chores. In my head, it would become a kind of tiny local coding agent.

In practice, that is not what happened.

This article is a write-up of setting up Ollama and Aider on a Raspberry Pi 5, trying a few realistic coding tasks, and realizing that even the beefiest 16 GB Pi 5 is not really a substitute for a proper local coding machine.

The idea

The goal was modest enough. I was not expecting the Pi to rival a desktop GPU box or a cloud model. I just wanted it to do a few useful coding tasks locally:

• generate some unit test
• help with small refactors
• write tiny Bash/Python utilities
• maybe handle quick back-and-forth tasks through OpenClaw from Telegram

That felt reasonable. Modern small coding models are getting better, Ollama makes local serving easy, and running it through a CLI is very nice and easy with the help of Aider. So I decided to give it a try and set it up.

Part 1. Ollama on the Pi

The good news is that the setup part is actually quite easy.

I started with a pretty standard Ollama install:

curl -fsSL https://ollama.com/install.sh | sh

Then I checked that it was alive:

ollama --version

After that, I pulled a few coding models — I was experimenting with Qwen coder variants:

ollama pull qwen2.5-coder:1.5b
ollama pull qwen2.5-coder:3b
ollama pull qwen2.5-coder:7b

Lastly, I exposed ollama outside of Pi by editing it’s systemd unit — nano /etc/systemd/system/ollama.service — and adding the following two lines:

Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_CONTEXT_LENGTH=8192"

After restarting the serivce you should be able to access it from within your local network

curl http://<pi-ip>:11434/api/tags

{"models":[{"name":"qwen2.5-coder:7b","model":"qwen2.5-coder:7b","modified_at":"2026-03-29T15:50:06.938558636+02:00","size":4683087561,"digest":"dae161e27b0e90dd1856c8bb3209201fd6736d8eb66298e75ed87571486f4364","details":{"parent_model":"","format":"gguf","family":"qwen2","families":["qwen2"],"parameter_size":"7.6B","quantization_level":"Q4_K_M"}},{"name":"qwen2.5-coder:1.5b","model":"qwen2.5-coder:1.5b","modified_at":"2026-03-29T15:30:35.122956073+02:00","size":986062089,"digest":"d7372fd828518a4d38b1eb196c673c31a85f2ed302b3d1e406c4c2d1b64a0668","details":{"parent_model":"","format":"gguf","family":"qwen2","families":["qwen2"],"parameter_size":"1.5B","quantization_level":"Q4_K_M"}},{"name":"qwen2.5-coder:3b","model":"qwen2.5-coder:3b","modified_at":"2026-03-29T14:51:00.437576892+02:00","size":1929912626,"digest":"f72c60cabf6237b07f6e632b2c48d533cef25eda2efbd34bed21c5e9c01e6225","details":{"parent_model":"","format":"gguf","family":"qwen2","families":["qwen2"],"parameter_size":"3.1B","quantization_level":"Q4_K_M"}}]}

Part 2. Adding Aider

Next step was Aider, because I did not want to just chat with the model in a terminal. The whole point was to see whether the Pi could act as a practical coding assistant.
I installed Aider with:

curl -LsSf https://aider.chat/install.sh | sh

Then I pointed Aider at the local Ollama endpoint:

export OLLAMA_HOST=http://<pi-ip>:11434
aider --model ollama/qwen2.5-coder:3b

As you can see, the setup is very straightforward — Ollama and Aider are both quite approachable. That is actually part of why this whole thing is tempting: you can get from zero to “local coding assistant” surprisingly fast.
The friction starts when you want the model to be useful enough that it saves time rather than consumes it.

Part 3. Testing it on real tasks

That’s where it finally gets interesting. I did not want to benchmark nonsense prompts or ask it trivia questions. I wanted to try small, normal routine developer tasks.

Task 1: generate a Go test

The first meaningful test was asking it to generate a Go unit test for an existing class or function. This is exactly the sort of thing I imagined a local coding model could help with: something structured, something repetitive, something where “pretty good” is often enough.

The repo I tried it on was a small Go project, and the target file was just under 300 lines. I tried 1.5B, 3B, and 7B models, and all of them were a total failure — the API timed out after 10 minutes without ever returning an answer. And obviously, waiting that long for a response to a coding request is not assistance — it is an interruption. And that is without even talking about concurrent requests.

Next, I decided to check whether it was actually capable of generating anything at all.

Task 2: a small Python script

Next, I decided to make the task much easier. Instead of asking the model to work with an existing Go codebase, I created a very small Python example just for testing:

def to_slug(title: str, max_length: int = 50) -> str:
    """
    Convert a title into a URL-friendly slug.

    Rules:
    - lowercase everything
    - keep letters, numbers, spaces, hyphens, underscores
    - replace spaces/underscores with hyphens
    - collapse multiple hyphens
    - strip leading/trailing hyphens
    - truncate to max_length, then strip trailing hyphens again

    Raises:
    - TypeError if title is not a string
    - ValueError if title is empty/blank or max_length < 1
    """
    if not isinstance(title, str):
        raise TypeError("title must be a string")
    if max_length < 1:
        raise ValueError("max_length must be at least 1")

    title = title.strip().lower()
    if not title:
        raise ValueError("title cannot be empty")

    # Remove unsupported characters, but keep spaces, hyphens, underscores.
    title = re.sub(r"[^a-z0-9\s\-_]", "", title)

    # Convert spaces/underscores to hyphens.
    title = re.sub(r"[\s_]+", "-", title)

    # Collapse repeated hyphens.
    title = re.sub(r"-+", "-", title)

    # Trim edges.
    title = title.strip("-")

    # Enforce max length.
    title = title[:max_length].rstrip("-")

    if not title:
        raise ValueError("title produced an empty slug")

    return title

This felt like a much more forgiving test. Less context, less structure, less code to produce. If the Pi could not handle a Go test in a small repo, maybe it could at least manage a tiny Python exercise.

The prompt was:

> Write a small pytest test file for string_tools.py. 
>  
> Focus only on the function to_slug. 
> Cover: 
> - normal title 
> - uppercase letters 
> - multiple spaces 
> - underscores 
> - punctuation removal 
> - repeated hyphens collapsing 
> - truncation with max_length 
> - blank input raising ValueError 
> - non-string input raising TypeError 
> - max_length < 1 raising ValueError 
>  
> Return only the test file code.                  

qwen2.5-coder:1.5b

The 1.5B model took around 4 minutes to generate a test file.
It did produce something usable:

import pytest
from string_tool import to_slug

def test_to_slug_normal_title():
    assert to_slug("Hello World") == "hello-world"

def test_to_slug_uppercase_letters():
    assert to_slug("HELLO WORLD") == "hello-world"

def test_to_slug_multiple_spaces():
    assert to_slug("  Hello   World  ") == "hello-world"

def test_to_slug_underscores():
    assert to_slug("Hello_World") == "hello-world"

def test_to_slug_punctuation_removal():
    assert to_slug("Hello, World!") == "helloworld"

def test_to_slug_repeated_hyphens_collapsing():
    assert to_slug("Hello---World") == "hello-world"

def test_to_slug_truncation_with_max_length():
    assert to_slug("Hello World", max_length=10) == "hello-worl"

def test_to_slug_blank_input_raising_ValueError():
    with pytest.raises(ValueError):
        to_slug("")

def test_to_slug_non_string_input_raising_TypeError():
    with pytest.raises(TypeError):
        to_slug(123)

def test_to_slug_max_length_less_than_1_raising_ValueError():
    with pytest.raises(ValueError):
        to_slug("Hello World", max_length=0)

but one of the tests was simply wrong:

======================================================================================= FAILURES =======================================================================================
___________________________________________________________________________ test_to_slug_punctuation_removal ___________________________________________________________________________

    def test_to_slug_punctuation_removal():
>       assert to_slug("Hello, World!") == "helloworld"
E       AssertionError: assert 'hello-world' == 'helloworld'
E         - helloworld
E         + hello-world
E         ?      +

test_string_tool.py:17: AssertionError
=============================================================================== short test summary info ================================================================================
FAILED test_string_tool.py::test_to_slug_punctuation_removal - AssertionError: assert 'hello-world' == 'helloworld'

============================================================================= 1 failed, 9 passed in 0.09s ==============================================================================

This was not a terrible result in absolute terms. The model mostly understood the task. But 4 minutes to produce a small pytest file with a wrong assertion is already far outside what I would consider useful in a real workflow.

qwen2.5-coder:3b

The 3B model took around 4 and a half minutes. This time, it generated a better test file:

import pytest
from string_tool import to_slug


def test_to_slug_normal_title():
    assert to_slug("Hello World") == "hello-world"


def test_to_slug_uppercase_letters():
    assert to_slug("HELLO WORLD") == "hello-world"


def test_to_slug_multiple_spaces():
    assert to_slug("  Hello   World  ") == "hello-world"


def test_to_slug_underscores():
    assert to_slug("Hello_World") == "hello-world"


def test_to_slug_punctuation_removal():
    assert to_slug("Hello, World!") == "hello-world"


def test_to_slug_repeated_hyphens_collapsing():
    assert to_slug("Hello---World") == "hello-world"


def test_to_slug_truncation_with_max_length():
    assert to_slug("Hello World", max_length=10) == "hello-worl"


def test_to_slug_blank_input_raises_value_error():
    with pytest.raises(ValueError):
        to_slug("")


def test_to_slug_non_string_input_raises_type_error():
    with pytest.raises(TypeError):
        to_slug(123)


def test_to_slug_max_length_less_than_one_raises_value_error():
    with pytest.raises(ValueError):
        to_slug("Hello World", max_length=0)

All 10 cases passed without modification:

10 passed in 0.02s

The output was decent, but the latency was still absurd for something this small. I could probably write the test faster myself than wait for the model to finish thinking about it.
There was also still room for improvement in the generated output. For example, I would probably add one extra test for an all-whitespace input.

with pytest.raises(ValueError):
    to_slug("   ")

qwen2.5-coder:7b

The 7B model took more than 10 minutes. Unlike the smaller two, it actually placed the test under test/test_string_tool.py, which at first looked like a smarter result:

import pytest
from string_tool import to_slug

def test_normal_title():
    assert to_slug("Hello World") == "hello-world"

def test_uppercase_letters():
    assert to_slug("HELLO WORLD") == "hello-world"

def test_multiple_spaces():
    assert to_slug("  Hello   World  ") == "hello-world"

def test_underscores():
    assert to_slug("Hello_World") == "hello-world"

def test_punctuation_removal():
    assert to_slug("Hello, World!") == "hello-world"

def test_repeated_hyphens_collapsing():
    assert to_slug("Hello--World") == "hello-world"

def test_truncation_with_max_length():
    assert to_slug("This is a very long title that should be truncated", max_length=10) == "this-is-a-very"

def test_blank_input_raising_ValueError():
    with pytest.raises(ValueError):
        to_slug("")

def test_non_string_input_raising_TypeError():
    with pytest.raises(TypeError):
        to_slug(123)

def test_max_length_less_than_1_rasing_ValueError():
    with pytest.raises(ValueError):
        to_slug("Hello World", max_length=0)

But the file did not even run:

E   ModuleNotFoundError: No module named 'string_tool'

And after correcting the import issue manually, one test was still failing:

======================================================================================= FAILURES =======================================================================================
___________________________________________________________________________ test_truncation_with_max_length ____________________________________________________________________________

    def test_truncation_with_max_length():
>       assert to_slug("This is a very long title that should be truncated", max_length=10) == "this-is-a-very"
E       AssertionError: assert 'this-is-a' == 'this-is-a-very'
E         - this-is-a-very
E         ?          -----
E         + this-is-a

test_string_tool.py:23: AssertionError
=============================================================================== short test summary info ================================================================================
FAILED test_string_tool.py::test_truncation_with_max_length - AssertionError: assert 'this-is-a' == 'this-is-a-very'
============================================================================= 1 failed, 9 passed in 0.09s ==============================================================================

The more disappointing part was that the tool did not really know how to complete the loop on its own. It could generate test code, but it was not capable of running the tests, observing the failures, and iteratively fixing the mistakes as part of the same workflow.

That matters, because the whole point of a local coding assistant is not just to dump out code. It is to help you get to a working result with minimal friction.

At that point, the experience was nowhere near something I would actually reach for during normal work. The latency alone made even small tasks feel heavier than they should.

Part 4. Where it falls apart

In hindsight, it seems obvious that a Raspberry Pi — even the most capable 16 GB model — is not a tool designed for AI coding assistance. That does not mean the Raspberry Pi 5 is bad hardware. On the contrary, it is actually impressive for what it is meant to do.

But LLM inference, especially for coding, is brutally demanding once you move from toy demos to actual use. Coding models need enough memory, enough throughput, and enough raw inference speed to keep the interaction tight. That is where the Pi falls behind.

Even when the model technically fits, the experience is bottlenecked by the kind of hardware the Pi is: limited CPU performance compared to real desktops, no serious GPU acceleration for this kind of workload, and memory bandwidth that simply is not designed for fast local inference with coding models.

Conclusion

My original idea was simple: use a Raspberry Pi 5 as a local coding agent to ease API usage a bit and finally give this expensive little machine a serious AI job.
After trying it, my conclusion is also simple: this is not the right machine for that task.

If your goal is to tinker, learn how local models work, or build small demos, the Pi is still fun. If your goal is to have something that genuinely helps with day-to-day coding, even the 16 GB Raspberry Pi 5 is not enough.

And honestly, I think that is worth saying out loud, because a lot of hobbyist AI content focuses on the fact that something can be made to run, not whether it should be used that way.

So my Pi probably has a different future now. Less “local coding agent”, more “smart home box”, Home Assistant host, or some other always-on project that actually fits the hardware.

Running large language models locally used to mean a power-hungry desktop GPU and a fair amount of patience. With the Raspberry Pi 5 and the Raspberry Pi AI HAT+ 2 (Hailo-10H), that barrier is much lower. You can build a compact, low-power system that runs LLM inference entirely on your desk — or, in my case, on a TV stand.

This guide walks through setting up a fully local LLM environment on a Raspberry Pi 5 using Ubuntu Server 24.04, the Raspberry Pi AI HAT+ 2, Hailo’s Ollama-compatible runtime, and OpenWebUI. By the end, you’ll have a persistent system service that starts automatically at boot and exposes a browser-based chat interface powered completely by local hardware.
The goal isn’t to compete with data-center GPUs. It’s to build a practical, private, always-on AI node that you control — something you can integrate into home automation, development workflows, or offline experimentation without relying on external infrastructure.

In the final setup, the architecture looks like this:

Browser
  ↓
OpenWebUI (systemd service, port 8080)
  ↓
hailo-ollama (systemd service, port 8000)
  ↓
/dev/hailo0
  ↓
Hailo-10H hardware accelerator

The main idea is to keep the stack native and persistent: no Docker, no manual launch steps after reboot, and as few moving parts as possible.

From “Build It From Scratch” to “Make It Stable”

The original plan was straightforward: run a fully local LLM on a Raspberry Pi 5 with Ubuntu and the AI HAT+ 2 using only native Linux components.
So I started the way many of us would: build everything manually.
That meant compiling the Hailo kernel driver against the running kernel, building the runtime libraries, and wiring the stack together piece by piece. In theory, this gives maximum control. In practice, it turned into a maintenance loop. Kernel module builds were sensitive to small changes — compiler versions, header mismatches, minor kernel updates. Fix one issue, hit the next. Even after the device showed up correctly as /dev/hailo0, the userspace runtime had to match the hardware generation exactly. Some versions detected the device but couldn’t communicate with it properly. Others simply weren’t packaged for Ubuntu yet.

At that point, the question wasn’t “Can this work?” — it clearly could.
The question was “Will this stay working?”

Instead of continuing to rebuild everything manually, I switched to using the Raspberry Pi repositories for the Hailo driver and runtime packages. Initially that felt like a compromise, since the base system is Ubuntu. But by limiting it strictly to the Hailo components, compatibility issues never surfaced. The difference was immediate:

• The kernel module loaded cleanly
• The runtime matched the hardware
• Models initialized straight away
• The inference server stayed stable across reboots

Building from source gives control, but using vendor-maintained packages gives stability. For this project, stability won — and that’s what turned it from an experiment into a reliable, always-on local AI node.

Next, let’s go over the installation steps one by one.

Note! At the time of writing, Raspberry Pi’s official LLM setup guide for the AI HAT+ 2 references version 5.1.1 of the Hailo GenAI Model Zoo package. I used that same version here to stay aligned with the vendor documentation. Besides that, version 5.2.0 wasn’t available in the official repo anyways.

Step 1. Install the Hailo Kernel Driver

The first step is getting the kernel-level pieces in place so Ubuntu can talk to the accelerator correctly. Since the required Hailo packages were available through the Raspberry Pi package repository, I added that repository and pinned it so only the Hailo-related packages would be pulled from there:

curl -fsSL https://archive.raspberrypi.com/debian/raspberrypi.gpg.key \
| sudo gpg --dearmor -o /usr/share/keyrings/raspberrypi-archive-keyring.gpg

echo "deb [arch=arm64 signed-by=/usr/share/keyrings/raspberrypi-archive-keyring.gpg] \
http://archive.raspberrypi.com/debian trixie main" \
| sudo tee /etc/apt/sources.list.d/raspberrypi.list

Then create an APT pinning file so Ubuntu does not start preferring Raspberry Pi packages for unrelated components:

nano /etc/apt/preferences.d/raspberrypi-pin

Package: *
Pin: origin archive.raspberrypi.com
Pin-Priority: 1

Package: h10-hailort-pcie-driver hailort hailort-* libhailort* hailo*
Pin: origin archive.raspberrypi.com
Pin-Priority: 1001

With that in place, install the kernel driver and matching kernel headers:

sudo apt update
sudo apt install h10-hailort-pcie-driver linux-headers-$(uname -r)

To verify that the device is visible to the system, check for the Hailo device node:

ls -l /dev/hailo*

crw-rw-rw- 1 root root 234, 0 Feb 24 23:00 /dev/hailo0

If /dev/hailo0 is present, the kernel side of the setup is in good shape.

Step 2. Install the Hailo Runtime

Once the driver is loaded and the device is exposed, the next layer is the Hailo runtime itself. This is what allows user-space applications to communicate with the accelerator properly. Install the Hailo-10H runtime:

sudo apt install h10-hailort

Then verify that the runtime can talk to the hardware:

hailortcli fw-control identify

The output should include something along the lines of:

Firmware Version: 5.1.1
Device Architecture: HAILO10H

This is an important checkpoint. At this stage, the hardware is not just detected by Linux — it is actually responding correctly through the runtime stack.

Step 3. Install the Hailo Ollama Runtime

With the low-level pieces working, the next step is the inference layer. In this setup, that role is handled by hailo-ollama, which exposes an Ollama-compatible API server. Install the model zoo package:

wget https://dev-public.hailo.ai/2025_12/Hailo10/hailo_gen_ai_model_zoo_5.1.1_arm64.deb
sudo dpkg -i hailo_gen_ai_model_zoo_*.deb

After installation, verify that the API is responding:

curl http://localhost:8000/hailo/v1/list

If everything is working, you should get a JSON response listing the available models:

{"models":[...]}

At that point, the backend inference service is up and ready.

Step 4. Install and configure OpenWebUI

For the frontend, I wanted a simple web interface without introducing containers, so OpenWebUI was installed directly into its own Python virtual environment.

First, create a dedicated service user:

sudo useradd -r -m -d /opt/openwebui -s /usr/sbin/nologin openwebui

Then switch to that user and install OpenWebUI inside a virtual environment:

sudo -u openwebui bash
cd /opt/openwebui
python3 -m venv venv
source venv/bin/activate
pip install open-webui
exit

Keeping OpenWebUI under its own system user makes the setup cleaner and easier to manage later.

Next, create a small environment file for OpenWebUI:

nano /opt/openwebui/env

OLLAMA_BASE_URL=http://localhost:8000
WEBUI_AUTH=False

The important part here is OLLAMA_BASE_URL=http://localhost:8000, which points OpenWebUI at the local hailo-ollama service.

Step 5. Create a systemd Service for Hailo Ollama

The inference service also needs to start automatically, so create a systemd unit:

nano /etc/systemd/system/hailo-ollama.service

[Unit]
Description=Hailo Ollama Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/hailo-ollama serve
Restart=always

[Install]
WantedBy=multi-user.target

Enable and start it:

sudo systemctl daemon-reload
sudo systemctl enable hailo-ollama
sudo systemctl start hailo-ollama

Step 6. Create a systemd Service for OpenWebUI

To make the web interface persistent across reboots, create a systemd service:

nano /etc/systemd/system/openwebui.service

[Unit]
Description=OpenWebUI Service
After=network.target hailo-ollama.service
Wants=hailo-ollama.service

[Service]
Type=simple
WorkingDirectory=/opt/openwebui
EnvironmentFile=/opt/openwebui/env
ExecStart=/opt/openwebui/venv/bin/open-webui serve --host 0.0.0.0 --port 8080

Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Then reload systemd, enable the service, and start it:

sudo systemctl daemon-reload
sudo systemctl enable openwebui
sudo systemctl start openwebui

With both services enabled, the system comes back up cleanly after a reboot and restores the full local LLM stack automatically.

Step 7. Access the Web UI

Finally, once both services are running, open a browser and connect to:

http://<raspberry-pi-ip>:8080

If the backend is reachable, the models should appear automatically in OpenWebUI and the system is ready to use.

Final Thoughts

The Raspberry Pi 5 and AI HAT+ 2 can absolutely be turned into a fully local LLM appliance, and that alone is impressive. But getting there is still more about careful integration than plug-and-play simplicity. Once the stack is stable, it is neat, quiet, and self-contained — but the performance is still limited, with relatively slow generation and a narrow practical model range. So, don’t expect it to be your “ChatGPT replacement” quite yet.

The bigger issue is software support — especially outside of Raspberry Pi OS. Raspberry Pi launched the AI HAT+ 2 in January 2026, yet its official documentation still references GenAI package version 5.1.1, while Hailo’s own ecosystem already points to 5.2.0 support for Hailo-10H. For a product aimed at local AI on the Pi, that lag is hard to ignore. Especially, combined with the fact that some of the models were replaced — most notably Llama-3.2–3B-Instruct model. It does not break the project per se, but it does make the platform feel immature — and it makes the buying decision harder to justify unless you specifically enjoy working through rough edges. For hobbyists who like experimenting, it is interesting. For anyone expecting a polished out-of-the-box local LLM platform, it is probably not there yet.

Working on an enterprise-level application, every team eventually faces a familiar challenge — users having doubts or questions about how certain parts of the system work. Answering those can quickly become a time-consuming, context-switching task for developers.

To make that process smoother at the company I’m with right now, I decided to build a support request Slack bot tailored to our needs. What I first thought would be a quick weekend project turned into a two-month journey.

In this article, I’ll share some lessons learned, tips and tricks, and parts of the implementation to help fellow developers build their own bots faster and more efficiently.

Part 1. Requirements

To set the right context, here’s what functionality the support bot should handle:

• A user goes to a dedicated Slack channel and starts a Workflow via, as we call it, “Big green button” that appears when you feature a given workflow in a channel
• A form appears with a set of fields — most importantly, the description of the request and the team that should take a look at it
• Upon submission, a message with the provided description, team, and a set of action buttons (Claim, Take Over, Resolve, and Create Jira Ticket) should be posted
• Someone from the team picks it up, works on the request, and eventually closes it

As with any KPI-driven organisation, we also wanted to collect metrics — such as how many requests come in each month and how long they take to resolve.

They say a picture is worth a thousand words, so here’s what the final flow looks like:

As you can see, the core requirements boil down to just two things: interactivity and simplicity.

In the next part, we’ll move from what the bot should do to how it’s built — exploring the architecture, integration options, and the deployment setup.

Part 2. Architecture

There are two ways to host a Slack bot — directly on Slack or self-hosted on your own infrastructure. That choice greatly affects how the bot will be built. Based on our requirements, the obvious decision was to self-host it. Building TypeScript-based workflows directly on Slack has certain limitations that just weren’t acceptable for what we needed.

When it comes to implementation, Slack provides several tools — you can choose from Java, JavaScript, or Python via their Bolt frameworks or lower-level API SDKs. Generally speaking:

• Bolt is the better choice when you want a clean abstraction of Slack-related logic.
• Slack API SDKs should only be considered if you’re building something highly customized or more complex.

Next, there are two integration modes you can use — Socket Mode or HTTP.
I won’t go too deep into the differences here (the official docs explain them well), but from experience: go with HTTP for production.
Socket Mode is fine for quick tests or prototypes, but it becomes painful when you try to make it production-ready. Save yourself the trouble unless you’re building something that will never leave the prototype stage.

Then comes another confusing part — how your bot is used in Slack. You can expose it through Slash Commands, Shortcuts, or Workflow Steps.
Here’s the catch: Slack Workflows can only use Workflow Steps.
There’s no way to wire Slash Commands or Shortcuts into them.
I initially assumed Slash Commands would be the most straightforward integration — big mistake. I ended up rewriting about 40% of the app.
Lesson learned: if you’re building for Slack Workflows, stick to Workflow Steps only:

app.function("support-request", (req, ctx) -> ctx.ack());

With all that in mind, for our requirements I went with:

• Java 25
• Bolt for Java
• Spring Boot
• Hosted on Google Cloud Run
• Connected to Slack via HTTP, integrated as a Workflow Step

Part 3. Implementation

Unlike most of my tutorials here, I won’t be sharing a complete GitHub repo — instead, I’ll focus on the key pieces of the implementation, with a few code snippets and hints along the way. I’ll also skip the “Create Jira Ticket” logic to keep things focused on the Slack side.

Let’s start with the app configuration. You’ll need two secret values:

• Bot User OAuth Token — found on the Install App page
• Signing Secret — found on the Basic Information page

With those two, you can create a simple Spring configuration class that wires up the Slack Bolt app, request handler, verifier, and parser beans:

@Configuration
public class AppConfiguration {

    @Bean
    public App slackBoltApp(final RequestHandler requestHandler,
                            @Value("${SLACK_BOT_TOKEN:}") final String botToken,
                            @Value("${SLACK_SIGNING_SECRET:}") final String signingSecret) {
        final var appConfig = buildAppConfig(botToken, signingSecret);
        final var app = new App(appConfig);
        requestHandler.registerHandlers(app);
        return app;
    }

    private AppConfig buildAppConfig(final String botToken,
                                     final String signingSecret) {
        final var config = new AppConfig();
        config.setSingleTeamBotToken(botToken);
        config.setSigningSecret(signingSecret);
        config.setRequestVerificationEnabled(true);
        config.setAttachingFunctionTokenEnabled(false);
        return config;
    }

    @Bean
    public SlackSignature.Verifier signatureVerifier(final App app) {
        final var signingSecret = app.config().getSigningSecret();
        final var generator = new SlackSignature.Generator(signingSecret);
        return new SlackSignature.Verifier(generator);
    }

    @Bean
    public SlackRequestParser requestParser(final App app) {
        return new SlackRequestParser(app.config());
    }
}

config.setAttachingFunctionTokenEnabled(false) part will be explained later — just keep it in mind for now.

Once the configuration is in place, the next step is implementing a controller to handle incoming Slack requests. This controller reads the request body, verifies the signature (to confirm it’s from Slack), and then passes it into app.run() for processing:

@RestController
public class SlackEventsController {
    private final App app;
    private final SlackRequestParser requestParser;
    private final SlackSignature.Verifier signatureVerifier;

    @PostMapping("/slack/events")
    public ResponseEntity<String> handleSlackEvents(final HttpServletRequest request) {
        final byte[] bodyBytes;
        try {
            bodyBytes = request.getInputStream().readAllBytes();
        } catch (final IOException e) {
            log.error("Failed to read Slack request body", e);
            return ResponseEntity.internalServerError().body("Unable to read request");
        }
        final var body = new String(bodyBytes, StandardCharsets.UTF_8);
        final var timestamp = request.getHeader(SlackSignature.HeaderNames.X_SLACK_REQUEST_TIMESTAMP);
        final var signature = request.getHeader(SlackSignature.HeaderNames.X_SLACK_SIGNATURE);

        if (!signatureVerifier.isValid(timestamp, body, signature)) {
            log.warn("Rejected Slack request due to invalid signature or timestamp. timestamp={} signature={} bodyLength={}",
                     timestamp,
                     signature,
                     body.length());
            return ResponseEntity.status(HttpStatus.UNAUTHORIZED).body("Invalid request signature");
        }

        final var slackRequest = requestParser.parse(buildHttpRequest(request, body));
        if (slackRequest == null) {
            log.warn("Unable to parse Slack request for uri={} body={}", request.getRequestURI(), body);
            return ResponseEntity.badRequest().body("Unrecognized Slack request");
        }

        try {
            final var slackResponse = app.run(slackRequest);
            return toResponseEntity(slackResponse);
        } catch (final Exception e) {
            log.error("Failed to process Slack request", e);
            return ResponseEntity.internalServerError().body("Something went wrong");
        }
    }

    private SlackRequestParser.HttpRequest buildHttpRequest(final HttpServletRequest request,
                                                            final String body) {
        return SlackRequestParser.HttpRequest.builder()
                                             .requestUri(request.getRequestURI())
                                             .requestBody(body)
                                             .headers(extractHeaders(request))
                                             .remoteAddress(request.getRemoteAddr())
                                             .socketMode(false)
                                             .build();
    }

    private RequestHeaders extractHeaders(final HttpServletRequest request) {
        final Map<String, List<String>> headers = new HashMap<>();
        final var headerNames = request.getHeaderNames();
        while (headerNames.hasMoreElements()) {
            final var name = headerNames.nextElement();
            headers.put(name, Collections.list(request.getHeaders(name)));
        }
        return new RequestHeaders(headers);
    }

    private ResponseEntity<String> toResponseEntity(final Response slackResponse) {
        final var statusCode = slackResponse.getStatusCode() != null
                ? HttpStatus.valueOf(slackResponse.getStatusCode())
                : HttpStatus.OK;
        final var headers = new HttpHeaders();
        slackResponse.getHeaders().forEach((name, values) -> values.forEach(value -> headers.add(name, value)));
        if (slackResponse.getContentType() != null) {
            headers.set(HttpHeaders.CONTENT_TYPE, slackResponse.getContentType());
        }
        return new ResponseEntity<>(slackResponse.getBody(), headers, statusCode);
    }
}

In short, what’s happening here is:

1. The raw HTTP request is read and validated using Slack’s signature verifier
2. The body is parsed into a structured Slack request using SlackRequestParser
3. The verified request is handed off to the app for handling via app.run()
4. The response from Slack Bolt is then mapped back to a standard ResponseEntity and returned

This setup handles all the foundational plumbing you’ll need — request validation, parsing, and forwarding to your core logic — while staying lightweight and fully compatible with Spring Boot.

With the bot setup out of the way, we can finally start building the actual logic and model. Thankfully, Slack provides a solid enough abstraction to make this part relatively straightforward:

@Service
public class ModalRenderer {

    public View renderModal(final String supportModalCallbackId,
                            final String channelId) {
        final var blocks = new ArrayList<LayoutBlock>();
        blocks.add(Blocks.input(input -> input
                .blockId(DESCRIPTION_BLOCK_ID)
                .label(PlainTextObject.builder().text("Describe the request").build())
                .element(RichTextInputElement.builder()
                                             .actionId(DESCRIPTION_ACTION_ID)
                                             .placeholder(PlainTextObject.builder().text("What do you need help with?").build())
                                             .focusOnLoad(true)
                                             .build())
                .optional(false)
        ));

        blocks.add(Blocks.input(input -> input
                .blockId(TEAM_BLOCK_ID)
                .label(PlainTextObject.builder().text("Which team should look at this?").build())
                .element(StaticSelectElement.builder()
                                            .actionId(TEAM_ACTION_ID)
                                            .options(Arrays.stream(Team.values())
                                                           .map(team -> OptionObject.builder()
                                                                                    .text(PlainTextObject.builder()
                                                                                                         .text(team.getMention())
                                                                                                         .build())
                                                                                    .value(team.getMention())
                                                                                    .build())
                                                           .toList())
                                            .build())
        ));

        ...

        return View.builder()
                   .type("modal")
                   .callbackId(supportModalCallbackId)
                   .title(ViewTitle.builder().type("plain_text").text("Support request").build())
                   .submit(ViewSubmit.builder().type("plain_text").text("Submit").build())
                   .close(ViewClose.builder().type("plain_text").text("Cancel").build())
                   .privateMetadata(channelId)
                   .blocks(blocks)
                   .build();
    }
}

The ModalRenderer service is responsible for rendering the support request form — essentially a modal view with a few input fields. Most of the code here is self-explanatory, but I’ll highlight two key parameters being passed in:

• supportModalCallbackId — kept private to the class that manages the main flow
• channelId — stored in privateMetadata so it can be accessed later by the class responsible for posting the final support message to the correct channel

Now that we have a form to render, the next question is: how do we actually show it to the user? From our earlier requirements, we know we want to integrate with Slack Workflows. To do that, you first need to add a new Workflow Step on Slack’s side:

On the Java side, you can define it with app.function(...). However — and here comes the first architectural surprise — a Workflow can’t trigger a modal directly. Only a real user action can open one.

While working on this bot, I ran into quite a few “interesting” design decisions from Slack — some made sense, others… not so much.
Sometimes I could work around them; other times, I had to redesign entire parts of the app.
I even opened an issue on GitHub to clarify a few things, but no reply so far.

After a lot of trial and error, I settled on a two-step implementation that worked reliably:

1. User starts the workflow
2. The bot sends an ephemeral message (visible only to the user) with a button to open the support request form

In code, this means setting up three main handlers:

• app.function — the main entry point that sends the ephemeral message with the “Open Form” button
• app.blockAction — triggered when the user clicks that button; this opens the modal form
• app.viewSubmission — fired when the form is submitted; this takes the input and posts a support request message in the target channel

@Service
public class RequestHandler {
    private static final String SUPPORT_MODAL_CALLBACK_ID = "support-request-modal";

    ...

    public void registerHandlers(final App app) {
        app.function("support-request", (req, ctx) -> {
            try {
                // inputs provided from Workflow builder
                final var inputs = req.getEvent().getInputs();
                final var channelId = inputs.get("channel_id").getStringValue();
                final var userId = inputs.get("user_id").getStringValue();
                ctx.client().chatPostEphemeral(r -> r
                        .channel(channelId)
                        .user(userId)
                        .text("Click below to open the support request form.")
                        .blocks(modalRenderer.renderEphemeralMessage(channelId, userId)));
            } catch (final IOException | SlackApiException e) {
                log.error("Error opening support request modal", e);
            }
            return ctx.ack();
        });

        app.blockAction(Action.SUPPORT.name(), (req, ctx) -> {
            try {
                final var channelId = req.getPayload().getActions().getFirst().getValue();
                final var triggerId = req.getPayload().getTriggerId();

                final var response = ctx.client().viewsOpen(r -> r
                        .triggerId(triggerId)
                        .view(modalRenderer.renderModal(SUPPORT_MODAL_CALLBACK_ID, channelId)));
                if (!response.isOk()) {
                    log.warn("Failed to open support request modal: {}", response.getError());
                }
                ctx.respond(r -> r.deleteOriginal(true));
            } catch (final IOException | SlackApiException e) {
                log.error("Error opening support request modal", e);
            }
            return ctx.ack();
        });

        app.viewSubmission(SUPPORT_MODAL_CALLBACK_ID, (req, ctx) -> {
            try {
                final var input = modalRenderer.parseSubmission(req.getPayload().getView().getState());
                final var userId = req.getPayload().getUser().getId();
                final var channelId = req.getPayload().getView().getPrivateMetadata();
                final var messageRequest = ChatPostMessageRequest.builder()
                                                                 .channel(channelId)
                                                                 .blocks(messageRenderer.renderBlocks(input, userId))
                                                                 .build();
                final var response = ctx.client().chatPostMessage(messageRequest);
                if (!response.isOk()) {
                    log.error("Failed to post support request message: {}", response.getError());
                    throw new IOException(response.getError());
                }
                return ctx.ack();
            } catch (final IOException | SlackApiException e) {
                log.error("Failed to create support request", e);
                return ctx.ackWithErrors(Map.of(SUPPORT_MODAL_CALLBACK_ID, "We hit an error while posting the request. Please try again."));
            }
        });
}

Remember that small line from earlier?

config.setAttachingFunctionTokenEnabled(false);

Here’s why it matters: By default, Slack uses a Workflow Function Token, which doesn’t always have permission to send messages or open modals.
Disabling that option ensures your bot uses the Bot Token instead — which has the correct permissions for these actions. Without it, your bot might fail to post messages or open forms, depending on the setup.

Afterwards

I hope this tutorial helps you kickstart your journey into Slack bot development. There’s still a lot more that could be covered — deployment pipelines, testing, persistence layer, action handling, and so on — but I wanted to keep this tutorial focused and practical rather than overwhelming.

What I really learned while building this bot is that Slack’s developer experience is powerful but quirky. Some design decisions make total sense once you understand the ecosystem; others you’ll just have to work around. But that’s what makes it fun — every challenge is an opportunity to learn something new (and maybe curse a little under your breath).

Continuous Integration and Continuous Deployment (CI/CD) have become foundational pillars of modern software development, enabling teams to ship code faster, safer, and more reliably. While Jenkins has long been the go-to solution, earning its place as a powerful, customizable, and battle-tested tool, the CI/CD landscape has evolved.

Today, cloud-native development demands more seamless integration and maintainability than traditional server-managed tools can provide. That’s where GitHub Actions comes into play — a modern CI/CD platform built directly into the version control system that has been quietly taking the DevOps world by storm. Its simplicity, combined with a vast ecosystem of reusable actions, makes builds and releases incredibly easy to implement, and even easier to maintain.

In this article, we’ll develop CI and release workflows using GitHub Actions for a traditional Spring Boot Java application running on Kubernetes.

Part 1. Project Setup and CI Action

For the sake of this tutorial, the most basic Spring Boot project was created using Spring Initializr. It doesn’t matter which build tool is used — Gradle would work just as well (you’d just replace mvn with the relevant gradle command).

Next, let’s create the CI workflow. You typically place it in the root of your project inside the .github/workflows folder. Here’s what the ci.yml file would look like:

name: CI

on: push

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up JDK
        uses: actions/setup-java@v4
        with:
          java-version: '24'
          distribution: 'corretto'

      - name: Cache Maven dependencies
        uses: actions/cache@v4
        with:
          path: ~/.m2/repository
          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
          restore-keys: |
            ${{ runner.os }}-maven-

      - name: Build with Maven
        run: mvn --batch-mode clean test

Short and concise, isn’t it?

While this workflow is mostly self-explanatory, let’s highlight a few key details:

• actions/checkout@v4: This checks out your code from the GitHub repository so it can be built and tested. It’s almost always the first step in any workflow.
• actions/setup-java@v4: This configures the correct version of the JDK. Here we use Amazon Corretto 24, but you can swap in any other supported distribution or version.
• actions/cache@v4: Caching ~/.m2/repository speeds up builds by avoiding repeated downloads of Maven dependencies. The cache key uses a hash of all pom.xml files, ensuring updates when dependencies change.
• mvn — batch-mode clean test: A clean, quiet Maven build that compiles your code and runs all unit tests. You can later extend this step with more complex goals or reporting tools.

It’s also worth noting that each step, like checkout, setup-java, or cache, is powered by a reusable GitHub Action, often open-sourced and versioned. These actions are typically written in JavaScript or TypeScript and can be customized or replaced if your project has specific requirements. If you ever hit a limitation, writing your own action is straightforward and well-documented.

Part 2. Release Action

With the CI pipeline in place, let’s move on to the next stage of the DevOps lifecycle: release. In this example, we’ll implement a rather simple release pipeline that deploys a single-instance Spring Boot service. While minimal, this pattern works perfectly well in microservices architectures, where each service is independently built, versioned, and deployed.

One principle worth emphasizing: start simple, then scale. Just because the industry pushes for multi-stage releases, Helm charts, or GitOps doesn’t mean your project needs them right away. A basic pipeline done right will take you far — complexity can always come later if your application demands it.

Here’s a GitHub Actions workflow for building and releasing a Docker image, versioning it, and deploying it to Kubernetes:

name: Release

on:
  pull_request:
    types: [ closed ]

env:
  # full docker image name without version
  IMAGE_NAME: us-docker.pkg.dev/project/repo/demo

jobs:
  release:
    if: github.event.pull_request.merged == true && github.base_ref == 'main'
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          # required for push back to main
          ssh-key: ${{ secrets.DEPLOY_KEY }}

      - name: Set up Java
        uses: actions/setup-java@v4
        with:
          java-version: '24'
          distribution: 'corretto'

      - name: Get current version
        id: get_version
        run: echo "::set-output name=current_version::$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)"

      - name: Build Spring Boot Docker image and push it
        run: |
          mvn --batch-mode spring-boot:build-image -DskipTests -Dspring-boot.build-image.imageName=$IMAGE_NAME:${{ steps.get_version.outputs.current_version }}
          gcloud auth configure-docker us-docker.pkg.dev --quiet
          docker push $IMAGE_NAME:${{ steps.get_version.outputs.current_version }}

      - name: Bump version
        run: |
          NEXT_VERSION=$((${{ steps.get_version.outputs.current_version }} + 1))
          mvn versions:set -DnewVersion=$NEXT_VERSION -DgenerateBackupPoms=false

      - name: Commit version
        run: |
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git commit -am "Release: ${{ steps.get_version.outputs.current_version }}"
          git push origin main

      # Don't forget to auth before applying
      - name: Deploy to k8s
        working-directory: deployment
        run: |
          sed -i "s|{{IMAGE_TAG}}|${{ steps.get_version.outputs.current_version }}|g" prod/deployment.yaml
          kubectl apply -f deployment.yaml

Let’s highlight a few key details of this workflow as well:

• Triggering on Merged PRs: This workflow is triggered when a pull request is closed. The if condition ensures it only runs on merges to the main branch, automatically treating merged PRs as release candidates.
• Version Detection: Instead of relying on Maven snapshots or complicated versioning plugins, we extract the current version from the pom.xml, and simply increment it for the next release. This keeps things clean and deterministic.
• Docker Image Build + Push: We use Spring Boot’s built-in Cloud Native Buildpacks via spring-boot:build-image to build OCI-compliant images. It’s fast, simple, and requires no Dockerfile.
• Kubernetes Deployment: The image tag placeholder ({{IMAGE_TAG}}) in the deployment manifest is replaced with the current version before being applied via kubectl.
• Committing Version Bump: After pushing the image, the workflow bumps the Maven version and commits it back to the main branch. This ensures the next PR starts from the correct version.
  There is one caveat though — given that most likely your main branch will be protected by the Rulesets so you cannot simply push the change. There is an open feature request to allow GHA to bypass such rules, but in a meanwhile you would need to configure a Deploy key following this instruction.

Maven is a powerful tool, but it’s versioning model (especially around SNAPSHOTs) can often feel unnecessarily complex for small teams or microservices. In this pipeline, we’ve adopted a simpler, incremental versioning model where every release is just a number. No snapshots, no guesswork.

Lastly, here’s what the Kubernetes deployment.yaml file looks like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo
  template:
    spec:
      containers:
        - name: demo
          image: us-docker.pkg.dev/project/repo/demo:{{IMAGE_TAG}}
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
          resources:
            requests:
              memory: "1Gi"
              cpu: "2"

This manifest defines a minimal but complete Kubernetes deployment for your Spring Boot application. It launches a single pod using the newly built and tagged Docker image. The {{IMAGE_TAG}} placeholder is dynamically replaced by the GitHub Action at release time, ensuring the correct version is deployed.

Afterwards

With this quick setup, you already have a functioning CI/CD pipeline: a merged PR automatically builds your application, tags and pushes the image, bumps the version, and rolls out the update to your cluster. Simple, clean, and production-friendly

As a long-time Jenkins user (and even a plugin developer), I’ve recently started migrating all my repositories to GitHub Actions, and for good reason. The integration into modern DevOps workflows is seamless, the setup is fast, and the learning curve is refreshingly low.

What we’ve covered here is just the beginning. GitHub Actions offers a rich ecosystem that supports: Conditional workflows based on pull request labels, Multi-language build pipelines, Secure, keyless authentication into cloud providers, Integration with tools like SonarQube, Snyk, and more.

If you’re doing DevOps — or moving toward GitOps — GitHub Actions is absolutely worth your time. Don’t sleep on it.

All of the code can be found at this GitHub repository.

When working with Java, there are times when you need to make use of native libraries written in other languages, like C++, Rust or even Go. Traditionally, Java Native Interface (JNI) has been the go-to solution for these tasks. However, JNI can be complex and cumbersome, requiring significant boilerplate code and deep knowledge of both Java and the target language’s memory management.

Java Native Access (JNA) offers a more streamlined and user-friendly alternative. With JNA, you can directly call native functions from dynamic libraries without the need for extensive coding or manual handling of pointers and memory. This ease of use makes JNA particularly appealing for developers who need to integrate native code into their Java applications but want to avoid the steep learning curve and pitfalls associated with JNI.

In this article, we’ll explore the practical use of JNA by demonstrating how to call methods from a Go library within a Java application.

Part 1. Building the Go Library

In this section, we’ll walk through the process of creating a simple Go library that we’ll later use in our Java application. The Go library will contain a Contains function that checks whether a specific string exists within a slice of strings:

package main

import "C"

//export Contains
func Contains(values []string, value string) bool {
 for _, v := range values {
  if v == value {
   return true
  }
 }
 return false

}

func main() {
}

The code might look simple and mostly self-explanatory, but there are a few key aspects to highlight.

• The import “C” statement is crucial when building Go libraries meant to be used with other languages. This import enables Go to interface with C, allowing us to create functions that can be called from C-compatible languages, like Java via JNA. Essentially, it bridges the gap between Go and the external world.
• //export Contains is more than just documentation — it instructs the Go compiler to export the Contains function, making it accessible to other languages, like Java in our case. The //export directive must be placed immediately before the function definition you intend to export.

This Go code is now ready to be compiled into a shared library that we can call from Java using JNA. In order to do that execute the following build command:

go build -buildmode=c-shared -o bin/library.so main.go

Under the hood it will use the cgo build command. You’ll end up with two important files: a shared object file (.so) and a C header file (.h). These files are essential for integrating the Go library with other programming languages like Java. For the sake of this article I’ve committed both of those files, but generally you shouldn’t do that.

The .so file is a shared object file that contains the compiled code of your Go library. It’s this file that Java, through JNA, will load and interact with. The shared object file is a dynamic library that can be loaded at runtime, making it possible for Java to call the Contains function defined in our Go code.

The .h file is a C header file generated by Go when you use cgo to build the library. This file serves as an interface between your Go code and other languages like C and, by extension, Java. You can check out the complete file over here, in this article however we will only focus on the key parts of it:

extern GoUint8 Contains(GoSlice values, GoString value);

This line declares the Contains function in a way that can be recognized by other languages. The function expects a GoSlice and a GoString as arguments, making it possible to pass Go-like slices and strings from Java to this function via JNA. That’s the function that we will be calling from the Java side.

typedef struct { const char *p; ptrdiff_t n; } _GoString_;

This line defines a type _GoString_, which is used to represent strings in Go when interfacing with other languages. Go strings are not simple null-terminated C strings; they include both a pointer to the string data p and the string’s length n. This ensures that strings are handled correctly even if they contain null bytes.

typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;

This line defines a type GoSlice, which is used to represent slices in Go. A Go slice includes a pointer to the underlying array data, its length len, and its capacity cap.

These structures are crucial because they ensure that data types like strings and slices, which are native to Go, can be correctly interpreted and used in other languages. Now let’s jump over to the Java side of this project.

Part 2. Building the Java Application

First we’ll create a Java interface called LibraryBridge that will serve as a bridge between our Java application and the Go library we previously created. This interface will define the structures and methods needed to interact with the Go code using JNA.

import java.util.List;
import com.sun.jna.Library;
import com.sun.jna.Pointer;
import com.sun.jna.Structure;

public interface LibraryBridge extends Library {
    class GoSlice extends Structure {
        public static class ByValue extends GoSlice implements Structure.ByValue {
        }

        public Pointer data;
        public long len;
        public long cap;

        @Override
        protected List<String> getFieldOrder() {
            return List.of("data", "len", "cap");
        }
    }

    class GoString extends Structure {
        public static class ByValue extends GoString implements Structure.ByValue {
        }

        public String p;
        public long n;

        @Override
        protected List<String> getFieldOrder() {
            return List.of("p", "n");
        }
    }

    boolean Contains(GoSlice.ByValue values,
                     GoString.ByValue value);
}

The LibraryBridge interface extends the Library interface provided by JNA, which enables us to load and interact with the native shared library generated from our Go code. Let’s break down the code:

• The GoString class represents the Go string structure in Java. It matches the _GoString_ struct in the Go library, which includes a pointer to the string data and the length of the string.
• The GoSlice class represents the Go slice structure in Java. This structure is defined to match the GoSlice type from the Go library, which includes a pointer to the data, the length of the slice, and its capacity.
• The ByValue nested class inside ensures that instances will be passed by value to the native function, mimicking the behavior of Go slices and strings.
• Both classes have the getFieldOrder method that returns a list of the field names in the order they are defined in the Go structure. JNA uses this method to correctly map the Java fields to their corresponding native fields.
• Lastly, Contains method corresponds to the Contains function in our Go library. The method takes two parameters: a GoSlice.ByValue representing the slice of strings, and a GoString.ByValue representing the string to search for within the slice. The method returns a boolean indicating whether the string was found in the slice.

Now that we’ve established the LibraryBridge interface to interact with the Go library, the next step is to create a class that will load this library into the Java application. This class will handle the process of extracting and loading the shared library (library.so) at runtime:

import java.io.IOException;
import com.sun.jna.Native;

public class LibraryLoader {
    public static LibraryBridge INSTANCE;

    static {
        loadLib();
    }

    private static void loadLib() {
        try {
            final var file = Native.extractFromResourcePath("library.so", LibraryLoader.class.getClassLoader());
            INSTANCE = Native.load(file.getAbsolutePath(), LibraryBridge.class);
        } catch (final IOException e) {
            throw new RuntimeException(e);
        }
    }
}

In this LibraryLoader class, the Go library is loaded statically using a static initializer block. By doing so, we ensure that the Go shared library is loaded as soon as the LibraryLoader class is accessed, typically at the start of the application. This guarantees that the native methods provided by the library are available and ready to be invoked throughout the application’s lifecycle.

In the loadLib() method, we use the Native.extractFromResourcePath() function to load the library.so file. This method works seamlessly with the Maven resources folder structure, which is a common setup in Java projects. With Maven one typically stores resource files, such as native libraries, in the src/main/resources directory. When you build your project, Maven includes these resources in the classpath, making them accessible at runtime. The Native.extractFromResourcePath(“library.so”, LibraryLoader.class.getClassLoader()) function extracts the library.so file from the application’s resources and places it in a temporary directory where it can be accessed by the operating system. This is particularly useful because it allows the native library to be packaged within the JAR file, and JNA handles the extraction process automatically. This approach ensures that your native library is always accessible, regardless of the environment in which your application is deployed. Whether running locally during development or as a packaged application in production, Native.extractFromResourcePath simplifies the process of managing native libraries, making them easy to distribute and use.

Finally, we’ll implement the actual method that interacts with our Go library through the LibraryBridge interface. This method will leverage JNA to convert Java data types into Go-compatible types, call the Go Contains function, and return the result. Below is the implementation, followed by a detailed explanation.

import com.sun.jna.Memory;

public class Main {

    public boolean containsGo(final List<String> valuesToCheck, final String valueToCheck) {
        final var values = convertValues(valuesToCheck);
        final var size = values[0].size();
        try (final var memory = new Memory((long) values.length * size)) {
            var offset = 0L;
            for (final var value : values) {
                value.write();
                memory.write(offset, value.getPointer().getByteArray(0, size), 0, size);
                offset += size;
            }

            final var valuesAsSlice = new LibraryBridge.GoSlice.ByValue();
            valuesAsSlice.data = memory;
            valuesAsSlice.len = values.length;
            valuesAsSlice.cap = values.length;

            return LibraryLoader.INSTANCE.Contains(valuesAsSlice, getGoString(valueToCheck));
        }
    }

    private static LibraryBridge.GoString[] convertValues(final List<String> valuesToCheck) {
        final var values = (LibraryBridge.GoString[]) new LibraryBridge.GoString().toArray(valuesToCheck.size());
        var i = 0;
        for (final var pair : valuesToCheck) {
            values[i].p = pair;
            values[i++].n = pair.length();
        }
        return values;
    }

    private static LibraryBridge.GoString.ByValue getGoString(final String value) {
        final var valueAsGoString = new LibraryBridge.GoString.ByValue();
        valueAsGoString.p = value;
        valueAsGoString.n = valueAsGoString.p.length();
        return valueAsGoString;
    }
}

The first step is to convert the list of Java strings (valuesToCheck) into an array of GoString structures, which can be passed to the Go function. This is achieved through the convertValues method. This method creates an array of LibraryBridge.GoString structures, where each GoString contains a pointer to a Java string p and its length n. This array mirrors the Go slice structure, making it compatible with our Go library.

After this we need to build a “slice” with the values to send to Go. The Memory object is used to allocate a block of memory that can store all the strings in the Go slice. This memory block is needed because Go expects a continuous block of memory for its slices, and JNA facilitates this process.

Each string in the array is written into the allocated memory block. The offset is updated after each string to ensure that each string is stored sequentially in memory. After writing the strings to memory, we create an instance of LibraryBridge.GoSlice.ByValue. We set the data field to point to the memory block, and set both len and cap fields to the number of elements in the slice. This structure mirrors the Go slice in a way that the Go library can understand.

With all of that we are now ready to call the Contains function using the static reference to the LibraryBridge instance.

Part 3. Testing

There are various ways to test this code, in this article we’ve decided to utilise the power of JUnit tests:

import java.util.List;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

public class MainTest {

    @Test
    void testContains() {
        final var valuesToCheck = List.of("value1", "value2", "value3");
        final var valueToCheck = "value2";

        final var main = new Main();
        final var contains = main.containsGo(valuesToCheck, valueToCheck);

        Assertions.assertTrue(contains);
    }

    @Test
    void testDoesNotContain() {
        final var valuesToCheck = List.of("value1", "value2", "value3");
        final var valueToCheck = "missing";

        final var main = new Main();
        final var contains = main.containsGo(valuesToCheck, valueToCheck);

        Assertions.assertFalse(contains);
    }
}

With this simple test suite we can assure that the code works correctly and that the Go library is returning expected results.

Afterwards

Integrating Java with native libraries, especially those written in Go, can seem daunting at first. However, as we’ve demonstrated, JNA significantly simplifies this process. With JNA, you can bypass much of the complexity typically associated with JNI, allowing you to focus more on functionality and less on boilerplate code. The relative ease of use that JNA offers makes it an appealing choice for developers looking to extend Java applications with the performance and capabilities of native code.

That said, while JNA is powerful, it’s not without its challenges. Real-world interactions between Java and native libraries often involve dealing with complex data structures, memory management, and the intricacies of different language ecosystems. JNA can abstract away much of this complexity, but it’s important to remember that it doesn’t eliminate it entirely. Issues like performance overhead, debugging difficulties, and the potential for subtle bugs when dealing with pointers and memory are still very much present. Therefore, JNA should be used wisely and where it is appropriate

All of the code can be found at this GitHub repository.

Ah, Jenkins. The venerable old guardian of the CI/CD realm, a tool that has seen the rise and fall of many a software trend, yet stands steadfast — much like that one quirky relative we all have who’s stuck in their ways but somehow indispensable at family gatherings. Jenkins, with its infinite configurability and the charming allure of its plugins is both adored and admonished. It’s the seasoned performer that can pull off any trick in the book, albeit with a rehearsal or two. Jenkins doesn’t just survive in the ever-evolving CI/CD landscape; it thrives, continuing to be a dominant player that commands both respect and a healthy dose of developer memes.

Indeed, the diverse opinions on Jenkins reflect its complexity and versatility in the CI/CD landscape. Amid this backdrop, I’ve contributed into the ecosystem by releasing a Jenkins plugin named pipeline-gcp. In this article I’ll share a high-level overview of the process.

While Jenkins provides comprehensive documentation on plugin creation, navigating through it can be somewhat challenging, as the information is dispersed across various pages. Moreover, sharing insights from a real developer’s experience can be invaluable, especially for first-time contributors looking to make their mark.

Conceptually, there are 3 stages that you will have to go through while creating a plugin: Development, Hosting request and, if accepted, Release.

Development

This phase is perhaps the most extensive yet straightforward part of the plugin creation process. The journey typically begins by setting up an empty project. Jenkins makes this step user-friendly by offering a convenient archetype, which you can initialize with the following Maven command:

mvn -U archetype:generate -Dfilter=io.jenkins.archetypes:

With your project created, the next step is to start actual development. In my experience, the most effective way to figure out what to build — specifically, what Java code to write — is to draw inspiration from existing plugins or to leverage a modern AI tool for guidance. For example, for my plugin I looked to pipeline-aws, a similar plugin designed for AWS. This approach helped me craft the first functional prototype of my plugin.

Next let’s turn our attention to one of the more deceptively challenging aspects of programming: naming. While the archetype efficiently lays the groundwork for your project, it’s essential to adhere to the established naming conventions when choosing the plugin name. Specifically, the artifactId should not include the word plugin, although it’s customary for the repository name to contain it. But if the plugin is accepted into jenkinsci, the repository name will incorporate plugin by default as you can see it in my plugin name at https://github.com/jenkinsci/pipeline-gcp-plugin.

An often overlooked yet critical factor in a plugin’s success is its documentation. For your plugin to be embraced and utilized by the Jenkins community, developers must have clear instructions on its usage. The most effective way to achieve this is by creating a comprehensive and easy-to-understand README file. The importance of this document cannot be overstated: it not only serves as the first point of contact for potential users but also lays the foundation for your plugin’s representation on https://plugins.jenkins.io once released. Therefore, ensure that the README has all the details on how to use the plugin, highlighting its features, setup instructions, and any prerequisites. Remember, a well-documented plugin significantly lowers the entry barrier for new users and increases its potential impact within the community.

As the plugin development progresses, you might ponder on enhancing the user interface, particularly how to incorporate text descriptions or functionalities within the Plugin Manager or Pipeline Syntax views. Jenkins employs Jelly scripting for UI customization in plugins. To add a plugin description visible in the Plugin Manager, you’ll need to include an index.jelly file in the resources directory. Here’s an example structure:

<?jelly escape-by-default='true'?>
<div>
   This plugin adds Jenkins pipeline steps to interact with the GCP API.
</div>

After that the description will be displayed as:

You would also use the same approach to define the pipeline syntax for your plugin functions, particularly for steps that accept parameters. To achieve that you should add config.jelly and help.html files to the resources folder, aligned with the Java class path of the given function. Here’s how you could structure these files:

<?jelly escape-by-default='true'?>
<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:f="/lib/form">
   <f:entry field="credentialsId" title="Credentials Id">
       <f:textbox/>
   </f:entry>
</j:jelly>

<div>
   The id of credentials, presumably stored as Secret file
</div>

Now if you navigate to the Pipeline Syntax view (Jenkins/job/<anyjob>/pipeline-syntax), you should see a proper description and a generated block:

To wrap up the development phase and before submitting the hosting request, it’s crucial to ensure that your plugin meets all necessary quality checks. Execute the following Maven command and make sure it passes:

mvn clean verify package

Jenkins projects typically employ the Spotless plugin to enforce consistent code formatting. Successfully passing this command indicates your code adheres to the required standards and is free from obvious issues.

With all the steps covered, and of course with the actual code implemented and tested, you are now ready to submit a hosting request.

Hosting request

To kick-start the hosting review for your plugin, head over to the repository-permissions-updater and create a new issue, detailing all necessary information. If you’re looking for a point of reference, feel free to draw inspiration from my submission.

The process involves clearing two automated checks — security and hosting — before moving to the manual review by a member of the hosting team. In my experience, this phase was actually quite swift; after addressing the review comments, my hosting request received approval within just two days. Following approval, you’ll be welcomed into the Jenkins organization via email, and your plugin repository will be officially forked into jenkinsci.

Additionally, you’ll be granted owner rights to your repository. This is a perfect time to fine-tune its settings, including merge strategies, pull requests, branch protections, and more, to suit your preferences.

A corresponding permissions file is typically auto-generated in the repository-permissions-updater, although in my case the pipeline was broken (fixed in this PR) so I had to create it manually. This step marks your exciting entry into the vibrant Jenkins community!

However, the journey doesn’t end here. The next step is to release your plugin into the wild, making it available for users worldwide.

Release

Gone are the days when plugin releases were manually executed from local development machines. In a fortunate turn of events, this process has been modernized and is now largely automated through GitHub Actions, with just a few adjustments to the project’s configuration needed:

• Submit a modification to the repository-permissions-updater to enable CD for your plugin.
• Remove release-drafter.yml files
• Add a cd.yml workflow file
• Insert -Dchangelist.format=%d.v%s into the maven.config file
• Remove revision from pom.xml and change changelist to 999999-SNAPSHOT

These adjustments were implemented in my project through a couple of PRs, detailed here and here. For those seeking a more comprehensive explanation, the official Jenkins documentation is an invaluable resource.

Following these modifications with the right labels in place, merging a PR will trigger the release of a new plugin version.

Keep an eye on Artifactory to see your plugin listed, and don’t forget to verify its presence on the Jenkins plugins page. It should only take a few hours to reflect the updates, but in my case there was yet another bug so the waiting time was quite long. It is resolved now, so hopefully you won’t experience anything like that.

Afterwards

Congratulations on successfully releasing your plugin into the Jenkins ecosystem! As you join the ranks of developers enhancing this powerful tool, remember that your efforts help in shaping the future of CI/CD processes. Here’s to many more contributions and continued collaboration within this dynamic community!

In the ever-evolving landscape of web development, choosing the right tools can make all the difference. Rust, renowned for its performance, safety, and concurrency, emerges as a powerhouse in this domain. Its unique blend of low-level control and modern language features has made it a favourite among developers seeking efficiency without compromising on safety.

In this tutorial, with the help of Rust, Yew, Actix Web, and Sqlx we’ll add an essential piece any website will need at some point — search functionality.

Prerequisites

I’m using Rust version 1.75 and Yew version 0.21. This tutorial assumes prior knowledge of Rust and Yew and focuses more on practical usage of the language and the framework.

I’ll be using the project I’m actively working on at the moment — illuvi-analytics.com. The relevant parts of it will be stripped into self-sufficient code snippets, and you are welcome to check the entire project out.

If you are not yet familiar with Yew, take a look at one of my previous articles — Rust: Building Web Frontend with Yew

There are 3 parts in this tutorial: Problem statement, Backend and Frontend

Background

If you’re intrigued by the project and the broader scope of web3 technologies, I recommend exploring the Background section in one of my previous articles — Rust: How to consume REST API. This will give you a solid foundation in the concepts we’re building upon.

While I employ a domain-specific data model in this tutorial, the concepts and techniques discussed are universal. Therefore, you can seamlessly adapt and apply this approach to your projects, regardless of your specific interest or expertise in the finer details of the domain in question.

Part 1. Problem statement

At first glance, the task seems simple: add a search box to the website, ideally in the header, allowing users to input their queries. However, the simplicity is deceptive, as deeper exploration reveals numerous edge cases that require careful consideration.

Frontend considerations
There are several critical design decisions to make that will define the overall user experience of the search functionality, making it user-friendly and visually appealing.

First, consider the interaction style of the search box. Will it trigger a search automatically as the user types, or will it require pressing “Enter”? This decision impacts the immediacy of the user’s search experience.

Next, decide how to present the search results. Options include displaying them in a popup window, on a dedicated results page, or a combination of both. This choice significantly influences the user’s navigation flow and ease of accessing the information.

Lastly, determining the number of results to fetch and display is crucial. This involves balancing the need to provide comprehensive information with maintaining a clean and uncluttered interface.

In my specific use case, I opted for a dynamic approach. The search box activates as the user begins typing, displaying results in a pop-up. This method offers immediate feedback and a seamless user experience, keeping the user engaged without navigating away from their current page.

Backend considerations
The backend receives an arbitrary search string, which must be safely used for querying multiple database fields. This scenario raises immediate concerns about SQL injection attacks and the need for robust database protection. My experience with SQLX in Rust has shown its effectiveness in guarding against such vulnerabilities, primarily through its support for prepared statements. For more details, refer to the SQLX documentation.

Speed and efficiency are paramount in search functionality. Initially, optimizing the search query is crucial. In databases like Postgres and MySQL, utilizing EXPLAIN ANALYZE with your query helps identify execution bottlenecks and areas for improvement. Strategies might include adding indexes to search fields, separating certain fields into distinct columns (particularly with JSON data), or creating views or materialized views. Depending on your specific requirements, you might also consider parallelizing queries, introducing a caching layer or even reevaluating your choice of database technology to better suit your needs.

Having outlined the key considerations for both the backend and frontend, we now have a solid foundation to begin the development phase. Let’s dive into the development process, starting with the backend part.

Part 2. Backend

We begin by defining a controller for our search functionality using actix-web. As we discussed previously, we will have a single search value sent from the frontend, which will be defined in a struct:

An important aspect of our backend is the use of a database pool, rather than opening and closing connections for each request. You can see it being passed as a parameter in the get_search_results method. This approach, implemented in the main method, significantly enhances query performance and reduces database overhead:

The create_pool function configures the database connection with essential parameters like host, port, and credentials. You might tweak many other parameters here as well:

You might recognize that env_utils from my previous tutorials:

Next, let’s create a database querying service:

In my case here the search works on two fields — integer token_id and json metadata with a field called name in it.

A noteworthy aspect of our query implementation is the concatenation technique used in wildcard searches (‘%’ || $1 || ‘%’). This approach is required due to the way parameter binding works.

A small but significant optimization is applied to the token_id field. Recognizing that token_id is a numeric field, we first verify the search input as a number before initiating a more resource-intensive wildcard search.

There were a few other improvements considered, but eventually discarded:

• Creating an index on the metadata.name field did not result in notable performance improvements and was therefore set aside.
• Extracting metadata.name into its own column and using a database trigger for updates. While this approach did reduce the response time for negative searches from around 3 seconds to under 2 seconds, it was not implemented due to its limited necessity for my website.
• Implementing a caching read-through layer was also explored. However, given its complexity it was an excessive measure at this point. It remains an area of interest for future enhancements though.

Finally, the data models SearchData and AssetContentData are defined. These models structure the data sent from the backend to the frontend, ensuring a consistent and predictable format for the search results. The file itself will be defined in a shared module used by both frontend and backend:

With that the backend part is now complete and we go ahead and implement the search box on the frontend.

Part 3. Frontend

In this section we’re combining the capabilities of Yew and Bootstrap 5 to build the search interface. While Yew is a robust framework, it’s important to note that unfortunately it often requires verbose code, particularly in complex scenarios like this one where there are multiple states to manage.

The frontend implementation can be logically divided into three main tasks:

1. Handling user input in the search box
   Our first challenge is to efficiently handle user input in the search box. Unlike the straightforward approach in JavaScript, Yew requires a more nuanced method. To prevent sending each keystroke to the backend immediately, we’ll utilize the gloo-timers crate for timeout functionality.
2. Querying the backend and receiving results
   After capturing the user’s input, it’s sent to the backend. This process involves using the reqwest crate to manage the data transfer.
3. Displaying the search results
   The final step involves presenting the results to the user. For an optimal user experience, we’ll display the results in a dropdown view. During the data retrieval process, a spinner will indicate loading to the user, maintaining engagement. While the creation of the spinner is outside the scope of this tutorial, its presence is crucial for a seamless user experience.

The end result Search function component will look like this:

As you can see in the code above there are 4 states defined:

• search: Captures the user’s input.
• search_data: Holds data retrieved from the backend.
• typed_state: Indicates whether the user has entered any input.
• timeout_state: Manages the timing to send a backend request, ensuring it’s triggered only after the user stops typing.

To enhance user experience, five distinct actions (callbacks) were implemented:

• oninput: Manages user input. This action uses the gloo-timers crate’s Timeout struct to delay updating the search state which is used as a dependency in use_effect_with dependency therefore delaying the backend request
• onsubmit: Prevents page reload on form submission (Enter key press) within an HTML form context.
• onfocusout: Automatically closes the popup when the user clicks outside the search.
• onfocusin: Reopens the popup when the user clicks back into the search box.
• onclick: Resets the state upon clicking a search result, typically leading to a redirect to the relevant page.

The verbosity in our Yew implementation arises primarily from the need for each callback to clone the state, resulting in numerous cloned objects. The rest of the code follows standard Yew practices, enhanced with Bootstrap 5 classes for aesthetic appeal.

For those interested in exploring the complete project, the code is available on GitHub. While some parts are specific to my project, the general principles and techniques are applicable across various Yew applications.

The final result will look like this:

Afterwards

Yew, despite being a newer player in the web framework arena, is fully capable of powering dynamic and functional websites with Rust’s robustness. This journey through creating a search functionality showcases Yew’s potential in real-world applications. It demonstrates that this framework can be used to build complex features with the efficiency and safety that Rust offers. I believe that with time it will mature and become the next big thing in web development!

Do check out my Yew-powered website https://illuvi-analytics.com

Support

If you like the content you read and want to support the author — thank you very much!
Here is my Ethereum wallet for tips:
0xB34C2BcE674104a7ca1ECEbF76d21fE1099132F0

Amazon Web Services has long been the go-to destination for startups and enterprises alike looking to launch new projects swiftly and efficiently. The allure lies in its extensive array of managed services, designed to streamline the development process and allow businesses to focus on their core competencies. However, as time passes and your project matures, the financial realities of operating within the AWS ecosystem may start to emerge. The convenience of AWS-managed services comes at a cost, and what begins as an economical and expedient choice can gradually evolve into a budgetary challenge. Scaling resources vertically and horizontally, while maintaining the performance and reliability expected by your users, can become a substantial financial burden.

For some, DynamoDB — the NoSQL database service provided by AWS — has been the backbone of their data storage needs. Coupled with DynamoDB Accelerator (DAX), it seemed like the perfect solution for fast, scalable, and fully managed databases. However, the reality of operating DynamoDB and DAX at scale has shown that they may not always live up to their promises of reliability and cost-effectiveness.

In this article, we embark on a journey to explore an alternative: ScyllaDB, open source edition. From cost considerations to reliability concerns, we’ll uncover the compelling arguments that drive this shift in database strategy, and how it might benefit your long-term operations. Apart from this, I’ll also show you how to deploy your very own ScyllaDB cluster in AWS.

Part 1. Why switching?

In the realm of software engineering, the saying “if it works, don’t touch it” is often mentioned. It’s a principle that makes perfect sense — why tamper with a well-functioning system? However, in the dynamic landscape of modern technology, staying agile and cost-effective is essential. This often necessitates considering alternatives, even when you have a functional solution in place.

For those of us who’ve embarked on this journey of optimization, AWS offers a valuable tool: the AWS Pricing Calculator. It serves as our compass when navigating the complex terrain of cloud service costs. Let’s illustrate this with a quick cost calculation example.

In our DynamoDB scenario, we’re dealing with a workload that ranges from 1000 to 5000 write/read operations per second, supported by three DAX nodes of r5.large size.

Now, let’s compare this with a self-hosted solution where you’d manage three EC2 i3.large instances, each equipped with ScyllaDB.

The difference is nothing short of staggering.

Of course, the above is a simplified example, reflecting minimal instance types and counts. With ScyllaDB, the cost curve tends to be linear; scaling often means a straightforward doubling of expenses. DynamoDB and DAX, however, present a more intricate cost structure — try experimenting with it in the calculator.
It’s worth noting that while ScyllaDB does require investments in setup and potentially ongoing maintenance, the same holds true for DynamoDB and DAX however. In my personal experience, spanning the past two years, there have been seven (!) production incidents of different scale involving the AWS-managed solution. While individual experiences may vary, based on my own journey, I’ve come to strongly recommend exploring alternatives to DynamoDB and DAX.

Now that we’ve addressed the “why,” let’s delve into the technical aspects of the deployment process.

Part 2. Building AMI

Before we can launch our cluster, the first step will be the creation of an Amazon Machine Image (aka AMI) to run these EC2 instances. ScyllaDB provides prebuilt AMIs, ready to launch directly from the AWS console. However, you might find it necessary to install additional packages on these AMIs, such as Datadog or SSM agents. To accomplish this, we’ll begin by creating an Ansible playbook that installs ScyllaDB and sets up the required configuration files.
Notably, ScyllaDB is versatile when it comes to supported operating systems, with the recommended one being Ubuntu 22.04 at the time of writing. But bear in mind that the landscape can change, so always refer to the latest ScyllaDB documentation for the most up-to-date information.

Here’s the Ansible playbook, main.yaml, which accomplishes this task:

The playbook itself is rather straightforward. Let’s focus on the essential configuration file, scylla.yaml, and the pivotal scripts needed to bootstrap the database.

The scylla.yaml file serves as the central hub for tailoring ScyllaDB’s behavior. Below is a portion of the configuration file with key settings:

Depending on your use case you might need to set up more (or less) configurations. Make sure to configure Ec2Snitch since we are running in the AWS environment!

We will have a couple of bash scripts in order to facilitate the bootstrap process. These scripts ensure that ScyllaDB is properly configured and initialized.

A cluster requires at least one seed node. Keep in mind that the scylla_setup script configuration is something you might need to adjust as per your specific use case.

Every other node will be bootstrapped with a reference to the seed node. Make sure that scylla_setup configurations are identical!

The database schema should be set up on the seed node only. All other nodes will “receive” it while joining the cluster.

Finally setup.cql is the CQL script to set up the initial state of DB. It would usually contain the creation of a keyspace and tables in it. Sample file is below:

As you can see we replace <DATACENTER> here with the region (but without a “-1” part). You can read more about this here.

Even though Bash scripts might seem old-school in today’s tech landscape, we use them here for a straightforward reason: they make things simple and reliable. These scripts automate the essential configurations, ensuring a reliable ScyllaDB deployment. However, if the prospect of dealing with Bash scripts feels antiquated or you prefer a more hands-off approach, there’s an alternative: ScyllaDB offers a managed service that handles these tasks for you, albeit for a fee.

Now that we have our AMI ready, it’s time to proceed with deploying it in the AWS environment, which we’ll explore in the upcoming section.

Part 3. Deploying cluster

Without further ado, let’s jump straight into the CloudFormation template.

Let’s get an overview of the key sections in the CloudFormation template:

• ScyllaInterNodeSecurityGroup: This is a security group for enabling communication between ScyllaDB nodes. It is crucial to have the required ports opened in the security group, otherwise the nodes won’t be able to communicate.
• ScyllaDBSeedInstance: An EC2 instance serving as the seed node for the ScyllaDB cluster. It runs the setup-seed-node.sh script during initialization.
• ScyllaDBAutoscalingGroup: An Auto Scaling group that manages the ScyllaDB not seed nodes. It runs the setup-not-seed-node.sh script during initialization.

Make sure to have the DependsOn attribute correctly configured — AutoScalingGroup relies on the seed node to be up and running with ScyllaDB bootstrapped.

The CloudFormation template allows customization by providing parameters like VPC ID, subnet IDs, AMI ID, and instance type. You can configure as many regions as you need like this:

As you might have noticed the given template will only launch two instances — one seed and one not seed. Bootstrapping a cluster might be a very time consuming activity, but it can be mitigated with automation — a Jenkins job to deploy the cluster and also upscale it to 3 instances in total.

Due to the way ScyllaDB replicates data the cluster can only be increased one node at a time. By automating this process, you can ensure that nodes are started sequentially, maintaining data consistency and cluster health. You can extend the Jenkins job to add more nodes, but in this case don’t forget to add the checks to guarantee that all ScyllaDB nodes are up and running.

Afterwards

In the ever-evolving world of cloud computing, the allure of AWS managed services can be irresistible. They offer convenience, scalability, and reliability for your projects. However, as we’ve explored in this article, it’s essential to keep a watchful eye on costs.

While AWS services can kickstart your project with ease, their pricing models may not be as friendly to your budget in the long run. In the pursuit of efficiency and cost-effectiveness, it’s worth considering open-source alternatives.

Embracing open-source solutions like ScyllaDB may require a bit more setup and ongoing maintenance, but the benefits can be substantial. You gain more control over your infrastructure, avoid unforeseen cost spikes, and often achieve better performance.

So, the next time you embark on a cloud journey, remember that striking the right balance between managed services and open source can be the key to success. It might be “harder” in some aspects, but in the end, it’s a strategic move that can yield superior results for your projects and your budget alike.

Support

If you like the content you read and want to support the author — thank you very much!
Here is my Ethereum wallet for tips:
0xB34C2BcE674104a7ca1ECEbF76d21fE1099132F0