@@ -10,21 +10,18 @@ team: Core Platform
10
10
author : rtyler
11
11
---
12
12
13
- Deploying small services in containers allowed us to easily replace a production service
14
- with a weekend project written in Rust.
15
-
16
- The Core Platform team maintains a project to provide an entry point for
17
- [ Fastly] ( https://fastly.com ) access logs to be written into
18
- [ Apache Kafka] ( https://kafka.apache.org ) .
19
-
20
- Our initial rollout of "View Analytics" used [ rsyslogd] ( https://www.rsyslog.com ) for the ingestion service,
21
- “rsyslog-kafka.” Using rsyslogd worked fairly well, but had a couple of significant
22
- downsides.
23
-
24
- Last month, we open sourced and deployed "rsyslog-kafka"'s replacement: a daemon
25
- that provides better metrics, JMESPath rules, and RFC compliant message parsing:
26
- [ hotdog] ( https://github.com/reiseburo/hotdog ) 🌭.
27
-
13
+ When we set our goals at the beginning of the year "deploy Rust to production"
14
+ was not among them, yet here we are. Our pattern of deploying small services in containers
15
+ allowed us to easily bring Rust into production, replacing a difficult to
16
+ manage service in the process. In January, the Core Platform team started working on a
17
+ project called "View Analytics". The effort was primarily to replace an aging
18
+ system which was literally referred to as "old analytics." As part of the View
19
+ Analytics design we needed to provide an entry point for [ Fastly] ( https://fastly.com ) to relay access logs as syslog
20
+ formatted messages which could then be written into [ Apache Kafka] ( https://kafka.apache.org ) , driving the entire
21
+ View Analytics data pipeline. Our initial rollout shipped an [ rsyslog] ( https://www.rsyslog.com ) -based solution
22
+ for the “rsyslog-kafka” service.. Using rsyslogd worked fairly well, but had a
23
+ couple of significant downsides. Last month, we deployed its replacement: a
24
+ custom open source daemon written in Rust: [ hotdog] ( https://github.com/reiseburo/hotdog ) 🌭.
28
25
29
26
(** Note:** _ This specific use-case was well suited to Rust. That does not mean
30
27
that anything else we do at Scribd should or will necessarily be written in
0 commit comments