Chiron log reduction on Palo Alto firewall logs: test results
40.5% log byte reduction without losing or migrating any data. For an already identified reduction strategy, the Cribl pack, we achieved another 21 points of reduction.
ChironVision · October 2026
Why we ran it
Every log owner faces the same decision point: reduce TCO, or keep developer experience and compliance. Every tool on the market makes this decision in the same way, by removing fields, sampling events, or routing data out of Splunk. All have hidden costs: less intelligence to draw from, a second store with a query bill, a compliance gap. Chiron makes this decision differently. Chiron merges related lines in flight into one record that Splunk indexes and searches as before. Chiron unrolls it back to the original lines when needed. Nothing is cut, nothing is routed. The index still shrinks. We ran Palo Alto firewall logs through Splunk twice, first time raw and second time merged, to see how much. 40.5% fewer bytes with every line kept. On top of the Cribl pack’s trimming, another 21 points.
How we did things
In our case, Chiron sits in line between the forwarders and Splunk and accepts the input stream of log entries. It then pushes the merged stream of logs directly to the output pipeline. In particular, the Chiron Streaming Engine aggregates the related log lines into a single, lossless entry and does not drop or duplicate any record. We compared this approach to running the exact same logs through the raw pipeline where there were no aggregated records at all.
Finding #1: Indexing fewer bytes; keep everything. Splunk indexes 40.5% fewer bytes.
Results: In total, approximately 40.5% fewer bytes were indexed and approximately 60% fewer events were indexed when using Chiron to aggregate the logs. As indicated in Chart 1, TRAFFIC bytes came down by approximately 44% and THREAT bytes by approximately 32%. Using the same sample set, turning off field deletions for the Cribl Pack resulted in saving only .04%, or essentially nothing.
How the merge works
Easier shown than told. Three TRAFFIC end lines, one firewall, same second, same rule, same app. Values are synthetic.
1,2026/10/22 09:14:07,013201001234,TRAFFIC,end,2561,2026/10/22 09:14:07,10.20.4.17,198.51.100.23,0.0.0.0,0.0.0.0,allow-web,,,ssl,vsys1,trust,untrust,ethernet1/2,ethernet1/1,fwd-splunk,,184233,1,51844,443,0,0,0x19,tcp,allow,7412,3310,4102,22,2026/10/22 09:13:49,18,computer-and-internet-info,…
1,2026/10/22 09:14:07,013201001234,TRAFFIC,end,2561,2026/10/22 09:14:07,10.20.4.88,203.0.113.9,0.0.0.0,0.0.0.0,allow-web,,,ssl,vsys1,trust,untrust,ethernet1/2,ethernet1/1,fwd-splunk,,184241,1,51902,443,0,0,0x19,tcp,allow,3188,1420,1768,14,2026/10/22 09:14:01,6,computer-and-internet-info,…
1,2026/10/22 09:14:07,013201001234,TRAFFIC,end,2561,2026/10/22 09:14:07,10.20.4.51,198.51.100.23,0.0.0.0,0.0.0.0,allow-web,,,ssl,vsys1,trust,untrust,ethernet1/2,ethernet1/1,fwd-splunk,,184247,1,52117,443,0,0,0x19,tcp,allow,12940,5106,7834,31,2026/10/22 09:13:40,27,computer-and-internet-info,…Sixty-nine columns, and across these three lines most of them never change. So: write the constant columns once, write each differing column as all three values in line order. Same layout as the original.
1,2026/10/22 09:14:07,013201001234,TRAFFIC,end,2561,2026/10/22 09:14:07,10.20.4.17|10.20.4.88|10.20.4.51,198.51.100.23|203.0.113.9|198.51.100.23,0.0.0.0,0.0.0.0,allow-web,,,ssl,vsys1,trust,untrust,ethernet1/2,ethernet1/1,fwd-splunk,,184233|184241|184247,1,51844|51902|52117,443,0,0,0x19,tcp,allow,7412|3188|12940,3310|1420|5106,4102|1768|7834,22|14|31,2026/10/22 09:13:49|2026/10/22 09:14:01|2026/10/22 09:13:40,18|6|27,computer-and-internet-info,…,3What the searches did
Searches require the least amount of explanation. Each value within a merged line will be indexed independently as if it were its own individual value, therefore searching for 10.20.4.88 would find the merged line in exactly the same manner that was done prior to merging the data. All searches that include counts or sums have an additional macro to unroll the merged columns back into individual lines before performing the statistics operations.
Dashboards and alerts fall under the same macro, as underneath both are saved searches. As Splunk performs field extraction during search time (schema on read) nothing gets re-indexed. Likewise, the unroll will be performed in addition to the original field extraction. In practice, Chiron’s Splunk app includes the macro and the field split, most of the searches continue unchanged while the others have one line added only once.
Performance-wise, there was an acceptable mixed picture. CPU cost increased search time when using the macro unroll. Still, searches that scan the entire index had better performance by anywhere from 5% to 43%, as Splunk reads approximately 60% fewer events than would otherwise exist. This greatly outweighs the overhead of the unroll. Sub-second searches that first hit a term filter were generally a wash, with some being a couple milliseconds faster and others slower. The macros are designed for the merged format, which limits the overhead, but you may see a small penalty for short searches and gains on longer ones.
All 18 searches came back with the same rows on both indexes. Row for row, we diffed them. The big ones were faster on the merged index. Fewer events for Splunk to scan, that is all it is.
The query edit, in full:
BEFORE
index=pan sourcetype=pan:traffic | stats sum(bytes) as bytes count as sessions by app | sort - bytesAFTER
index=pan sourcetype=pan:traffic | `chiron_unroll` | stats sum(bytes) as bytes count as sessions by app | sort - bytesThe macro and the search-time field split ship as a Splunk app. Dashboards and saved searches take the same one-line edit.
Getting the original lines back
Getting the originals back is the easy part. Chiron takes the merged line apart into the lines it came from. We compared every line in the sample both ways: as it was in the file going into Chiron, and after it had been processed through Chiron and rebuilt. Every single line came back exactly as it was when we sent it in.
Since the record never left Splunk, there is no second compliance copy to keep somewhere else.
Where the number moves
Our sample is about 20% THREAT by bytes. Yours is probably different. THREAT lines carry more values that change from line to line, so they merge less than TRAFFIC lines did in our run, 32% against 44%. More THREAT in the mix means a lower overall number.
What it takes to deploy, and to undo
Chiron goes in-line between the forwarders and Splunk. Pull it out and the original lines flow again. Merged lines you already indexed keep working with the macro and rebuild on demand. No migration coming or going. That was a requirement we set on day one, not luck.
Finding #2: Delete the same fields as the pack, and the merge adds an estimated 21 points
Cribl’s Pack is able to save money in this case because it removes fields. That works just fine as long as all of the savings are coming from removing fields. What happens when there are savings for both field removal and merging? We ran the pack’s pipelines (v1.1.5) line by line using the sample data and then merged what was left.
Trimming only: 37.2%. Trimming then Merging: 58.3%. Note about the 58.3%: This figure is based on an estimator we created using the trimmed data set and does not represent the actual lossless test result. Our estimator was .9 points high compared to the true value in the lossless test.
Get Chironvision Inc’s stories in your inbox
Join Medium for free to get updates from this writer.
Percentages aside, the thing we’d rather people sit with is what those deleted fields were for. Every deleted field is a question you’ve agreed, in advance, never to ask. The table lists ten of the 22 TRAFFIC fields the pack drops by default and the question each one answers. Chiron keeps all of them.
We didn’t just assert the first row, either, we tested it. One firewall in the sample was forwarding 80% of its lines 15 seconds late or worse.
The presence of both generated_time and receive_time in the full log makes it apparent immediately; in the trimmed log both have been deleted, and the question goes with them. This is exactly the type of firewall that is most likely to go undetected on a quiet Tuesday afternoon when an incident occurs, versus during the peak activity hours when all firewalls will have issues due to excessive TRAFFIC.
Finding #3: route TRAFFIC to S3 and keep summary in Splunk (17 of 20 questions remain in Splunk)
Both Cribl’s routing recommendations and Splunk’s routing recommendations are consistent with one another. They recommend sending threat logs into Splunk while routing TRAFFIC to S3. There is logic to this recommendation. As the chart shows, TRAFFIC represents approximately three-quarters of the total bytes.
We completed the routing. Also we maintained a Chiron summary of TRAFFIC in Splunk. The Chiron summary was not a top ten table. Rather it was a per-minute state record keyed by the fields an analyst filters on, source and destination included. It included sessions, bytes, packets and worst delay.
The Chiron summary has already been demonstrated on something much more complex than this: ORCA-bench, a Kubernetes incident benchmark, where six days of logs (12.5 GB) were reduced to a 27 MB summary, and an agent that read only the summary produced better results on RCA than every agent in the ORCA-bench paper.
The run was tuned, and the write-up says so. The blog post about how Chiron improves the way AI investigates incidents on ORCA-bench is called How Chiron Changes AI Incident Investigation on ORCA-bench. The benchmark itself is described in the ORCA-bench paper, arXiv:2607.28545.
Next question: which of the twenty questions does each of the six setups still answer using Splunk?
In terms of the summary: 17 of 20 answers in Splunk; and all ten of the detection query answers are included in those seventeen. All answers were identical to their raw line counterparts upon comparing diffs.
Trim-and-route without the summary: nine of twenty. Three requests for archives are reasonable. One session’s entire record. Original audit lines. Full delay distribution, since the worst delay per row is carried by the summary, the worst-case per-firewall will be identical, but the distribution itself will not be.
How much will I pay for the archives?
The other (unpriced) part of the S3 story is the questions. Splunk describes how it provides a federated search service over S3 for low frequency, ad-hoc uses. One of the Splunk customers who discussed their plans on the community forums described their plans clearly; when you need to perform a search in S3, do it during an incident. Send all of your TRAFFIC to this location, and that is what your TRAFFIC becomes.
Splunk states that 97 percent of queries run back no farther than yesterday. When routed, each of these queries (in addition to those that run a longer period) will represent a scanned item. Using Athena’s price, if you perform 100 queries each day for a year, the total amount of money spent on Athena would equal $160K. If you perform 1,000 such queries per day for a year, the total would equal $1.58M. This could result in you either stopping your searches or limiting how many you send.
Athena is used as an example because its price is public. Splunk sells scanning in 10 TB units and does not publish a price per unit; its own guidance says high frequency use will likely cost more than indexing. Cost assumptions throughout: Splunk at $1.50 per GB, the rate in Cribl’s Meter Reader; S3 Standard holding a year of raw lines at 8.75:1 gzip; scans at Athena’s $5 per TB; no product subscriptions.
Finding #4: Every published strategy, plus a merge rate of 37–42% on top
In order to find out whether we can say anything about the published percentages (i.e., “we save more”) we decided to see what else we could find. There is some information provided in the README regarding the pack. According to the README, the percentage savings are in the range of 15–30%. An example was found in a Cribl Power Hour blog. For one customer, they reported a percentage savings of 62.5% after dropping start events and sampling allowed traffic. Axoflow’s case study indicates that they reduced SIEM cost by 33%. As for Splunk’s Edge Processor Guide, although it explains how to trim and drop items, it does not provide any specific percentage savings.
We took each of the rule sets which have been published and attempted to implement them (using their rules, but using our own software), and then applied them to our sample. Then, we tried to merge whatever remained. The rates did not really budge. Each implementation resulted in approximately the same results, 36.6 to 41.8%.
It should come as little surprise that this occurred. Their methods remove rows or columns. Therefore, regardless of how many rows remain, they are going to continue to be just as repetitive as before. Repetition is the basis for a merge’s entire diet.
Their numbers will always be theirs. The one exception is Cribl’s 62.5%. They apparently had thousands of zero-byte sessions to drop in their customer’s environment. Our sample contains none. Therefore, we make no claims that we were able to reproduce this.
Summary
Keep everything: 40% savings and lose nothing. Remove like the pack removes: The merge adds 21 points on top of that. Route TRAFFIC to S3: Detection remains in Splunk on the summary. Archive returns to being an archive, for infrequent pulls. None of the published methodologies touch what the merge touches. Therefore, it stacks on top of all of them.
Check it on your logs
Download the estimator tool (coming soon !) and run it with your log sample using your AI copilot (Claude, Cursor, etc.). No data leaves your data center. The tool will generate a detailed report showing how much you can save. Or send us a sample of one index and the report comes back within hours.
chironvision.ai
Earlier: The part of your Splunk bill no tool has been allowed to touch and Why the Log-Cost Playbook Needs to Change.
