Notifications
Clear all
Topic starter
18/08/2026 2:42 pm
TL;DR: AxoSyslog pipelines can lose throughput without crashing when log-iw-size(), batch-lines(), and worker queues are misaligned, because in-application ACKs determine how many messages can stay in flight, according to Axoflow. The practical lesson is that window sizing, buffering, and batching must be tuned as one system, not as isolated knobs.
NHIMG editorial — based on content published by Axoflow: AxoSyslog internals: flow control, window size, queues, and batching
By the numbers:
- The ratio of non-human to human identities now exceeds 25x to 50x in modern enterprises.
- Only 5.7% of organisations have full visibility into their service accounts.
- 71% of NHIs are not rotated within recommended time frames, increasing the risk of compromise over time.
Questions worth separating out
Q: How should security teams size log pipeline windows for high-volume NHI telemetry?
A: Treat the window as a capacity model, not a buffer setting.
Q: Why do batching settings sometimes reduce throughput instead of improving it?
A: Because batching depends on enough messages being in flight at once.
Q: What signals show that a telemetry pipeline is under-sized?
A: Look for recurring batch-timeout flushes, consistently small batch sizes, and growing gaps between source activity and destination delivery.
Practitioner guidance
- Recalculate in-flight capacity before scaling connections Compare log-iw-size() against max-connections() * batch-lines() * workers() so the pipeline can actually fill batches under peak load.
- Prefer dynamic window allocation for uneven connection loads Use dynamic-window-size() when max-connections() is large but active connections are usually far fewer.
- Adopt disk buffering for durability-sensitive telemetry Move to disk-buffer() when losing in-flight events is unacceptable or when worker batching should be decoupled from source ACK timing.
What's in the full article
Axoflow's full article covers the configuration mechanics this post intentionally leaves at the architectural level:
- Exact interactions between log-iw-size(), flow-control, and batch-lines() under multi-connection load
- Detailed sizing examples for source windows, destination workers, and batch timing
- Behavioural differences between reliable(yes) and reliable(no) disk-buffer() modes
- Tuning guidance for batch-idle-timeout() when internal processing delays slow message arrival
👉 Read Axoflow's analysis of AxoSyslog flow control and batching →
AxoSyslog batching and flow control: where throughput quietly collapses?
Explore further
18/08/2026 3:25 pm
Silent throughput collapse is a governance problem, not just a tuning problem. The article shows how an AxoSyslog pipeline can degrade without failing outright when window size, batching, and connection count drift out of alignment. That matters because security teams often equate visible errors with operational risk, while the real problem is silent under-delivery of telemetry. In practice, log fidelity is part of control assurance, not just plumbing.
A question worth separating out:
Q: Should teams choose disk buffering or flow control for reliability?
A: Use disk buffering when durability and decoupled batching matter most, and keep flow control when preserving end-to-end synchronous delivery matters more than raw throughput. The decision is really about whether you want the source to wait for final delivery or for safe persistence.
👉 Read our full editorial: AxoSyslog flow control and batching can cap pipeline throughput
