Join our Newsletter — 33% off our NHI Course

Why does max-connections() affect AxoSyslog throughput even when traffic is steady?

Because the source window is divided across connections, a high max-connections() setting can shrink the per-connection share enough that workers cannot gather enough messages to fill batches. That creates underutilised queue space and slower delivery even when the source itself is active.

Why max-connections() can cap throughput even when the source is busy

Max-connections() is not only a limit on parallelism, it also changes how incoming traffic is partitioned across workers. When the setting is high, each connection may receive too little traffic to form efficient batches, so the scheduler spends more time moving small chunks than delivering full queue loads. The result is lower throughput despite steady ingress.

The key detail is that throughput depends on both arrival rate and batch efficiency. A source can remain continuously active while still underperforming if its traffic is spread too thinly across many active connections. In practice, that means the bottleneck is not source idleness, but fragmentation of work and the overhead of handling many underfilled batches.

For AxoSyslog, this makes max-connections() a tuning parameter with a direct effect on aggregation efficiency. A lower value can increase per-connection message density, letting workers accumulate fuller batches before dispatch. A higher value can improve concurrency in some cases, but once the per-connection share drops below the level needed for effective batching, the system often slows down instead of speeding up.

What changes inside the pipeline when connections are overprovisioned?

Each connection competes for the same incoming stream. If the stream is steady but divided too widely, workers may repeatedly wake up, process partial batches, and return to waiting before they have enough messages to make the next step efficient. That creates a form of internal inefficiency: the queue is active, but not enough messages are available per batch to keep delivery cost low.

This is why the effect can look counterintuitive. Operators often expect more connections to mean more capacity, but batching systems depend on message density, not just connection count. Once the per-connection slice becomes too small, the system spends proportionally more effort on scheduling, buffering, and handoff, which reduces net throughput even though the source is still producing data.

The practical consequence is that max-connections() behaves like a throughput shaper, not a pure scaling knob. It influences whether workers see enough contiguous load to amortise queue and processing overhead. In environments with steady but moderate traffic, too many connections can therefore become a self-inflicted fragmentation problem.

How to tune max-connections() without guessing

Start from the batching target, not from the number of possible clients. If the average message rate per connection is too low to fill batches consistently, reduce max-connections() until each worker sees enough traffic to operate efficiently. If latency or fairness is the concern, increase it only as far as the batching profile still holds up.

A useful test is to compare connection count against observed batch fill behaviour. If throughput rises when you lower max-connections(), you have likely been over-partitioning the stream. If throughput falls sharply, the value may already be near the minimum that preserves useful concurrency for your workload.

When you change it, validate with steady-state traffic rather than short bursts. Burst traffic can hide the problem because batches fill quickly during spikes, while the real issue appears under normal, sustained load. The right setting is the one that keeps workers busy with full or near-full batches during ordinary operation, not the one that looks best during a brief test.

Practitioner Guidance

What to prioritise: Tune for batch density first, then for concurrency. In a steady-load pipeline, that usually means reducing the number of active connections until each one carries enough traffic to fill batches predictably.

What to verify: Confirm that lower max-connections() improves delivered messages per unit time without creating unacceptable queuing delay. Watch for the point where additional connections stop adding useful parallelism and start diluting batch efficiency.

Common mistake: Treating connection count as a proxy for capacity. For this class of pipeline, more connections can mean more fragmentation, more wakeups, and less efficient queue use.

Practitioner takeaway: If steady traffic slows as max-connections() rises, the system is usually limited by how thinly work is split, not by source availability.