What is the difference between reliable and non-reliable disk buffering in syslog-ng?

Reliable disk buffering writes the cache and output queue to disk, so nearly all outgoing messages survive abnormal termination. Non-reliable buffering keeps the cache and queue in memory, but spills to disk when needed and can still lose in-flight messages if the process or host crashes. Reliable mode improves durability, while non-reliable mode usually offers better performance and lower disk usage.

Why This Matters for Security Teams

Disk buffering is one of those controls that looks operational until a failure turns it into a data integrity and observability issue. The choice between reliable and non-reliable buffering determines whether syslog-ng preserves log traffic across crashes, host loss, and restart events, or whether it optimises for speed by accepting some loss risk. That matters for incident response, auditability, and downstream detection because lost log messages can erase the very evidence teams rely on to reconstruct activity. The broader logging lesson is that durability is a design choice, not a default outcome. In practice, many teams discover the gap only after an outage or process crash has already interrupted message flow.

How It Works in Practice

Reliable disk buffering persists both the output queue and the cache to disk, which gives syslog-ng a much stronger recovery posture after abnormal termination. If the process dies or the host reboots, messages already accepted into the buffer are far more likely to survive and be forwarded later. That makes reliable mode appropriate when the logging path is part of a security or compliance control chain and message loss would create material blind spots.

Non-reliable buffering uses memory for the queue and cache under normal conditions, then spills to disk when pressure builds or when the destination is unavailable. This can reduce disk writes and often performs better, but it does not provide the same survivability guarantee. If syslog-ng or the host crashes while messages are in memory, those in-flight messages can disappear. The difference is not abstract, it is about where the authoritative state lives when failure happens.

Use reliable buffering when log completeness matters more than throughput.
Use non-reliable buffering when short interruptions are acceptable and the environment can tolerate some loss.
Check the storage path, because durable buffering only helps if the underlying disk is available and healthy.
Validate the downstream destination, because a durable queue still cannot compensate for permanently broken routing or misconfiguration.

These controls tend to break down in high-volume pipelines with undersized or slow disks, because buffering can become the bottleneck and operators may silently relax durability settings to recover performance.

Common Variations and Edge Cases

Tighter buffering durability often increases disk I/O and storage overhead, so teams have to balance message survivability against latency and capacity. The right choice also depends on whether the logs are merely convenient telemetry or part of a defensible security record. If the output destination is intermittently unreachable, reliable buffering usually provides a better failure posture; if the source is extremely chatty and the messages are low value, non-reliable buffering may be the more practical trade-off.

There is also a deployment nuance: buffering protects only what syslog-ng has already accepted. It does not fix upstream loss, badly sized retention windows, or destination systems that cannot keep up over time. In clusters, appliances, and remote collectors, the weakest point is often not the logging daemon itself but the storage and failover design around it. The common mistake is treating disk buffering as a universal safety net when it is really a bounded durability mechanism with clear performance and loss trade-offs.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.

Framework	Control / Reference	Relevance
NIST CSF 2.0	DE.CM-7 — Continuous Monitoring	Syslog buffering affects the continuity of log data used for monitoring.
	PR.PT-1 — Audit Logging	Disk buffering determines whether audit logs survive interruptions.
Recommendation — Preserve log continuity so monitoring retains evidence through system failures. Configure logging to retain audit records across abnormal termination.
CIS Controls v8	8.2 — Ensure Audit Log Storage Capacity	Buffering choice impacts whether log storage remains available during outages.
	8.3 — Ensure Adequate Logging	Reliable buffering supports complete event capture for security logging.
Recommendation — Size and protect log storage so buffering does not become a loss point. Enable durable logging paths where evidence retention matters.

Practitioner Guidance

What to prioritise: Decide whether log preservation or throughput is the primary operational requirement before choosing the buffering mode. If the logs support investigations, compliance evidence, or control validation, durability should usually win over marginal performance gains.

What to verify: Confirm how much data can be lost during a process crash, host crash, or destination outage, and test that behaviour under realistic failure conditions. A buffering setting is only trustworthy if the team has validated what survives and what does not.

What good looks like: The logging path should degrade predictably, with clear storage headroom, observable queue growth, and an understood recovery pattern after restart. Teams should be able to explain exactly which failures may still cause loss and why.

Practitioner takeaway: Choose reliability according to the value of the log record, not according to the convenience of the pipeline, because the real decision is how much evidence the organisation is willing to risk losing during failure.