How should teams evaluate forecast accuracy when actual values can be zero or near zero?

Teams should avoid relying on MAPE alone when actual values can be zero or very small, because the metric becomes undefined or wildly distorted. Use MAE for unit-based clarity, RMSE when large misses matter most, and WMAPE for mixed-volume portfolios. The right choice depends on whether you need comparability, outlier sensitivity, or stable portfolio-wide accuracy.

Why This Matters for Security Teams

Forecast accuracy looks simple until the dataset includes zeros, near-zeros, or intermittent demand. At that point, percentage-based error can become undefined, explode on small denominators, or overstate tiny misses that are operationally irrelevant. Security teams evaluating anomaly rates, incident volumes, or low-frequency event forecasts need error measures that stay interpretable under sparse conditions, not just elegant on a slide.

This is the same pattern NHI teams run into when they depend on a single metric to describe a complex control surface: NHI Mgmt Group notes that only 5.7% of organisations have full visibility into their service accounts in the Ultimate Guide to NHIs, which makes clean measurement even harder when the underlying counts are intermittent. For governance decisions, consistency matters more than mathematical convenience, and the wrong metric can make a stable process look broken or a broken process look acceptable. In practice, many teams discover this only after reporting has already been distorted by a few near-zero actuals.

How It Works in Practice

When actual values can be zero or very small, the first step is to separate the forecasting question from the metric question. MAPE is fragile because it divides by actuals, so a zero actual makes it undefined and a near-zero actual can dominate the result. For that reason, teams usually pair a unit-based metric with a scale-aware metric instead of forcing one percentage score to do everything.

MAE is the most straightforward option when stakeholders need to understand error in native units, because it answers how far off the forecast was on average. RMSE is better when larger misses should carry more weight, since squaring errors penalises spikes more aggressively. WMAPE works well for mixed-volume portfolios because it normalises total absolute error by total actual volume, which makes it more stable across periods with many small observations. For governance and auditability, NIST’s NIST SP 800-53 Rev 5 Security and Privacy Controls is useful as a general control reference for measurement, monitoring, and accountability processes, even though it does not prescribe a specific forecast metric.

• Use MAE for operational teams that need plain-language error in units.
• Use RMSE when large misses create outsized business or security impact.
• Use WMAPE for portfolio reporting where volume varies across time or business units.
• Keep MAPE only for datasets where actuals are reliably non-zero and the denominator is stable.

For NHI-heavy environments, the metric discipline matters because low-count events are common and the reporting surface is often noisy. The broader visibility gap and credential sprawl described in Ultimate Guide to NHIs — 2025 Outlook and Predictions reinforces why teams need stable measures that still work when the data is sparse. These controls tend to break down when the forecasted series is mostly zeros with rare spikes, because percentage denominators stop reflecting operational reality.

Common Variations and Edge Cases

Tighter error measurement often increases reporting overhead, requiring organisations to balance statistical robustness against executive simplicity. That tradeoff becomes visible in intermittent-demand systems, sparse security telemetry, and any environment where actuals are frequently zero.

There is no universal standard for this yet, but current guidance suggests treating metric selection as context-dependent rather than universal. If the question is whether a team is systematically missing a process, MAE may be enough. If the concern is occasional high-impact misses, RMSE is more honest. If the portfolio contains many low-volume or highly uneven segments, WMAPE is usually more defensible than MAPE. The practical mistake is presenting one score as if it answers every audience. Finance may want percent comparability, operations may want unit error, and risk teams may care most about tail misses.

Edge cases also matter. If actuals can be zero for legitimate reasons, report the count of zero-actual periods alongside the error metric. If forecasts are evaluated across multiple segments, avoid averaging segment-level MAPE scores because the smallest actuals can distort the combined picture. In recurring reviews, document why a metric was chosen and what it excludes. That helps reviewers distinguish model weakness from denominator instability. Current best practice is to use a metric set, not a single metric, when zeros are part of the operating reality.

Related resources from NHI Mgmt Group

• How should security teams prioritise NHI remediation in cloud environments?
• How should security teams govern non-human identities at scale?
• How should security teams govern non-human identities for compliance?
• How should security teams govern non-human identities in Salesforce?