Stability

Expanded Definition

Stability describes how consistently a model, scanner, or assessment workflow reproduces the same result when it is run again against the same target under small, controlled variations. In cybersecurity and AI security, this matters because a tool that alternates between finding and missing the same issue creates uncertainty in triage, reporting, and remediation planning. Stability is closely related to repeatability and reproducibility, but it is narrower in practice: it focuses on whether the output remains dependable enough to support security decisions.

Definitions vary across vendors and research teams, especially when stability is discussed alongside robustness, sensitivity, or deterministic behaviour. For glossary use, NHI Management Group treats stability as an operational quality of the detection or evaluation process, not as a measure of model accuracy or business risk. A tool can be accurate on average and still be unstable if its findings fluctuate across near-identical runs. That distinction is especially important for AI-assisted security tooling, where prompt variation, sampling settings, or environmental drift can change the result set. The most common misapplication is treating unstable findings as valid signal, which occurs when teams compare outputs from inconsistent runs and assume the differences reflect changing risk rather than tool variance.

For broader governance context, the NIST Cybersecurity Framework 2.0 is useful for framing why dependable security outcomes matter, even though it does not define stability as a standalone term.

Examples and Use Cases

Implementing stability rigorously often introduces extra evaluation overhead, requiring organisations to balance faster release cycles against the cost of repeated testing and result comparison.

• A vulnerability scanner flags the same misconfiguration in repeated scans of the same container image, showing stable detection behaviour.
• An AI-assisted code review tool reports the same insecure dependency pattern across reruns with unchanged input, which helps security teams trust the finding.
• A cloud posture check produces different answers after identical queries because of nondeterministic enrichment, making the result unstable and difficult to govern.
• A phishing detection model is tested against the same message corpus over multiple runs to confirm that alerting does not swing due to minor input noise.
• An NHI inventory workflow surfaces the same orphaned secret across repeated assessments, helping teams prioritise credential cleanup with confidence.

Where stability matters most is in tooling used for non-human identity discovery, secrets exposure checks, and AI-assisted triage, because inconsistent output can hide the very exposures teams are trying to reduce. It also matters in evaluation pipelines that compare one model version to another, since unstable baselines can make a safe release look worse or a risky release look benign. In practice, security teams often validate stability by rerunning the same assessment under controlled conditions and checking whether findings persist, not just whether they appear once.

Why It Matters for Security Teams

Unstable security results create downstream problems that are hard to see at first. Analysts may waste time re-investigating findings that disappear on rerun, while genuine issues can be dismissed as noise. Over time, this erodes confidence in scanners, model outputs, and automated workflows, which then slows remediation and weakens governance. For AI-enabled security tools, instability can also mask prompt sensitivity, sampling variance, or environment-dependent behaviour that should have been identified before production use.

From an identity and NHI perspective, stability is critical when teams rely on automated discovery of secrets, service accounts, API keys, and other machine identities. If a detector is unstable, orphaned credentials may remain visible in one run and absent in the next, complicating containment and audit evidence. The same concern applies to agentic AI systems that use tools on behalf of users, because unstable evaluations can produce inconsistent safety decisions, access recommendations, or policy checks.

Organisations typically encounter the cost of instability only after a scan result cannot be reproduced during incident response, at which point stability becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• Signal Stability