A run is one unit of work inside a trace, such as an LLM call, prompt formatting step, or tool invocation. Runs let teams inspect behaviour at a finer level than a single request log. They are useful for diagnosing failures, measuring latency, and understanding agent decisions.
Expanded Definition
A run is the smallest inspectable execution unit inside a trace, usually representing one discrete action such as an LLM call, prompt transformation, retrieval step, or tool invocation. In NHI and agentic AI operations, runs provide the granularity needed to reconstruct how an autonomous workflow behaved, which inputs it saw, and what it returned.
Definitions vary across vendors on whether a run should include only model execution or also pre-processing, post-processing, and tool-side activity. NHI Management Group treats the term operationally: if a step has its own timing, inputs, outputs, or failure state, it is usually worth tracking as a run. This matters because NIST Cybersecurity Framework 2.0 emphasises visibility, logging, and response as core governance functions, and run-level telemetry is often the evidence layer behind those functions.
The most common misapplication is treating a whole agent session as one run, which occurs when observability is collapsed into a single request log and the sequence of model and tool actions is lost.
Examples and Use Cases
Implementing run-level observability rigorously often introduces telemetry volume and correlation overhead, requiring organisations to weigh forensic detail against storage, cost, and privacy exposure.
- A customer-support agent issues one run for prompt assembly, one for the model call, and one for a ticketing-system tool action, allowing operators to isolate which step caused a bad answer.
- A secrets-remediation agent records separate runs for discovery, validation, and rotation, making it easier to prove whether a failed rotation came from policy, API access, or downstream dependency.
- A retrieval-augmented workflow logs a run for document fetches and another for summarisation, which helps distinguish a retrieval failure from a model hallucination.
- During incident review, teams compare run timing and payloads against the broader trace to identify where an agent exceeded scope or invoked an unexpected tool.
- For governance reporting, NHI teams correlate run metadata with the lifecycle controls described in the Ultimate Guide to NHIs and with telemetry expectations in NIST Cybersecurity Framework 2.0.
In practice, run boundaries are most useful when they match actual security-relevant decision points rather than arbitrary code modules.
Why It Matters in NHI Security
Run data is where hidden agent behaviour becomes visible. Without it, organisations can see that an agent failed, but not whether the failure came from a malformed prompt, a misrouted tool call, a stale token, or an excessive privilege path. That distinction is critical in NHI security because the same underlying identity can behave safely in one run and dangerously in another, especially when permissions, secrets, and context are assembled dynamically.
This is also where visibility gaps become measurable. NHI Management Group’s Ultimate Guide to NHIs reports that only 5.7% of organisations have full visibility into their service accounts, which is a strong indicator that many teams will also lack sufficient execution-level insight into agent activity. Run telemetry supports anomaly detection, least-privilege validation, and post-incident reconstruction when a workflow touches credentials, APIs, or privileged systems. It also helps align identity governance with operational resilience expectations in NIST Cybersecurity Framework 2.0.
Organisations typically encounter the need for run-level analysis only after an agent has leaked data, invoked the wrong tool, or exhausted a secret, at which point the term becomes operationally unavoidable to address.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST CSF 2.0, NIST Zero Trust (SP 800-207) and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Agentic AI Top 10 | AI-02 | Run-level logging is central to understanding agent step execution and tool use. |
| OWASP Non-Human Identity Top 10 | NHI-09 | Run telemetry helps trace NHI-backed actions and exposure across workflows. |
| NIST CSF 2.0 | DE.AE-3 | Anomalous activity detection depends on detailed execution records such as runs. |
| NIST Zero Trust (SP 800-207) | SA | Zero Trust relies on continuous verification of each action, which run data supports. |
| NIST AI RMF | AI risk management needs traceable operational evidence for model and agent actions. |
Record each agent step as a distinct run and review it for unsafe tool calls or prompt misuse.
