Static model scanning inspects the artifact before it is loaded, looking for unsafe deserialization, suspicious opcodes, and known vulnerabilities. Runtime red teaming tests how an application behaves after deployment. They solve different problems, and teams need both when models come from external sources or shared registries.
Static Scanning Checks the Artifact, Not the Behaviour
static model scanning answers a pre-deployment question: is the model file, package, or artifact itself safe to trust? It is used to inspect structure, serialisation, embedded code paths, and indicators of known weakness before anything is executed. That makes it useful for supply-chain hygiene, especially when teams import models from third-party registries, partner systems, or internal marketplaces. Runtime ai red teaming asks a different question: once the model is live inside an application, how does it behave when users probe it, chain prompts, or push it toward unsafe outputs?
Both practices matter because they cover different failure surfaces. A clean static scan does not prove safe behaviour under adversarial interaction, and a strong runtime exercise does not guarantee the underlying artifact is free of malicious or fragile components. In practice, many security teams encounter the gap only after a model passes intake but later misbehaves under real usage pressure.
What Each Technique Can and Cannot Tell You
Static scanning is best understood as a control for trustworthiness at rest. It looks for defects that can be detected without running the model, such as unsafe deserialisation paths, suspicious embedded instructions, malformed objects, or known vulnerable dependencies bundled with the asset. That means it can block bad inputs early, but it cannot observe emergent behaviour, tool use, or downstream side effects once the model is placed inside a product.
Runtime red teaming is behavioural. It exercises the deployed model in context and tries to surface prompt injection susceptibility, policy bypasses, unsafe content generation, leakage from retrieved data, and tool abuse. For AI systems connected to RAG pipelines, plugins, or agentic actions, that live context is often where the real risk appears. Anthropic Frontier Red Team - Claude Mythos technical analysis is useful here because it illustrates the kind of post-deployment behavioural testing that static inspection cannot replace.
- Use static scanning to reject suspicious artifacts before trust is granted.
- Use runtime red teaming to measure how the deployed system responds under adversarial pressure.
- Treat the two findings as complementary, not interchangeable.
The practical distinction is timing and evidence: static scanning is a gate on the artifact, while runtime red teaming is a test of the operating system, prompting layer, tools, and policy stack around it. This guidance breaks down when the model is not separable from the application or when custom execution wrappers hide the true runtime behaviour.
When Teams Need Both, and Where the Boundary Gets Blurry
Tighter AI assurance often increases review effort, so organisations have to balance faster model adoption against deeper trust validation. The boundary becomes blurry when a model artifact includes executable components, custom ops, or packaging choices that can affect both pre-load safety and live behaviour. In those cases, a static pass may reveal supply-chain concerns, while runtime testing may expose governance or prompt-injection failures that the file format itself could never show.
This is especially important for models sourced from shared registries, external vendors, or internal reuse libraries. A model can be structurally sound yet still behave unsafely in a particular prompt environment, and a model can appear safe in a demo but contain hidden weaknesses that only become relevant when deserialised or embedded into a larger workflow. There is no consensus that one method can substitute for the other across all deployment styles; practitioners should treat that claim with caution.
What changes at scale is the blast radius. Once the same model is reused across multiple products or agent workflows, a missed static issue becomes a distribution problem, while an untested runtime failure becomes a repeated operational failure. The right answer is usually to place static scanning upstream of promotion and runtime red teaming after integration, then re-test when the model, wrapper, prompts, or tools change.
Risk and Threat Considerations
The material risk is that organisations confuse artifact safety with behavioural safety. Static scanning can miss adversarial prompt handling issues, tool misuse paths, and unsafe interactions with retrieval or external actions, while runtime red teaming can miss malicious or fragile components embedded in the model package itself.
Failure mechanism: A weak intake process lets a risky model artifact into the environment, or a weak live test process leaves the deployed system unchallenged against prompt injection, data leakage, or unsafe action execution. The control gap appears when teams rely on only one layer and assume it covers the other.
Impact: The result can be unsafe model execution, leakage of sensitive context, untrusted outputs entering business workflows, or repeated exposure across every application that reuses the same model.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Agentic AI Top 10 and OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST AI RMF, NIST AI 600-1 and CIS Controls v8 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST AI RMF | GV — Govern | AI governance must distinguish pre-deployment assurance from runtime behavioural validation. |
| Recommendation — Define separate assurance gates for model intake and deployed-system evaluation. | ||
| NIST AI 600-1 | Model Evaluation — Model Evaluation | Compares model artefact review with testing of model behaviour in context. |
| Recommendation — Evaluate both the model artifact and its live behaviour before broad release. | ||
| OWASP Agentic AI Top 10 | A2 — Input and Context Injection | Runtime red teaming often probes prompt injection and tool misuse in agentic apps. |
| Recommendation — Test deployed agents for prompt injection and unsafe tool invocation paths. | ||
| OWASP Non-Human Identity Top 10 | NHI-01 — Inventory and Governance | Shared model registries and external sources create lifecycle trust and intake risk. |
| Recommendation — Inventory model sources and block untrusted artifacts before promotion. | ||
| CIS Controls v8 | 16 — Application Software Security | Both techniques support secure evaluation of software-like AI assets and release gates. |
| Recommendation — Use security testing gates to catch model and application weaknesses before release. | ||
Practitioner Guidance
What to prioritise: Decide first whether the question is about trust in the artifact or trust in the deployed behaviour. If the concern is supply-chain acceptance, start with static scanning; if the concern is user-facing safety, tool abuse, or prompt resilience, start with runtime red teaming.
What to verify: Confirm that the model lifecycle has both a pre-load gate and a post-integration behavioural test. The common mistake is treating a passing static scan as evidence that the deployed system is safe enough for production use.
Practitioner takeaway: The useful mental model is “file risk versus system risk”: static scanning reduces what you are willing to load, while runtime red teaming reduces what you are willing to operate.
Related resources from NHI Mgmt Group
- What is the difference between static scanning and runtime protection for Java?
- What is the difference between static vulnerability scanning and runtime risk management?
- What is the difference between prompt testing and red-teaming agentic AI?
- What is the difference between red teaming an AI system and proving it is safe?
Deepen Your Knowledge
Reviewed and updated by the NHIMG editorial team on September 7, 2026.
NHI Mgmt Group — the #1 independent authority on Non-Human Identity, IAM, and Agentic AI security. nhimg.org
