What is the difference between foundation testing and application-specific AI security testing?

Foundation testing checks the model for broad, general vulnerabilities such as jailbreak susceptibility, unsafe completions, and baseline policy failures. Application-specific testing targets the way the model is used, including unauthorized commitments, excessive agency, false information, and compliance obligations. Mature programmes need both, because a model can be broadly safe yet still fail inside a real workflow.

Why Foundation and Application-Specific Testing Answer Different Security Questions

Foundation testing asks whether the model itself can be induced to behave unsafely across broad conditions, while application-specific testing asks whether the deployed use case creates a new failure mode that the base model review would never reveal. That distinction matters because governance, product safety, and enterprise risk often fail at the integration layer, not in the model benchmark. The Anthropic Project Glasswing material is useful here because it frames how higher-level testing must reflect the real system, not just the underlying model.

In practice, many security teams encounter serious defects only after a model is connected to tools, policies, or human workflows, rather than through intentional model-only review.

How the Two Test Types Work in Practice

Foundation testing is usually the starting line. It checks for baseline model behaviour such as prompt injection susceptibility, refusal reliability, unsafe instruction following, and whether the model can be coerced into violating safety constraints under adversarial prompts. This type of testing is useful because it establishes a floor: if the model cannot hold a basic safety posture, no downstream deployment can be considered trustworthy.

Application-specific testing then shifts the question from “Can the model fail?” to “How does it fail in this product, workflow, or decision process?” That means testing the surrounding orchestration, permissions, retrieval layer, tool use, escalation paths, logging, approval steps, and user experience. A model can pass broad safety checks and still cause harm if the application lets it draft an unauthorised commitment, trigger an action without review, surface restricted data, or present uncertain output as fact.

The practical difference is that foundation testing treats the model as the subject under review, while application-specific testing treats the whole system as the subject. For AI security teams, that means the second test type is where business impact usually appears, because real-world damage comes from how the model is embedded into a process, not only from what the model says in isolation. The best programmes use both layers together: the foundation layer to confirm baseline model behaviour, and the application layer to validate the exact control points that matter to the organisation.

Foundation testing looks for model-level weaknesses that generalise across prompts and contexts.
Application-specific testing looks for workflow-level failures introduced by tools, data access, routing, and permissions.
Foundation results are necessary but not sufficient for deployment confidence.
Application-specific results are the stronger indicator of operational and governance risk.

Where this guidance breaks down is when teams treat a single red-team exercise as proof that both layers are safe.

Where the Boundary Gets Blurry in Real Deployments

Tighter AI controls often increase testing overhead, requiring organisations to balance broader assurance against the time and cost of evaluating each workflow variant.

Some systems sit between the two categories. A foundation model exposed through a narrow chat interface may need more foundation-style testing than a fully autonomous agent, but the moment the interface adds retrieval, memory, external actions, or policy exceptions, the application-specific layer becomes the more important risk lens. Industry practice is still converging on exact terminology, so teams should be precise about what is being tested: the model, the wrapper, or the end-to-end system.

Another edge case is vendor-provided assurance. A model report may describe broad safety properties, but that does not tell you whether your own prompt templates, connectors, approval logic, or fallback behaviours are safe. Similarly, application testing without baseline model testing can miss brittle refusal behaviour or unsafe output patterns that later surface under different inputs. The right interpretation is not either-or. It is layered: model assurance informs whether the base system is acceptable, and application testing confirms whether the deployed implementation remains acceptable under your operating conditions. For broader AI governance context, teams often pair this thinking with CSA MAESTRO agentic AI threat modeling framework when the question extends into tool use and autonomous action.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

MITRE ATLAS address the attack surface, NIST AI RMF, NIST CSF 2.0 and CIS Controls v8 set the technical controls, and ISO/IEC 42001:2023 define the regulatory obligations.

Framework	Control / Reference	Relevance
NIST AI RMF	GV-1 — Govern AI risk throughout the lifecycle	Distinguishes model-level assurance from deployment-specific AI risk management.
Recommendation — Govern both model and application testing as separate lifecycle assurance activities.
ISO/IEC 42001:2023	A.5 — Policies for AI system development and use	Applies when testing must cover how the AI system is governed in deployment.
Recommendation — Define test scope and acceptance criteria for the deployed AI use case.
NIST CSF 2.0	GV.RM-01 — Risk Management Strategy	Supports program-level decisions on layered AI assurance and residual risk.
Recommendation — Set separate risk acceptance criteria for foundation and application testing.
MITRE ATLAS	ATLAS-IM-0001 — Prompt injection	Foundation testing commonly checks for adversarial prompt abuse against the model.
Recommendation — Exercise prompt-injection scenarios against the base model and measure refusal resilience.
CIS Controls v8	6.3 — Data Recovery	Application testing must validate controls around data exposure and operational recovery impacts.
Recommendation — Validate that AI workflows do not expose or mishandle protected data in use.

Practitioner Guidance

What to prioritise: Test the base model first only to establish a safety floor, then move quickly to the deployed workflow because that is where the material business risk usually emerges.

Decision rule: If the model can act, call tools, retrieve data, or influence an external process, treat application-specific testing as mandatory rather than optional.

What to verify: Confirm that the test scope matches the actual control boundary. A pass on model behaviour does not verify permissions, logging, approval logic, or human review steps in the application layer.

What practitioners underestimate: The highest-risk failure is often not obvious unsafe content, but a technically plausible output that becomes harmful only because the surrounding system trusts it too much.

Practitioner takeaway: Use foundation testing to establish baseline model safety, but use application-specific testing to decide whether the real system can be trusted in production.