AI Security Architecture: Controls Layer by Layer

Most AI security failures are architecture failures: an exposed training cluster, a model with more tool permissions than any human, an API key in a notebook, or a retrieval index that ignores the source system's access rules. Point controls such as a prompt filter cannot compensate for a design that trusts model output or gives an agent standing credentials. This guide sets out a layered reference architecture for AI systems, from data and the model supply chain through infrastructure, model serving, the application and agent layers, to identity and monitoring, and lists the controls that belong at each layer with their mapping to the OWASP Top 10 for LLM Applications 2025 and MITRE ATLAS. It is for security architects, platform engineers and reviewers who design or approve AI platforms, and it links to the deeper NHI Mgmt Group guides on each layer rather than repeating them.

Key takeaways

Treat the model as an untrusted component. Inputs can be attacker-controlled (prompt injection, OWASP LLM01) and outputs must be validated before they reach browsers, shells, databases or tools (LLM05).
Enforce authorisation outside the model. Tools, retrieval and data access are decided by deterministic policy using the end user's and the agent's identity, never by the system prompt.
Every AI component is a non-human identity. Model services, agents, pipelines and vector stores need owned, short-lived, least-privilege credentials, and stolen keys are a leading AI attack path.
Use one layer model across frameworks. Google SAIF's four component areas (data, infrastructure, model, application) and the NCSC-led lifecycle guidelines fit together; pick one model and map every control to it.
Log the whole chain. Prompts, retrieved context, tool calls, model versions and outputs must be logged and correlated to an identity, or you cannot detect or investigate AI incidents.

The reference layers

Google's Secure AI Framework (SAIF) groups AI system components into four areas: data (sources, filtering and processing, training data), infrastructure (model frameworks and code; training, tuning and evaluation; data and model storage; model serving), model (the model, input handling, output handling) and application (the application and agents). This guide uses those areas and adds the two layers that cut across them in practice: identity and access, and monitoring and response. Governance sits above all of them.

Layer	Components	Main threats	OWASP LLM 2025	MITRE ATLAS examples
Data	Data sources, pipelines, labelling, training and fine-tuning sets, RAG corpora	Poisoning, sensitive data in training, unlicensed data	LLM04 Data and Model Poisoning, LLM02 Sensitive Information Disclosure	AML.T0020 Poison Training Data, AML.T0070 RAG Poisoning
Model supply chain	Base models, adapters, model hubs, ML libraries, containers	Malicious or tampered models, unsafe serialisation, vulnerable packages	LLM03 Supply Chain	AML.T0010.003 AI Supply Chain Compromise: Model, AML.T0011.000 Unsafe AI Artifacts
Infrastructure	Training clusters, notebooks, GPU nodes, model and data storage, orchestration	Exposed dashboards and APIs, weight theft, lateral movement	LLM03, LLM10 Unbounded Consumption	AML.T0024.002 Extract AI Model, AML.T0055 Unsecured Credentials
Model serving	Inference endpoints, AI gateway, rate limits, system prompts	Extraction, cost abuse, denial of service, system prompt leakage	LLM07 System Prompt Leakage, LLM10	AML.T0034 Cost Harvesting, AML.T0029 Denial of AI Service, AML.T0056 Extract LLM System Prompt
Application (input and output)	Chat UI, APIs, RAG retrieval, output rendering, guardrails	Direct and indirect prompt injection, jailbreaks, unsafe output handling, misinformation	LLM01 Prompt Injection, LLM05 Improper Output Handling, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation	AML.T0051.001 LLM Prompt Injection: Indirect, AML.T0054 LLM Jailbreak
Agent and tools	Agent runtime, tool and MCP servers, memory, code execution sandboxes	Excessive agency, tool misuse, memory poisoning, exfiltration through tools	LLM06 Excessive Agency	AML.T0053 AI Agent Tool Invocation, AML.T0086 Exfiltration via AI Agent Tool Invocation, AML.T0080.000 AI Agent Context Poisoning: Memory
Identity and access	Model service identities, agent identities, API keys, OAuth tokens, user delegation	Stolen keys, over-privileged service accounts, credentials in agent configuration	LLM06, LLM02	AML.T0083 Credentials from AI Agent Configuration, AML.T0055 Unsecured Credentials
Monitoring and response	Prompt and tool logs, SIEM, model behaviour monitoring, kill switches	Undetected abuse, no forensic trail, slow containment	All	AML.T0031 Erode AI Model Integrity (detected only through monitoring)

For threat modelling against these layers, see the AI Agent Threat Modelling Guide and the MITRE ATLAS Guide.

Data and model supply chain layer

Data provenance: record source, licence, collection date and transformations for every training, fine-tuning and retrieval data set; accept data only from trusted, verified sources, as the OWASP LLM08 mitigations recommend for knowledge bases.
Sensitive data minimisation: scan and redact secrets and personal data before training or indexing; data in model weights cannot be selectively deleted later.
Integrity verification: verify cryptographic hashes and signatures on models, adapters and data sets before use, and inspect imported pre-trained models in a separate zone with AI-specific scanners, as the joint NSA AISC, CISA, FBI and Five Eyes guidance "Deploying AI Systems Securely" recommends.
Safe formats: prefer non-executable weight formats (such as safetensors) over pickle-based formats that can run code on load.
AI-BOM: keep an inventory of models, data sets, libraries and their versions so a compromised component can be traced to every deployment. See the AI Supply Chain and AI-BOM Guide and the AI Data Poisoning and Backdoors Guide.

Infrastructure and model serving layer

Segmentation: place training clusters, notebooks and inference services in dedicated network segments with boundary protection and allow-listed egress. Never expose cluster dashboards or job APIs to the internet; the ShadowRay 2024 campaign showed attackers exploiting exposed Ray AI clusters to steal cloud credentials.
Hardened runtimes: run models in hardened containers or VMs, patch GPU drivers and ML frameworks promptly, and scan images. See the Container Security Guide.
Protect model weights: encrypt at rest with keys held in an HSM or KMS, store weights in a restricted enclave or vault, and require two-person control for access to high-value weights, per the joint deployment guidance. Weights are both intellectual property and an attack accelerator.
AI gateway in front of every model: a single control point for authentication, per-identity quotas, token and cost limits, routing to approved models, content policy and logging. See the AI Gateway and LLM Gateway Guide.
Limit what the API returns: return only what the application needs (no raw logits or full probability vectors unless required), which makes extraction and inversion harder.
Treat system prompts as public: OWASP LLM07 is clear that system prompts should not hold secrets or be relied on for access control.

Model-level hardening, evaluation and weight protection are covered in depth in the AI Model Security Guide.

Application layer: inputs, retrieval and outputs

Separate instructions from data: mark and isolate untrusted content (web pages, emails, documents, tool results) and assume any of it can carry indirect prompt injection. No filter is complete, so design so that a successful injection has limited effect. See the Prompt Injection Guide.
Permission-aware retrieval: enforce the source system's access rights at query time, partition vector stores by tenant and sensitivity, and filter results by the requesting user's identity before they enter the context window. See the RAG Security Guide.
Output handling: encode output for its destination (HTML, Markdown, SQL, shell), block rendering of external images and links that could exfiltrate data, and never pass output to an interpreter without validation, as LLM05 requires.
Guardrails as a layer, not the boundary: input and output classifiers catch known jailbreaks, sensitive data and policy violations, but they are probabilistic. See the AI Guardrails and AI Firewalls Guide.
Grounding and human review: for misinformation risk (LLM09), require citations to retrieved sources and human review for consequential outputs. The general application controls are in the LLM Security Guide.

Agent and tool layer

Agents add the ability to act, which turns a content risk into an integrity and availability risk. OWASP LLM06 Excessive Agency names the core controls: minimise the extensions an agent can call, minimise each extension's functionality (read-only mailbox access for a summariser), avoid open-ended tools such as unrestricted shell or URL fetch, and require human approval for high-impact actions.

Tool policy enforcement point: every tool call passes through a deterministic policy check (allowed tool, parameters, data scope, rate) bound to the agent's identity and the user it acts for. See the AI Agent Tool and Plugin Security Guide and the MCP Security Guide.
Sandbox code execution: isolated, ephemeral runtimes with no ambient credentials and restricted egress. See the AI Agent Sandboxing and Code Execution Guide.
Memory hygiene: scope memory per user and task, validate what is written, and expire it. See the AI Agent Memory Security Guide.
Approval and spending limits: thresholds for payments, deletions, external messages and permission changes. See the AI Agent Action Approvals and Spending Limits Guide.
Kill switch: the ability to revoke an agent's credentials and stop its runs in minutes. See the AI Agent Kill Switch and Containment Guide.

The OWASP GenAI Security Project published its Top 10 for Agentic Applications in December 2025; see the OWASP Top 10 for Agentic Applications Guide for the agent-specific risk list.

Where identity fits

An AI platform contains more non-human identities than human ones: the inference service calling a model provider, the pipeline writing to storage, the retrieval service reading a document store, each agent, and each tool server. Each needs an owner, a purpose, scoped permissions and a lifecycle.

Identity	Typical credential	Preferred pattern	Common failure
Application calling a model API	Provider API key	Key held only by the AI gateway in a secrets manager, per-application keys with quotas, rotation	Keys in code, notebooks or front-end bundles, leading to LLMjacking
Training and inference workloads	Cloud service account or role	Workload identity federation with short-lived tokens, separate roles for training and serving	One broad role shared across notebooks, training and serving
Agent	OAuth client credentials or delegated user tokens	Distinct agent identity, delegated scopes bound to the user, short token lifetimes, sender-constrained tokens where supported	Agent runs with a human admin's token or a static all-powerful key
Tool and MCP servers	Downstream API tokens	Per-tool least-privilege credentials, never exposed to the model context	Credentials stored in agent configuration files readable by the agent
Vector store and data connectors	Database or SaaS connector credentials	Read-only, scoped to indexed sources, with user-level filtering at query time	Connector with admin rights indexing everything it can see

The LLMjacking campaigns of 2024 to 2026 show the result when cloud credentials and model API keys leak: attackers run models at the victim's expense. Detailed patterns are in the Agentic AI Identity Guide, the AI Infrastructure Workload Identity Guide, the LLMjacking Guide and the Secrets Management Guide.

Monitoring, detection and measurement

What to log: per request, the calling identity (user and agent), model and version, system prompt version, prompt and retrieved document identifiers, tool calls with parameters and results, guardrail decisions, output, token counts and cost. Store logs in immutable storage, as the joint deployment guidance recommends, and redact sensitive fields.
What to alert on: spikes in token use or cost per key, high-frequency repetitive queries (extraction), guardrail block clusters from one identity, tool calls outside the agent's normal pattern, new egress destinations from model hosts, and unauthorised changes to model files or configuration.
Integrate with the SOC: SAIF's "detection and response" element is about bringing AI systems into existing threat monitoring and incident response, not building a parallel process. See the AI Agent Observability and Incident Response Guide.

Metrics that show the architecture is working:

Percentage of model traffic passing through the AI gateway (target 100% for production).
Percentage of AI non-human identities with a named owner and credentials under 24 hours lifetime or rotated within policy.
Number of models and data sets in production without verified hash or signature and AI-BOM entry (target zero).
Percentage of agent tools with a documented least-privilege scope and policy enforcement.
Mean time to revoke an agent's or model service's credentials in an exercise.
Red-team findings per release by layer, and time to remediate. See the AI Red Teaming Guide.

How to implement the reference architecture

Inventory first: list every model, endpoint, agent, data set, vector store and AI credential, including shadow use. See the Shadow AI Discovery Guide.
Tier the use cases: apply the full control set to high-risk and agentic systems first. See the AI Use Case Risk Tiering Guide.
Put the gateway and identity controls in place: route all model traffic through a gateway, move keys into a secrets manager, replace static keys with federated short-lived credentials. Start in log-only mode to learn traffic, then enforce.
Close infrastructure exposure: remove public access to clusters, notebooks and storage; segment and restrict egress.
Add application and agent controls: permission-aware retrieval, output encoding, tool policy enforcement and approvals, piloted with one agent in a limited user ring.
Test: threat model each system, red-team against injection, extraction and tool misuse before each release, and re-test after model changes, in line with NIST AI 600-1 action MS-2.7-007.
Plan rollback: keep the previous model version, prompt version and tool policy deployable; maintain immutable backups so you can restore the last known good state; rehearse the kill switch.

Decision guidance

Hosted API versus self-hosted model: hosted APIs shift infrastructure and weight protection to the provider but put more weight on key management, data residency and contract terms (see the AI Procurement Checklist). Self-hosting gives control of data and weights but makes you responsible for the whole infrastructure layer.
Central gateway versus per-team integration: a central gateway gives consistent policy and logging; per-team integrations are faster but usually leave keys scattered and logs missing. Most organisations above a handful of AI applications benefit from the gateway.
Agent autonomy: start read-only with human approval for writes, and widen only with evidence from monitoring. See the AI Agent Autonomy Levels Framework.
Small organisations: focus on hosted models through one gateway or account, secrets management, permission-aware retrieval and logging. Large organisations need a platform team owning shared controls, an AI-BOM, model risk management (see the AI Model Risk Management Guide (SR 11-7)) and red-team capacity.

Framework mapping

Source	How it structures AI security	Where it fits this architecture
NCSC, CISA and partners: Guidelines for secure AI system development	Four lifecycle areas: secure design, secure development, secure deployment, secure operation and maintenance	Lifecycle view across all layers: threat modelling, supply chain, documentation, infrastructure protection, incident management, logging and update management
ETSI EN 304 223 V2.1.1	13 principles across secure design, development, deployment, maintenance and end of life	Baseline requirements, including asset tracking, documenting data, models and prompts, and proper disposal
Google SAIF	Six core elements and four component areas	The layer model used in this guide
NIST AI 600-1	12 generative AI risks, including information security and value chain and component integration, with suggested actions	Risk register and test planning (MS-2.7-001 threat assessment, MS-2.7-007 red-teaming, GV-6.1-005 supplier risk). See the NIST AI 600-1 Generative AI Profile Guide
NIST AI 100-2 E2025	Taxonomy of adversarial machine learning attacks and mitigations	Common vocabulary for model-layer threats
NIST CSF 2.0	ID.AM-08 lifecycle management, PR.AA-05 least privilege, PR.DS-01 data at rest, DE.CM-09 runtime monitoring, GV.SC-07 supplier risk	Enterprise control objectives for each layer
NIST SP 800-53 Rev. 5	AC-6 Least Privilege, IA-5 Authenticator Management, SC-28 Protection of Information at Rest, SR-3 Supply Chain Controls and Processes, SI-4 System Monitoring, AU-2 Event Logging, SA-11 Developer Testing and Evaluation	Control catalogue for assessments
MITRE ATT&CK v20	T1078.004 Valid Accounts: Cloud Accounts, T1552.001 Unsecured Credentials: Credentials In Files, T1190 Exploit Public-Facing Application	Conventional attack paths into AI infrastructure

Common failure patterns

Exposed ML infrastructure: dashboards, notebooks and job APIs reachable from the internet without authentication, as in ShadowRay.
Authorisation in the prompt: "only answer questions about the user's own account" in a system prompt is not access control; injection bypasses it.
Retrieval that ignores permissions: a connector with broad rights indexes HR and finance documents and any user can retrieve them through the assistant.
Static, shared model keys: one provider key copied across teams and repositories, with no quota or owner, so a leak is both costly and hard to trace.
Agent with human admin rights: an agent running under its creator's privileged token, with no separate identity, so its actions cannot be scoped, attributed or revoked independently.
No prompt or tool logging: logs stop at the HTTP request, so investigators cannot see what was injected or which tools ran.

Practitioner checklist

Inventory models, endpoints, agents, data sets, vector stores and AI credentials, and record owners.
Map every AI system to the layers in this guide and record the controls in place at each.
Route all production model traffic through an AI gateway with authentication, quotas and logging.
Store model API keys in a secrets manager and replace static cloud keys with short-lived federated credentials.
Verify hashes and signatures on models and data sets, prefer safe weight formats, and maintain an AI-BOM.
Remove public exposure of training and inference infrastructure and restrict egress.
Enforce retrieval permissions and tool authorisation outside the model, with approvals for high-impact actions.
Encode and validate model output for its destination before rendering or execution.
Log prompts, context, tool calls and outputs against identities in immutable storage and feed the SOC.
Red-team each release and rehearse credential revocation and model rollback.

AI Security Architecture Reference Guide: Controls Layer by Layer