AI attack taxonomies need objective models, not prompt catchalls

At a glance

What this is: This is HiddenLayer’s update to its APE taxonomy, and the key finding is that prompt attack analysis works better when objectives are modeled separately from prompts and techniques.

Why it matters: It matters to IAM and security teams because the same separation-of-concerns logic now applies to AI agents, tool use, and identity-driven workflows that need clearer governance boundaries.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.

👉 Read HiddenLayer’s update to the APE taxonomy and AI attack objective model

Context

AI attack taxonomies matter because defenders cannot secure what they cannot name. In generative systems, a single prompt can be both delivery vehicle and attack method, while the resulting model behaviour may reveal data, trigger tools, or steer workflows in ways that are not visible from the prompt text alone.

HiddenLayer’s update argues that adversarial prompting needs a cleaner separation between observable technique and inferred objective. That is directly relevant to AI agent governance because security teams now have to reason about prompts, tool execution, and downstream identity effects in the same control model, not as unrelated problems.

For identity programmes, the important shift is that AI systems are no longer just content generators. They increasingly sit inside business workflows, retrieve data, call APIs, and hand outputs to other systems, which means prompt abuse can become an identity and authorisation problem as quickly as it becomes a content-safety problem.

Key questions

Q: How should security teams classify adversarial AI prompts in practice?

A: Classify them by the observable technique, the attacker objective, and the resulting security impact, not by a single catchall label. That prevents prompt injection from obscuring whether the real issue was data exposure, unauthorized tool use, workflow manipulation, or service disruption. A cleaner taxonomy produces better controls, better tests, and better incident reporting.

Q: Why do AI systems need separate objective and impact categories?

A: Because the same prompt can lead to very different outcomes depending on the model, tools, and workflow context. Separating objectives from impacts lets teams distinguish the adversary’s intent from the actual security consequence. That is essential for red teaming, policy mapping, and prioritising mitigations against the real failure mode.

Q: What do security teams get wrong about prompt injection?

A: They often treat it as one attack type when it is really a family of behaviours that can lead to different outcomes. That mistake blurs the line between content-safety issues and broader security incidents. Teams should analyze what the system did after the prompt, then map that result to the right control domain.

Q: How do AI judge models change the security model?

A: Judge models create a second decision layer that can be manipulated just like the primary model. If an attacker can influence the evaluator, the safety boundary stops being trustworthy. Practitioners should test that layer as a privileged control path and monitor it as part of the trust boundary.

Technical breakdown

Why prompt injection is too broad for AI security models

Prompt injection has become shorthand for many different adversarial behaviours, but it collapses delivery, technique, intent, and outcome into one label. That is useful for conversation, not for security engineering. HiddenLayer’s APE model separates prompts from tactics, techniques, objectives, and impacts so teams can describe what the attacker did, what the system did in response, and what security consequence followed. That separation is especially important in agentic workflows, where the same prompt pattern can lead to data exposure, unauthorized tool use, or workflow manipulation depending on context.

Practical implication: build detection and red-team taxonomies around observable technique and separate them from business impact.

How AI attack objectives map to confidentiality, integrity, and availability

The revised taxonomy anchors objectives in the traditional CIA triad, then adapts those categories to AI-specific outcomes. Confidentiality covers exposures such as system prompts, user data, secrets, and training data. Integrity covers behavior subversion, task redirection, unauthorized tool actions, and manipulated outputs. Availability covers denial of service, token exhaustion, latency inflation, and cost abuse. This model is more useful than a flat list because it ties AI abuse to outcomes security teams already understand while still preserving the distinct mechanics of model misuse.

Practical implication: align AI threat models to CIA outcomes so controls, logging, and escalation paths match the real failure mode.

Why multi-model systems create new control-plane failure modes

HiddenLayer’s additions such as Refusal Hijacking and Safety or Judge Model Manipulation reflect a broader shift toward multi-model architectures, where one model generates content and another evaluates or filters it. That creates a new control plane that can itself be manipulated. If an attacker can influence a judge model, safety decision, or moderation layer, they may bypass the very mechanism meant to contain abuse. For identity teams, this matters because AI governance is increasingly about who can cause which model to act, evaluate, or approve at runtime.

Practical implication: treat model-to-model evaluation paths as part of the trust boundary and monitor them like privileged control flows.

Threat narrative

Attacker objective: The attacker aims to turn model behaviour into a controllable path for data leakage, tool misuse, or business process manipulation.

1. entry: An attacker uses adversarial prompts such as pretexting, control token spoofing, or refusal hijacking to get a generative system to accept a malicious instruction.
2. escalation: The model interprets the prompt as legitimate context, then leaks data, generates prohibited content, or invokes a tool in a way the attacker can steer.
3. impact: The adversary achieves data exposure, workflow manipulation, unauthorized actions, or service disruption through the AI system’s own execution path.

Breaches seen in the wild

• DeepSeek breach — DeepSeek breach exposed 1M+ log lines and sensitive secret keys.
• Schneider Electric credentials breach — exposed credentials gave attackers access to Schneider Electric Jira, exfiltrating 40GB.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

AI attack taxonomies are becoming identity taxonomies by another name. Once a generative system can retrieve data, call tools, and drive workflows, the line between prompt abuse and identity abuse starts to disappear. The useful question is no longer only what the model said, but what it was allowed to reach, trigger, or change. For practitioners, the taxonomy must therefore map to authorization boundaries as much as to content safety.

Objective models are more operational than prompt labels because they preserve security meaning. A prompt pattern can support very different outcomes, from system prompt exposure to unauthorized tool use to denial of wallet. Flattening those into one label hides the control failure. Security teams need the objective layer because that is where policy, logging, and response design become materially different.

Content policy violation is not the same thing as compromise, and the taxonomy is better when it says so. HiddenLayer’s choice to treat policy violation as a separate objective family reflects a real distinction between harmful content generation and broader CIA impact. That distinction matters for governance because a team that confuses policy enforcement with security enforcement will under- or over-rotate its controls. Practitioners should keep those two concerns analytically separate.

Model-to-model evaluation paths are emerging as a privileged decision layer. When one model judges another, the safety boundary becomes a control surface that can be manipulated. That is structurally similar to abusing a privileged workflow step in IAM or PAM, except the actor is now steering a model rather than a human approver. The implication is that AI governance needs to treat judge models, filters, and moderation gates as part of the access path.

The field needs a named concept for objective leakage, not just prompt leakage. We would call this the objective ambiguity gap: the attacker’s intent is inferred from behaviour, but the observable prompt alone does not tell defenders which control failed. That gap weakens threat modeling, detection tuning, and incident classification. Practitioners should classify attacks by the objective they enabled, not by the prompt form they used.

From our research:

• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation, according to AI Agents: The New Attack Surface report.
• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
• As AI workflows expand, compare those findings with OWASP NHI Top 10 to pressure-test where your governance model still assumes static, human-paced access.

What this signals

Objective-led AI governance is quickly becoming the practical minimum. Teams that still label every adversarial prompt the same way will miss the difference between content abuse, tool abuse, and identity abuse. The programme signal is clear: logging, triage, and policy enforcement need to map to what the system actually did, not just what the attacker typed.

With 33% of organisations already reporting AI agents accessing sensitive data beyond intended scope, per the AI Agents: The New Attack Surface report, objective ambiguity is now a governance problem, not a research problem. The next control discussion should focus on whether your workflows can prove what a model was allowed to reach and why.

Model-to-model trust chains will need the same scrutiny IAM teams already apply to privileged delegation. As judge models, filters, and moderation layers become runtime decision points, identity governance has to ask who can influence those layers and what downstream authority they effectively hold. That is the bridge between AI security and access governance.

For practitioners

• Separate prompt, technique, objective, and impact in your AI threat model Map adversarial prompts to the behaviour they trigger, then classify the security consequence separately as confidentiality, integrity, or availability impact. That distinction makes detection rules, escalation paths, and reporting more precise. Use the same model for red teaming and post-incident review.
• Treat tool-using AI workflows as identity-relevant control paths Inventory which models can reach data, invoke tools, or write to downstream systems, then define explicit authorization boundaries for each path. If a prompt can change a business record or trigger an action, it belongs in the same governance conversation as privileged automation.
• Add judge-model and moderation-layer abuse to your testing scope Test whether a prompt can manipulate a safety filter, scoring model, or human review aid into approving content it should block. Include these paths in red-team scenarios because they are now part of the attack surface, not just supporting infrastructure.
• Classify AI incidents by observable behaviour, not by prompt style A pretexting prompt, a refusal hijack, and a control-token spoof can all produce different operational outcomes even if they look similar in text. Use the resulting behaviour and impacted assets to drive triage, evidence collection, and lessons learned.

Key takeaways

• HiddenLayer’s update makes AI attack analysis more usable by separating prompts, techniques, objectives, and impacts.
• The most valuable shift is the move from a flat prompt taxonomy to CIA-aligned objective modeling for AI systems.
• Security teams should treat model-to-model decision paths and tool-using workflows as privileged trust boundaries.

Key terms

• Adversarial Prompt Engineering: Adversarial Prompt Engineering is the study of how attackers craft prompts to manipulate model behaviour. In practice, it is less about one prompt string and more about the interaction between instruction hierarchy, context, tools, and downstream system effects that turn model output into an attack path.
• Objective Model: An objective model groups attacks by what the adversary is trying to achieve, such as data exposure, behavior subversion, or availability disruption. For AI systems, this helps teams separate intent from technique and map attacks to the control domain that actually failed.
• Judge Model: A judge model is a separate model used to score, filter, classify, or approve output from another model. It becomes part of the security boundary because attackers may influence the evaluator itself, which can weaken moderation, routing, or safety decisions downstream.
• Content Policy Violation: Content Policy Violation is an objective category for prompts that try to make a system generate restricted or prohibited output. It matters because policy failure and security failure are not the same thing, even when they occur in the same workflow or interface.

What's in the full report

HiddenLayer's full research covers the operational detail this post intentionally leaves for the source:

• The full taxonomy view for tactics, techniques, objectives, and impacts, including the updated objective hierarchy and subtype structure.
• Detailed examples of Refusal Hijacking, Pretexting, and Safety / Judge Model Manipulation in adversarial AI testing.
• The changelog for deprecated, demoted, and renamed techniques, which is useful when updating detection content or red-team playbooks.
• The interactive website experience, including graph and matrix views, which helps analysts browse the taxonomy at implementation depth.

👉 HiddenLayer's full research shows the updated taxonomy structure, new techniques, and the changelog behind the revisions.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are responsible for identity security strategy or governance maturity, it is worth exploring.