Five LLM weaknesses exposing AI safety and security gaps in 2025

At a glance

What this is: ActiveFence’s red-team research identifies five LLM failure modes that let attackers manipulate reasoning, hide tool activity, leak sensitive data, abuse generation pipelines, and spoof message identity.

Why it matters: AI security teams and IAM practitioners need to treat LLMs and assistants as governed systems that can inherit, obscure, or amplify trust failures across content, tools, and identity signals.

By the numbers:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.
• 92% agree governing AI agents is critical to enterprise security, yet only 44% have implemented any policies to do so.

👉 Read ActiveFence's analysis of the five LLM weaknesses uncovered in 2025

Context

LLM security failures are rarely limited to prompt content alone. The more important issue is that modern AI systems combine reasoning, tool use, content processing, and external identity signals in one workflow, which creates a larger governance surface than traditional application security controls were designed to monitor. In practice, that means AI agent and assistant deployments can fail in ways that look like policy violations, data leakage, or social engineering even when the model appears to be functioning normally.

This article is really about the security boundary between model behaviour and enterprise trust. Where AI systems summarize email, call tools, or generate content from untrusted inputs, IAM, PAM, and identity-verification assumptions become part of the attack surface. That intersection matters because organisations are increasingly treating LLM outputs as operationally trusted without putting equivalent controls around the inputs, delegation paths, and identity signals the system consumes.

Key questions

Q: How should security teams prevent AI models from acting on hidden instructions in prompts and documents?

A: Put a policy boundary between untrusted content and model execution. Inspect inputs for hidden instructions, reasoning-tag references, and tool directives before they reach the model, and require separate approval logic for any action that can change data, send messages, or invoke external tools.

Q: Why do AI assistants create a new trust problem for identity governance?

A: AI assistants create a new trust problem because they can read data, choose tools, and act on external text in ways traditional review processes do not expect. Identity governance has to account for action promotion, provenance, and egress, not only authentication or entitlement assignment.

Q: What breaks when AI tool use is not fully logged and attributable?

A: Incident response breaks first, because teams cannot tell whether an action came from a user request, a model decision, or injected content. Compliance review also fails, since there is no reliable evidence trail for why the system accessed data, sent a message, or called a tool.

Q: How can organisations reduce risk from AI-assisted attacks on identities?

A: Organisations should shorten credential lifetime, narrow privilege scope, and watch for rapid multi-stage identity abuse. AI-assisted attacks can compress reconnaissance, exploitation, and exfiltration into a short window, so identity controls must assume faster attacker iteration. The safest response is to limit what any single identity can do if misused.

Technical breakdown

Reasoning prompt injection and corrupted internal decision paths

Reasoning prompt injection targets the model’s internal decision process rather than only the visible prompt. In agentic systems, the model may maintain hidden or semi-hidden reasoning steps before taking an action, and attackers can place instructions inside that flow to alter the model’s plan, policy checks, or tool selection. Once corrupted reasoning is treated as legitimate by the model itself, the attack can persist into later steps and influence multiple outputs. The underlying problem is not just bad text. It is a lack of trustworthy separation between user content, internal reasoning, and execution authority.

Practical implication: isolate reasoning from untrusted input and enforce independent guardrails before tool execution.

Ghost calling and invisible tool execution

Ghost calling is a control failure where an AI system executes a tool action but does not log, explain, or attribute that action in a way humans can review. That makes the model’s behaviour hard to audit and creates a gap between what actually happened and what reviewers believe happened. In operational terms, this is an identity and accountability issue as much as a model issue, because the system can act on behalf of a user or agent without a durable evidence trail. If the action is not observable, it is not governable.

Practical implication: require tool-call logging, reason-for-action traces, and policy checks that block unaudited execution.

Indirect prompt injection through email and document pipelines

Indirect prompt injection hides malicious instructions inside content the model is expected to process, such as emails, documents, or other inbound data. This matters because summarization and extraction workflows often trust the source format while failing to treat the content as hostile. The result can be disclosure of private data, unsafe actions, or manipulated summaries that mislead users. The broader architecture problem is that AI assistants often inherit the permissions and context of the workflow they sit inside, but they do not inherit the verification controls that would normally protect a sensitive enterprise process.

Practical implication: treat all inbound content as hostile and separate data processing from action authority.

Threat narrative

Attacker objective: The attacker aims to manipulate AI behaviour so the system reveals data, executes hidden actions, or produces deceptive output that can be used for fraud or further compromise.

1. Entry occurs when attackers place hidden instructions inside content the model is designed to read, such as reasoning tags, emails, or documents.
2. Escalation happens when the model accepts those instructions as part of its own reasoning and uses them to select tools, alter outputs, or expose sensitive information.
3. Impact follows when the AI system generates phishing content, leaks private data, or performs unaudited actions that users wrongly trust.

NHI Mgmt Group analysis

Reasoning integrity is becoming an AI governance control, not just a model-safety concern. The article shows that attackers can manipulate what the model thinks before they influence what it says or does. That shifts the control problem from output filtering to decision-path integrity, which is where agentic AI governance now intersects with identity and authorization. In practical terms, organisations need a verifiable boundary between user input, internal reasoning, and execution authority.

Invisible execution creates an accountability gap that traditional application logging does not close. If a tool call happens without an auditable explanation, reviewers lose the ability to distinguish model intent from attacker influence. That is a governance failure because the system can carry out a delegated act while leaving no reliable trail for incident response or compliance review. The practitioner takeaway is that AI tool use must be logged as a first-class control, not inferred from downstream effects.

Identity signals are now part of the AI trust chain, which makes spoofing attacks more effective. The email assistant example shows that if an AI system does not verify SPF, DKIM, or DMARC, it can sanitize an attacker’s identity into something users trust. That is a direct bridge between identity verification and AI governance. Security teams should treat email, document, and workflow metadata as security inputs, not just convenience features.

Malformed-token and hidden-instruction attacks expose a new governance debt in AI pipelines. The article’s most technical failures show that moderation layers alone do not address adversarial manipulation of the processing pipeline itself. That means the category now needs stronger provenance controls, content boundary validation, and model-adjacent policy enforcement. Practitioners should assume that AI safety and security must be designed into pipeline architecture, not added as a post-processing layer.

What this signals

Reasoning integrity is now a programme-level control point: if the model can be steered through hidden instructions, teams need guardrails that sit before inference, not just after generation. For practitioners, that means treating model input validation, tool authorization, and traceability as one control chain rather than separate tasks.

AI assistants that process email and documents should be reviewed like privileged workflows because they can transform untrusted content into trusted action. Where identity verification signals are stripped away, the assistant becomes a trust amplifier, which is why provenance checks and delegated-access scoping matter for security operations.

The operational signal to watch is whether your AI systems can explain why they acted, not just what they returned. If provenance, tool calls, and sender validation are missing from the review path, your governance model is already behind the threat model.

For practitioners

• Harden reasoning and tool boundaries Separate untrusted user content from internal reasoning and require a policy check before any tool call can execute. Block attempts to reference reasoning tags, hidden instructions, or tool commands inside prompt content.
• Make AI actions auditable Log every tool invocation with the triggering input, model rationale, and final action so reviewers can reconstruct what happened. Without a durable trace, hidden execution becomes a compliance and incident-response blind spot.
• Treat inbound content as adversarial Scan email, document, and message pipelines for indirect prompt injection before the model processes them. Where the workflow handles sensitive content, separate read permissions from action permissions.
• Verify identity signals before trusting AI summaries Require authentication checks such as SPF, DKIM, and DMARC to remain visible to the assistant and the user. Do not let summarization layers erase sender-validation context that would change the trust decision.

Key takeaways

• LLM weaknesses are no longer limited to unsafe outputs because attackers can now manipulate reasoning, execution, and identity trust paths inside AI workflows.
• The evidence points to a governance gap, not a single bug, with hidden tool use, indirect prompt injection, and spoofed identity all undermining review and accountability.
• Security teams should separate untrusted content from execution authority, preserve provenance signals, and make every AI action auditable before deploying at scale.

Key terms

• Reasoning Injection: An attack where a threat actor inserts a false internal premise into an AI agent’s thought process so the model continues from a misleading assumption. The goal is not only to change the output, but to steer the agent toward unsafe action by corrupting the logic it trusts.
• Ghost Calling: A failure mode where an AI system executes a tool action without leaving a clear audit trail or explanation. This creates an accountability gap because reviewers cannot reliably see what triggered the action or whether the model acted on attacker influence.
• Indirect Prompt Injection: Indirect prompt injection is an attack where malicious instructions are hidden inside content that an AI system reads later. The model may treat that content as context rather than as hostile input, which can influence tool use, data access, or workflow actions if controls are weak.
• Identity Signal Preservation: The practice of keeping authentication and provenance signals visible as content moves through AI workflows. It matters because assistants can strip away the cues humans rely on, such as sender validation or source authenticity, and turn untrusted input into trusted action.

What's in the full article

ActiveFence's full article covers the technical red-team examples this post intentionally leaves at the pattern level:

• Step-by-step reproductions of the five LLM weaknesses and the payload styles used to trigger them
• Proof-of-concept detail on how reasoning prompt injection and ghost calling bypass normal review paths
• Examples of how malformed tokens and indirect prompt injection move from text manipulation to security impact
• Practical guardrail approaches for teams testing public-facing AI applications in production-like environments

👉 ActiveFence's full post includes the proof-of-concept detail and control guidance behind each weakness.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, secrets management, and agentic AI identity. It is suitable for practitioners building controls across human, non-human, and AI-driven access pathways.