Join our Newsletter — 33% off our NHI Course

Notifications
Clear all

LLM weaknesses in 2025: what AI security teams need to fix


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 19382
Topic starter  

TL;DR: Its 2025 red-team findings exposed five LLM weaknesses, including reasoning prompt injection, hidden tool execution, indirect prompt injection, malformed-token image abuse, and email identity spoofing, according to ActiveFence. The wider lesson is that AI apps inherit trust assumptions they cannot verify, so governance must cover model behaviour, tool use, and identity signals together.

NHIMG editorial — based on content published by ActiveFence: The 5 Most Shocking LLM Weaknesses We Uncovered in 2025

By the numbers:

  • 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, inappropriately sharing sensitive data, and revealing access credentials.

Questions worth separating out

Q: How should security teams prevent AI models from acting on hidden instructions in prompts and documents?

A: Put a policy boundary between untrusted content and model execution.

Q: Why do AI assistants create a new trust problem for identity governance?

A: AI assistants create a new trust problem because they can read data, choose tools, and act on external text in ways traditional review processes do not expect.

Q: What breaks when AI tool use is not fully logged and attributable?

A: Incident response breaks first, because teams cannot tell whether an action came from a user request, a model decision, or injected content.

Practitioner guidance

  • Harden reasoning and tool boundaries Separate untrusted user content from internal reasoning and require a policy check before any tool call can execute.
  • Make AI actions auditable Log every tool invocation with the triggering input, model rationale, and final action so reviewers can reconstruct what happened.
  • Treat inbound content as adversarial Scan email, document, and message pipelines for indirect prompt injection before the model processes them.

What's in the full article

ActiveFence's full article covers the technical red-team examples this post intentionally leaves at the pattern level:

  • Step-by-step reproductions of the five LLM weaknesses and the payload styles used to trigger them
  • Proof-of-concept detail on how reasoning prompt injection and ghost calling bypass normal review paths
  • Examples of how malformed tokens and indirect prompt injection move from text manipulation to security impact
  • Practical guardrail approaches for teams testing public-facing AI applications in production-like environments

👉 Read ActiveFence's analysis of the five LLM weaknesses uncovered in 2025 →

LLM weaknesses in 2025: what AI security teams need to fix?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 3 months ago
Posts: 18973
 

Reasoning integrity is becoming an AI governance control, not just a model-safety concern. The article shows that attackers can manipulate what the model thinks before they influence what it says or does. That shifts the control problem from output filtering to decision-path integrity, which is where agentic AI governance now intersects with identity and authorization. In practical terms, organisations need a verifiable boundary between user input, internal reasoning, and execution authority.

A question worth separating out:

Q: How can organisations reduce risk from AI-assisted attacks on identities?

A: Organisations should shorten credential lifetime, narrow privilege scope, and watch for rapid multi-stage identity abuse. AI-assisted attacks can compress reconnaissance, exploitation, and exfiltration into a short window, so identity controls must assume faster attacker iteration. The safest response is to limit what any single identity can do if misused.

👉 Read our full editorial: Five LLM weaknesses exposing AI safety and security gaps in 2025



   
ReplyQuote
Share: