From Prompt Injection to Model Poisoning: Inside the New Age of AI Attacks

Read full article from Salt Security here: https://salt.security/blog/from-prompt-injection-to-a-poisoned-mind-the-new-era-of-ai-threats/?utm_source=nhimg

Prompt injection was the first wake-up call. Context poisoning — corrupting an agent’s goals, memory, or tools — is the next, far more dangerous wave. When attackers change an autonomous agent’s mission brief, your security stack may never see an “attack” — it only sees a trusted agent doing trusted things. That’s why defenders must move from perimeter checks to behavioral and context protection.

Pull quote: “If you’re still watching the door, you won’t notice the saboteur rewriting the mission on the desk.”

Why this matters

• Autonomous agents act with legitimate credentials and approved APIs — so malicious behavior often looks normal to WAFs, static scanners, and gateway logs.
• Context poisoning doesn’t exploit code; it corrupts intent (goals), knowledge (memory), or capability (tools), turning trusted agents into stealthy attackers.
• Attacks are persistent (memory poisoning), immediate (goal hijack), or resource-exhausting (recursive loops), and they can enable espionage, data exfiltration, service disruption, or privilege escalation — all while appearing legitimate.

Four attack patterns to watch for

1. Forged Orders (Goal Hijacking)
   Change the agent’s primary objective so it performs unauthorized actions while logging legitimate access.
2. Memory Poisoning (Slow Corruption)
   Seed bad facts or feedback into an agent’s knowledge store so future decisions are skewed across many tasks.
3. Tool Escalation (Master Key Injection)
   Manipulate context so the agent gets more powerful tools or uses existing tools in unapproved ways.
4. Recursive Loops (Resource Denial)
   Force agents into infinite or expensive subtask loops that consume resources or create DoS without triggering traditional alerts.

Why traditional tools miss it

• API calls are valid and routed correctly → no WAF alarm.
• Code is unchanged and clean → static analysis passes.
• Requests come from approved agents → access logs look fine.
  This is an intent/behavior problem, not a signature problem.

What a modern defense looks like
Shift from “single-request” monitoring to intent-aware, behavioral defenses that treat agent context as a first-class asset.

Core defenses:

• Baseline agent behavior — Build per-agent profiles of normal sequences, tool usage, and data access patterns.
• Protect the mission brief — Treat goals, memory stores, tool lists, and MCPs as sensitive assets with versioning, signing, and access controls.
• Sequence-aware detection — Detect abnormal API call chains or sudden tool usage outside an agent’s normal role.
• Memory integrity checks — Validate knowledge sources and flag anomalous updates or feedback that diverge from trusted data.
• Least-capability enforcement — Limit agents to minimal tools and require explicit elevation workflows for new capabilities.
• Red-teaming & policy-as-code — Continuously adversary-test agents (goal forgery, memory poisoning) and codify permitted behavior and escalation paths.

Practical playbook — What to do now

1. Map agent capabilities — Inventory which agents exist, their approved APIs, and permitted data access.
2. Treat context as crown jewels — Apply strong access controls, cryptographic signing, and audit trails to goal and memory artifacts.
3. Baseline sequences — Instrument APIs to capture and model normal call chains per agent; focus on sequences not single calls.
4. Add sequence detectors — Alert on deviations like sudden cross-domain calls or unexpected privilege-using patterns.
5. Limit tool surfaces — Restrict agents to whitelisted tools; require multi-party approval for granting new capabilities.
6. Red-team the agent — Simulate goal hijacking, memory poisoning, and tool escalation to validate detection and response.
7. Automate containment — On confirmed deviation, automatically rollback mission changes, revoke newly granted tools, and isolate the agent’s sessions.

How Salt Security frames it

Behavioral API security is central: baseline agent intents, stitch API sequences into narratives, and flag “mission drift.” Rather than blocking single requests, look for mission-level anomalies — that’s where context poisoning reveals itself.

Final thought

Prompt injection was one-level-of-threat. Context poisoning is systemic and strategic. In an era of autonomous agents, the attack surface is the agent’s mind — and protecting it requires treating goals, memory, and tools as sensitive, auditable assets. Start mapping, start baselining, and start red-teaming your agents today — otherwise the saboteur inside will keep rewriting tomorrow’s mission.