Document poisoning is the deliberate insertion of misleading, malicious, or manipulative content into documents that an AI system later reads or learns from. In AI security, it can corrupt retrieval, training, or agent context, causing false outputs, unsafe actions, or biased decisions when models trust compromised source material.
How Document Poisoning Works
Document poisoning is a content-integrity attack against the materials an AI system relies on. The attacker does not need to tamper with the model itself if they can influence the documents, notes, tickets, wikis, web pages, or knowledge bases the system later trusts.
The attack is dangerous because retrieval and generation systems often treat source text as authoritative. A poisoned document can seed false facts, distort summaries, steer recommendations, or alter downstream reasoning even when the underlying model behaves normally.
Where It Enters the AI Pipeline
Document poisoning can happen at ingestion, indexing, retrieval, or during human curation of source material. That makes it relevant to retrieval-augmented generation, knowledge bases, training corpora, and any workflow that copies external text into an AI-facing context without strong validation.
The most common entry points are low-friction ones: shared drives, helpdesk articles, internal wikis, uploaded files, scraped web sources, and collaborative documents. If those sources are assumed to be trustworthy, the poisoned content can persist long enough to shape multiple outputs.
In RAG-heavy environments, the problem is especially acute because the system may surface the poisoned passage with apparent confidence. NHIMG’s RAG Security Guide is useful here because it focuses on source trust, corpus poisoning, and grounding checks.
Why Document Poisoning Is Hard to Notice
Document poisoning often looks like normal content, not malware. The dangerous part is semantic, not technical: the text can be plausible, incomplete, subtly biased, or designed to influence model behaviour without triggering obvious security alerts.
It can also survive routine controls that focus on file type, phishing, or executable malware scanning. A poisoned paragraph inside an otherwise benign document can be enough to mislead a system that lacks source provenance, review workflows, or content-level integrity checks.
When the poisoned material enters an agentic workflow, the effect can extend beyond misinformation. The AI may select the wrong tool, follow an unsafe instruction, or carry a corrupted assumption into a later decision path. The OWASP Agentic AI Top 10 is relevant because it frames identity, privilege, and poisoned context as security concerns, not just model quality issues.
Security and Governance Implications
Document poisoning is fundamentally a trust problem. Organisations need to know which sources are authoritative, which are editable, which are externally supplied, and which have enough integrity to influence an AI answer or action.
That means the control challenge is not only document review. It also includes source allowlisting, provenance tracking, tenant or repository separation, change control, and post-ingestion validation of what the AI can actually retrieve or learn from.
For adversary modelling, poisoned documents can be treated as an input manipulation path that sits alongside prompt injection and corpus tampering. MITRE’s MITRE ATLAS adversarial AI threat matrix is a useful reference for situating context poisoning and related AI attack techniques.
Risk and Threat Considerations
Document poisoning can create silent, durable exposure because the compromised text may be reused by multiple models, workflows, or teams before anyone notices. The risk is not limited to false answers, it can also include unsafe actions, bad prioritisation, and corrupted business decisions driven by trusted but manipulated source material.
Failure mechanism: An attacker inserts misleading or malicious text into a document source that the AI later indexes, retrieves, or learns from, then relies on the system’s trust in that source to shape output or actions.
Impact: The AI may propagate falsehoods, adopt unsafe instructions, or amplify bias across repeated uses, with downstream consequences for security, reliability, and decision quality.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
MITRE ATT&CK addresses the attack and risk surface, while OWASP ASVS, NIST AI RMF, NIST CSF 2.0 and NIST SP 800-53 Rev 5 set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP ASVS | V15 — Secure Architecture | Document poisoning affects trusted content flows used by AI systems. |
| Recommendation — Design document ingestion and retrieval paths so untrusted content cannot influence AI decisions without validation. | ||
| NIST AI RMF | GOVERN — GOVERN | AI governance must define provenance, accountability, and trust for source material. |
| Recommendation — Establish governance for AI source provenance and content trust before documents enter model workflows. | ||
| NIST CSF 2.0 | ID.AM-01 — Identities and Assets Are Inventoried | Preventing poisoning depends on knowing which document sources feed AI systems. |
| Recommendation — Inventory the document sources, repositories, and pipelines that can influence AI outputs. | ||
| NIST SP 800-53 Rev 5 | SI-4 — System Monitoring | Monitoring is needed to detect suspicious content changes or poisoning indicators in AI-fed sources. |
| Recommendation — Monitor AI-relevant content repositories for anomalous changes and suspicious insertions. | ||
| MITRE ATT&CK | T1565 — Data Manipulation | Poisoning is a form of manipulating trusted data so downstream systems behave incorrectly. |
| Recommendation — Map poisoned-document findings to data-manipulation techniques and hunt for the affected source paths. | ||
Practitioner Guidance
Why practitioners should care: Treat document integrity as part of AI security engineering, not just knowledge management. If the system can read it, it can potentially act on it, so source trust must be designed and governed explicitly.
What to watch for: Pay special attention to editable shared content, externally sourced materials, and documents that sit close to retrieval or training workflows. Those are the places where poisoned text is most likely to become operationally meaningful.
Practitioner takeaway: The safest AI knowledge pipelines assume that documents are untrusted until provenance, review, and retrieval controls prove otherwise.