What is the difference between document integrity and AI output integrity?

How document integrity differs from AI output integrity

document integrity is about the evidentiary input, not the model. A document can be syntactically valid and still be untrustworthy if it contains hidden instructions, selective edits, missing context, or embedded directives that change how a reader or system should interpret it. AI output integrity is about whether the system’s response is correct, but that only matters after the input has been treated as trustworthy evidence.

The practical difference is that document integrity asks, “Can I rely on this source as a faithful representation of what it claims to be?” AI output integrity asks, “Did the model generate an accurate answer from the material it was given?” If the input layer is compromised, a correct-looking output can still be built on a corrupted premise.

That distinction matters whenever a document is being used as evidence, policy input, or task instruction. A prompt, memo, contract, ticket, or report may contain adversarial content that is invisible to the casual reviewer yet still capable of steering downstream interpretation. The integrity failure is therefore not just in the model’s answer, but in the chain of trust that allowed the document to be treated as ordinary evidence.

Where the failure actually occurs in an AI-assisted workflow

In AI-enabled workflows, the most common mistake is to treat the model as the only trust boundary. In reality, the pipeline has at least two distinct checks: first, whether the document is complete, authentic, and contestable; second, whether the model’s answer is faithful, grounded, and not overconfident. A system can pass the second check and still fail overall if the first check was never enforced.

That is why concealed instructions, altered quotations, and missing provenance are so dangerous. They do not merely create “bad input”, they can change the meaning of the task before the model even starts reasoning. The correct control response is to validate the document as evidence before letting it shape the prompt, the summary, or the decision trail. For integrity-oriented review workflows, MITRE ATLAS Guide is useful when you need to think about how adversarial content can be staged to influence AI systems.

AI output integrity also has a separate boundary: even with clean input, the model can misread, omit, or overgeneralise. So the reviewer should distinguish “the source was manipulated” from “the model reasoned poorly.” Those are different failures, and they need different controls, different evidence, and different escalation paths.

How practitioners should separate source trust from model trust

The cleanest operational approach is to preserve the source artifact, inspect the source artifact, and only then assess the model’s response against it. That means keeping provenance, version history, redaction status, and any embedded markup or instructions visible to the reviewer. If the source cannot be shown clearly enough for challenge and verification, document integrity is already weak, regardless of whether the model answer sounds plausible.

For model-side checks, compare the output to the verified source rather than to the user’s expectation. A strong answer that ignores a hidden instruction may be acceptable if the instruction was malicious; a weak answer that repeats the hidden instruction may be a sign the system trusted the wrong layer. The reviewer’s job is to decide which layer failed first, because remediation depends on that order.

Where document handling is part of a broader assurance process, SLSA is a useful reference for integrity thinking around provenance, while OpenSSF provides broader supply-chain security context that helps teams treat inputs as verifiable artifacts rather than presumed truth.

Risk and Threat Considerations

When document integrity fails, the main risk is silent trust abuse. A concealed instruction or manipulated document can steer analysis, summarisation, classification, or approval without looking obviously malicious, which makes the downstream AI output seem more reliable than it really is. That is especially dangerous in workflows where the output is used to make a business, legal, or operational decision.

Failure mechanism: The attacker or faulty process changes the evidentiary layer, for example by embedding hidden directives, altering context, or removing material facts, and the AI system then processes that contaminated source as if it were ordinary input.

Impact: The model may produce a coherent but misleading answer, and reviewers may incorrectly attribute the failure to model hallucination instead of source tampering, delaying containment and corrective action.

Practitioner Guidance

What to verify: Confirm whether the source document is itself reviewable, versioned, and contestable before you judge the model’s answer. If the document cannot be independently trusted, do not treat a polished output as proof that the workflow is sound.

Decision rule: If the issue is embedded or concealed content in the input, treat it as a document integrity problem first; if the source is clean but the answer is wrong, treat it as an AI output integrity problem. That ordering determines whether you fix ingestion, review, or model behaviour.

Practitioner takeaway: The key judgement is to separate “was the source trustworthy?” from “was the answer correct?” because the right remediation depends on which layer broke first, not on which layer looked most visible at the end.

Related resources from NHI Mgmt Group

• What is the difference between AI access control and AI output control?
• What is the difference between output quality and accountability in AI agents?
• What is the difference between input guardrails and output guardrails in an AI gateway?
• What is the difference between input filtering and output filtering in AI safety controls?