Recomposition risk

The possibility that an AI system will combine individually authorised fragments into a new output that exceeds the intent of the original access rules. It is a control problem because permissioning for source data does not automatically govern what a model can synthesise.

Expanded Definition

Recomposition risk describes the control gap that appears when an AI system can lawfully access many small pieces of information, then assemble them into an output that reveals more than any single source was meant to expose. In NHI Management Group terms, the issue is not merely data access, but inference through synthesis: the model may remain within individual permissions while still producing a higher-risk combined answer. This is especially relevant in retrieval-augmented generation, agent workflows, and enterprise copilots that can query multiple repositories before responding.

The concept sits at the boundary between access control, information governance, and prompt-mediated data use. Traditional permission models assume the sensitivity of each source is sufficient, but recomposition shows that authorised fragments can become sensitive in aggregate. The closest governance lens in NIST Cybersecurity Framework 2.0 is the need to manage data handling risk across the full lifecycle, not only at the point of retrieval. Definitions vary across vendors on whether this is treated as a privacy issue, an AI safety issue, or a data leakage issue, and that ambiguity can delay ownership.

The most common misapplication is assuming source-level access controls are enough, which occurs when teams review permissions on databases or documents but ignore what the model can infer after combining them.

Examples and Use Cases

Implementing controls against recomposition risk rigorously often introduces friction for users and higher engineering complexity, requiring organisations to weigh answer quality and workflow speed against tighter synthesis boundaries.

• An employee asks an internal assistant for “the status of all pending investigations,” and the model combines authorised fragments from case notes, ticket metadata, and calendar entries into a summary that reveals confidential trends.
• A procurement copilot joins vendor pricing, contract terms, and email excerpts into a consolidated answer that exposes commercial position data no single source explicitly authorised for disclosure.
• A support agent uses a knowledge assistant over segmented incident records, but the model recombines redacted fields with context from adjacent systems and reconstructs sensitive personal data.
• An engineering assistant reads design docs, logs, and runbooks from separate repositories and synthesises a deployment path that unintentionally discloses internal architecture constraints.
• In a governed RAG deployment, policy allows retrieval from multiple sources, but output filtering is used to reduce synthesis risk before the response leaves the system, consistent with the control intent of NIST SP 800-53 Rev 5 Security and Privacy Controls.

Why It Matters for Security Teams

Recomposition risk matters because security teams can approve every input source and still fail to control the output. That makes it a governance problem as much as a technical one. If an AI system is allowed to reason across multiple datasets, the organisation must decide whether the permissible unit is the document, the field, the conversation, or the final answer. Without that decision, policy enforcement becomes inconsistent and audit evidence becomes weak.

This is particularly important where AI sits inside identity and access workflows. A copilot that helps with access reviews, entitlement analysis, or privileged operations may lawfully see fragments of records tied to users, roles, and secrets, yet still create a consolidated view that should never be exposed outside the review context. In NHI and agentic AI environments, recomposition risk also affects tool-using agents that can chain calls across systems and produce new privileged knowledge from individually authorised steps. The right response is not only to restrict retrieval, but to classify outputs, constrain synthesis paths, and test for unintended aggregation.

Organisations typically encounter the impact only after an AI response exposes more context than any single source intended, at which point recomposition risk becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• Why is DevOps such a significant source of NHI risk?
• What is the biggest long-term risk of unmanaged NHIs multiplying at exponential rates?
• What is secrets sprawl and why does it create security risk?
• What is credential injection risk and how does it occur?