How should organisations handle AI models that retain prompts and outputs by default?

Why default retention changes the governance question

When an AI model keeps prompts and outputs after the session, the issue is no longer just “can people use the tool?” It becomes a data-handling decision about what enters the model, who can review it later, how long it persists, and whether the organisation can justify that processing for the intended use case. That shifts the control point from convenience to governance.

Retention matters because prompts often contain source code, customer data, incident details, credentials, or regulated information that users would not normally expect to persist inside a third-party system. Once retention exists, organisations need to decide whether the model is being used for low-risk drafting, controlled internal work, or sensitive workflows where persistence would be unacceptable without stronger safeguards.

How retention affects classification, access, and approval

Retention should be matched to data classification before the tool is widely approved. If a model stores content beyond the session, the organisation needs to know whether that content can be reviewed by the vendor, retained for service improvement, accessed by administrators, or surfaced in audit or support workflows. That is why approval should be tied to approved use cases, not just to the product name or the vendor’s default settings.

This is where data minimisation becomes practical. Teams should define which classes of data may be submitted, which must be redacted, and which should never be entered into a retained model at all. For many organisations, the safest pattern is to prohibit sensitive submissions unless the retention behaviour is contractually constrained and the review path is explicit.

Where retention is unavoidable, AI and privacy guidance helps frame the question around processing purpose, personal data exposure, and post-session persistence rather than treating the model as a simple interface.

What organisations should verify before allowing use

Before approving a retained model, organisations should verify the vendor’s retention defaults, the admin controls available to disable or limit retention, the audit trail for reviewer access, and whether submitted content can be excluded from training or human review. They should also confirm how prompts and outputs are segregated by tenant, workspace, or account, because retention without separation can create broad unintended exposure.

Security, privacy, and legal sign-off should be explicit when sensitive data is in scope. That sign-off should confirm not only the business need, but also the legal basis, retention period, access boundaries, and incident response expectations if retained content is later exposed or misused.

For teams building usage standards, the AI Governance Guide is useful for structuring approvals, roles, and use-case review so retention is governed as part of the broader AI operating model.

Risk and Threat Considerations

Default retention increases the blast radius of ordinary user behaviour. A prompt that would have disappeared after a session may instead become recoverable content, creating exposure through overbroad reviewer access, support workflows, tenant compromise, or later discovery during legal or administrative review. The same persistence can also make sensitive prompts available to insiders or external attackers if account controls fail.

Failure mechanism: Sensitive input is retained longer than intended, then exposed through administrator access, mis-scoped review permissions, weak segregation, or vendor-side compromise. That turns a transient interaction into durable stored data with a larger attack and governance surface.

Impact: The organisation can face confidentiality loss, regulatory exposure, loss of customer trust, and harder incident response because the prompt or output may now exist in multiple places outside the original user’s control.

For prompt-heavy workflows, generative AI data loss prevention guidance is especially relevant because it addresses prompt redaction, oversharing, and retention-aware controls for everyday use.

Practitioner Guidance

What to prioritise: Start with the highest-risk use cases, not the most common ones. If a model may retain prompts containing customer data, code, secrets, legal material, or incident details, treat that path as requiring formal approval rather than informal adoption.

Decision rule: If users cannot clearly explain what is retained, who can see it later, and how long it persists, the model should not be approved for sensitive content. Ambiguity about post-session storage is itself a control failure.

What to verify: Confirm the vendor’s retention defaults, the availability of admin controls, the reviewer-access model, and whether content is excluded from training or human review under the chosen plan. Verify that the permitted use cases are written down and communicated to users.

Practitioner takeaway: Retention is a data-governance and access-control decision, so the safe default is to approve the workflow only when persistence, review access, and sensitive-data boundaries are all explicit.

Related resources from NHI Mgmt Group

• How should security teams handle risks from AI browser extensions?
• How should organisations test AI models that handle sensitive data?
• How can organisations govern AI tools that may route prompts to different models?
• How should organisations govern AI models that cannot fully explain their outputs?