Encryption protects data in transit and at rest, but it does not stop an authorised identity from over-collecting, reusing, or disclosing data through the model. Privacy risk persists when the approved request becomes a broader inference, export, or training action. The control question is who can access what, for which purpose, and whether that access is still justified.
Why This Matters for Security Teams
Encrypted data can still become a privacy incident when an authorised system can read, combine, or export information beyond the original user intent. That is especially true for AI systems, where a model can transform a narrow prompt into a broader inference, a downstream lookup, or a training artefact. NIST’s Cybersecurity Framework 2.0 is useful here because it treats governance, access, and data handling as operational controls, not just cryptographic ones.
For security teams, the issue is not whether encryption works. It is whether the identity behind the request is still entitled to see, retain, or reuse the data after the request changes shape. NHIMG research on the Ultimate Guide to NHIs — Why NHI Security Matters Now highlights how often non-human access is trusted too broadly once it is authenticated. In practice, many security teams encounter privacy exposure only after a model has already copied, summarized, or leaked sensitive content into places encryption was never designed to protect.
How It Works in Practice
Privacy risk appears when the protected object is no longer the only concern. Encryption secures storage and transport, but AI systems operate on decrypted content inside trusted runtime environments. Once a model, agent, or integration can access that plaintext, the security question shifts to purpose limitation, minimisation, and disclosure control. That is why current guidance increasingly treats data governance as a runtime problem, not just a perimeter problem.
A practical control stack usually includes:
- Data minimisation before the prompt or workflow is assembled, so the model never sees unnecessary fields.
- Purpose-bound authorisation, where access is allowed only for the specific task the system is performing.
- Short-lived credentials and scoped tokens for AI workloads, so reuse is limited if the system branches into a new action.
- Logging and redaction for prompts, outputs, and tool calls, because privacy failures often occur in the orchestration layer rather than the encrypted store.
NHIMG’s Top 10 NHI Issues and the OWASP NHI Top 10 both reflect the same operational reality: if an identity can call a model, the model can become a disclosure path unless access is tightly scoped. For control design, teams should align this with NIST guidance on least privilege and with policy-driven enforcement at the request layer, not just the storage layer. In environments where models can chain tools, cache context, or write into shared knowledge bases, encrypted data still leaves the organisation exposed because the unencrypted processing path is where misuse happens.
These controls tend to break down when systems reuse a broad service account across many prompts, tools, and tenants because the model can legally access too much once it is authenticated.
Common Variations and Edge Cases
Tighter prompt filtering and access scoping often increases friction for product and data teams, requiring organisations to balance privacy protection against workflow speed and model usefulness. Best practice is evolving, and there is no universal standard for every AI deployment yet, especially where retrieval, fine-tuning, and agentic tool use overlap.
One common edge case is the internal assistant that is “encrypted end to end” but still connected to HR, CRM, or code repositories through a service identity. Another is model training or feedback collection, where data is encrypted in transit yet still repurposed into future outputs. The risk also changes when the system handles regulated data, because even a small amount of inferred metadata can be privacy-sensitive if it reveals health, employment, or location patterns.
Security teams should treat this as a question of identity, scope, and retention. The most reliable control is not stronger encryption alone, but a clear rule for what the AI is allowed to see, why it can see it, how long it can retain it, and when the access must be revoked. NHIMG’s Ultimate Guide to NHIs — Key Research and Survey Results reinforces that non-human access becomes risky quickly when governance lags behind operational reality.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 address the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-03 | Controls over secret scope and rotation matter when AI can reuse decrypted access. |
| NIST CSF 2.0 | PR.AA-01 | Identity proofing and access governance are central when encryption does not stop misuse. |
| NIST AI RMF | AI RMF addresses governance and harmful information handling beyond cryptography. |
Use AI RMF GOVERN and MAP functions to define privacy purpose limits and review model data use.
