Copilot email access shows the AI governance gap in agentic tools

At a glance

What this is: Microsoft 365 Copilot was reported accessing confidential emails outside intended scope in Outlook Sent and Draft folders, exposing a gap in how agentic AI respects data labels and access boundaries.

Why it matters: IAM, data security, and AI governance teams need to treat agentic AI as a governed access path, because label-based controls and human-only privacy assumptions do not reliably contain machine-initiated access.

👉 Read Swarmnetics's analysis of Copilot accessing confidential emails

Context

Agentic AI tools can act on data, not just generate text, which means they inherit access and governance risks that traditional assistant software did not create. In this case, the primary issue is not model quality but whether a system that can read, summarise, and potentially retain confidential content is operating inside the boundaries set by IAM, data handling, and internal privacy policy.

For identity and governance teams, the key question is whether AI tools are being treated as controlled consumers of enterprise data or as trusted extensions of the user. When access rules depend on folder location, metadata, or assumptions about what the AI will not do, the programme is already relying on behaviour that may not be consistently enforced.

Key questions

Q: How should security teams govern data access for agentic AI workflows?

A: Security teams should treat data access as part of the agent’s decision boundary, not as a separate storage problem. Scope access by use case, classify the datasets that influence actions, and verify that policies can constrain runtime behaviour as agents select tools and next steps. The goal is to prevent an agent from turning broad data reach into uncontrolled action.

Q: Why do confidential labels often fail to contain AI access risk?

A: Because labels only work if the enforcement layer applies them consistently across every path the AI can use. If the assistant can read a message through Sent, Draft, search, or cached references, the label has not truly constrained access. The control must be enforced on the object, not on the interface.

Q: What breaks when AI assistants retain summaries of confidential content?

A: The programme loses control over derivative data. A summary may be less sensitive than the source, but it still becomes a governed artefact that can be stored, searched, shared, or exposed through backend systems. Without retention and classification rules, the original access event multiplies into a longer-lived exposure problem.

Q: Who is accountable when an AI agent accesses sensitive data it was not meant to use?

A: Accountability sits with the team that approved the agent, its connectors, and its policy boundaries, not with the runtime behaviour alone. Organisations need ownership for intent, permissions, monitoring, and validation so they can prove whether the agent stayed inside its approved purpose. Without that, audit and regulatory response become retrospective guesswork.

Technical breakdown

Why folder-based access controls fail for agentic AI

Agentic AI features often operate through application-level permissions rather than a distinct identity boundary. That means the tool can inherit broad mailbox access and then apply its own logic to decide what to summarise or surface. If the implementation only recognises certain folders or labels, data that sits elsewhere in the same mailbox can fall outside the intended control plane even though it remains sensitive. The failure is not in encryption or authentication alone. It is in assuming that human-oriented information architecture maps cleanly to machine-driven access patterns.

Practical implication: review whether AI tools enforce policy at the data object level, not just at the folder or interface level.

Metadata and retention create secondary exposure paths

Once an AI system ingests confidential content, the risk extends beyond the immediate output. Summaries, prompts, telemetry, and stored references can create additional copies, fragments, or derivatives that sit in logs and backend services. Even when a vendor states that training is excluded, that does not automatically eliminate retention, processing, or internal reuse risk. For identity governance, this matters because access is no longer a simple yes or no. It becomes a question of how long the AI can observe data, what it stores, and who can later retrieve those artefacts.

Practical implication: classify AI-generated outputs, prompt history, and cached references as governed data assets with explicit retention rules.

Agentic AI behaves like an identity consumer, not a passive feature

An AI tool that can read mail, summarise it, and act across workflows is effectively participating in access decisions. That makes it closer to a non-human identity than a conventional software utility, because its runtime behaviour can change what data is touched and when. The governance challenge is therefore not only model safety but delegated access control, auditability, and revocation. If the organisation cannot trace what the tool accessed, why it accessed it, and what downstream data it produced, the access model is too weak for agentic use.

Practical implication: bring agentic AI into the same control conversation as NHI, privileged access, and audit logging.

Threat narrative

Attacker objective: The objective is not necessarily external compromise but unintended access expansion that lets the AI process confidential information outside the user’s intended trust boundary.

1. Entry occurs when the AI assistant is granted mailbox access that is broader than the intended confidential-email policy boundary.
2. Escalation happens when the tool can summarise content from Sent and Draft folders even though those messages were marked confidential.
3. Impact is the exposure of sensitive business data into AI outputs, stored references, or backend processing paths that are harder to govern than the original mailbox content.

Breaches seen in the wild

• Replit AI Tool Database Deletion — Replit vibe coding AI assistant deletes live production database and creates 4,000 fake user records.
• Meta AI Instagram Account Takeover — 20,225 Instagram accounts hijacked via compromised Meta AI support chatbot with overprivileged access.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Agentic AI policy gaps are now an identity governance problem, not just an AI safety issue. When a tool can read sensitive content and produce new artefacts from it, it is operating as a non-human identity with delegated authority. That means IAM, PAM, and data governance all need to participate in the control model. The practitioner conclusion is straightforward: if the AI can access it, summarise it, or store it, then it is in scope for identity governance.

Label-based confidentiality controls are too brittle for agentic systems. This incident shows that mailbox labels or folder logic can fail to express actual data sensitivity when the AI is allowed to traverse multiple content locations. The governance assumption that a human-defined label will constrain a machine consumer is unreliable unless the enforcement layer is centralised and testable. Practitioners should treat label enforcement as advisory until proven otherwise.

AI memory and retention create a new derivative-data problem. Even if source mail is not directly exposed, summaries, metadata, and service-side artefacts can extend the exposure window. That creates a governance debt similar to unmanaged NHI secrets: the original access event is only the first control point. The practitioner implication is to govern AI outputs and processing records as part of the same control domain as the source content.

Agentic AI security will increasingly converge with NHI controls. The more an AI system can act across workflows, the more it resembles a privileged service account with dynamic behaviour. That makes OWASP NHI, NIST CSF, and access governance relevant in the same conversation as AI safety frameworks. The field should stop asking whether AI is autonomous enough and start asking which delegated privileges it actually exercises in production.

Named concept: AI access boundary drift. This is the gap between the access a system is authorised to hold and the data it can practically touch once runtime behaviour, metadata, and content traversal are involved. The concept matters because many AI control failures arise not from stolen credentials but from drift between policy intent and actual machine access. Practitioners should test for boundary drift whenever AI tools are allowed to read or transform sensitive data.

From our research:

• 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems (39%), inappropriately sharing sensitive data (31%), and revealing access credentials (23%).
• Only 52% of companies can track and audit the data their AI agents access, leaving 48% with a complete blind spot for compliance and breach investigation.
• OWASP NHI Top 10 is the practical next read for teams mapping agentic AI behaviour to identity and privilege risk.

What this signals

AI access boundary drift: organisations will need to test whether AI assistants obey policy at the object level, not just at the UI level. That means mailbox labels, document tags, and retention rules must be verified against real access paths, including summaries and stored references. The governance test is no longer whether the AI is useful, but whether its runtime behaviour stays inside the approved access boundary.

The shift from human-only privacy assumptions to machine-mediated access will force identity teams to extend audit design to AI read actions, derived outputs, and stored artefacts. That aligns naturally with NIST AI Risk Management Framework thinking, but the operational control question remains identity-led: who authorised the AI, what did it touch, and what evidence proves containment?

The programme implication is straightforward. If agentic tools can read confidential data, they belong in the same governance review cycle as privileged service accounts, workload identities, and other non-human identities. The organisations that build those controls now will have a clearer path to scaling AI without turning every productivity feature into a data exposure problem.

For practitioners

• Map AI tools to delegated identity scope Inventory every mailbox, document, and workflow permission granted to AI assistants, then compare that scope with the data they actually touch during normal use. Pay close attention to Sent, Draft, shared mailboxes, and delegated folders.
• Enforce object-level sensitivity controls Validate that confidential labels are enforced at the object or record level, not only through folder placement or client-side UI rules. Test whether the same message is blocked consistently across inbox, sent items, drafts, search, and summary functions.
• Treat AI outputs as governed derivatives Classify summaries, prompt history, and cached references as controlled data with retention and access rules. Add review points for where these artefacts are stored, who can retrieve them, and how long they persist.
• Extend audit logging to AI read actions Capture when the AI accesses sensitive content, which policy decision allowed it, and what derived output was created. Use those logs to support incident review, compliance evidence, and misuse detection.

Key takeaways

• Agentic AI turns data access into a delegated identity problem when it can read and summarise confidential content.
• Label-based controls are not enough if the AI can reach the same data through alternate folders, cached references, or retained summaries.
• The immediate governance response is to bound AI scope, classify derived outputs, and prove auditability before broader deployment.

Key terms

• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.
• Non-Human Identity (NHI): A digital identity assigned to a non-human entity such as a software application, service account, API key, bot, machine, or AI agent that enables it to authenticate and interact with systems without direct human involvement. NHIs now outnumber human identities in most enterprises by 25 to 50 times.
• AI Access Boundary Drift: AI access boundary drift is the gap between the access a system is supposed to have and the data it can actually reach once runtime behaviour, search paths, metadata, or retention come into play. It is a governance failure, not just a technical bug.
• Derived Data: Derived data is information created from original content through analysis, transformation, or extraction. For image AI, that includes descriptions, labels, and OCR text, all of which may carry the same sensitivity as the source and therefore need equivalent handling.

What's in the full analysis

Swarmnetics's full analysis covers the operational detail this post intentionally leaves for the source:

• The exact Outlook folder behaviour that caused confidential items in Sent and Draft to remain accessible to Copilot.
• The vendor's stated privacy and retention handling for Microsoft 365 data, including what is and is not used for training.
• The patch timing and product-specific workflow details practitioners would need to validate their own controls.
• The practical implications for businesses deciding whether to limit AI access to sensitive mailboxes and similar content.

👉 Swarmnetics's full post covers the Outlook folder edge case, privacy posture, and containment implications.

Deepen your knowledge

NHI Mgmt Group covers identity security, NHI governance, and agentic AI through the NHI Foundation Level course, the industry's only accredited NHI security programme. It gives practitioners a common control language for managing delegated access, secrets, and machine identities across modern programmes.