Open-source AI security depends on data custody across agents

At a glance

What this is: Cyberhaven argues that open-source AI security depends on data custody controls that follow information across prompts, tools, agents, and outputs.

Why it matters: IAM, NHI, and AI security teams need this lens because agent-driven workflows break single-transaction controls and make policy enforcement dependent on end-to-end data lineage.

Context

Open-source AI changes the control problem from model selection to data custody. When prompts, tool calls, agent actions, and outputs all touch sensitive information, the security question becomes whether an organisation can trace where data moved, who or what handled it, and whether that use stayed within policy.

That shift matters for IAM and NHI governance because agents act as software identities that can access files, invoke tools, query systems, and communicate with other agents. Traditional controls built around a single request or session do not fully explain how data is transformed across an agent workflow, so enforcement has to follow the data path rather than the interface boundary.

Key questions

Q: How should security teams govern data access for agentic AI workflows?

A: Security teams should treat data access as part of the agent’s decision boundary, not as a separate storage problem. Scope access by use case, classify the datasets that influence actions, and verify that policies can constrain runtime behaviour as agents select tools and next steps. The goal is to prevent an agent from turning broad data reach into uncontrolled action.

Q: Why do AI agents make non-human identity governance harder?

A: AI agents make governance harder because they can request tools, act autonomously, and change behaviour across sessions while still relying on machine credentials. That increases the number of access paths security teams must supervise. The result is a stronger need for task-scoped access, explicit ownership, and continuous monitoring of what the agent can reach.

Q: What breaks when prompt instructions are used as a security control?

A: Prompt instructions fail when they are treated as the source of truth for permissions, data boundaries, or action approval. They can be leaked, overwritten, or ignored by malicious input. Security rules need to live in deterministic application logic and downstream enforcement points, not only in natural-language prompts.

Q: How do organisations keep AI security consistent when models and harnesses change?

A: They need a control plane that applies the same data-policy logic across models, frameworks, endpoints, and cloud environments. If policy is tied to one model or harness, every platform change becomes a security change, which is too fragile for enterprise operations.

Technical breakdown

Why prompt inspection is not enough for agentic AI

Prompt inspection treats each interaction as a discrete event, but agentic workflows chain multiple events together. A prompt may include copied code, customer records, or other sensitive material. The agent can then call tools, fetch context, rewrite content, and hand results to another system. The security failure is not the prompt itself, but the inability to preserve custody context as data crosses models, harnesses, and execution environments. That makes policy enforcement dependent on lineage, not just content scanning.

Practical implication: instrument AI workflows so policy decisions can see the full path of a datum, not only the last prompt or output.

How data custody maps to non-human identity governance

An AI agent is not just a model outputting text. It is a runtime identity that can hold permissions, trigger tools, and act across systems on behalf of a task. That creates a governance problem familiar to NHI programmes: if the agent can access sensitive data, the organisation needs to know what identity was used, what scope it held, and whether that scope matched the task. Without that mapping, access review and audit evidence become incomplete because the action and the data movement are separated.

Practical implication: treat AI agents as governed non-human identities and align their access scope, ownership, and review process to the task lifecycle.

Why model- and harness-agnostic controls matter

The article’s core argument is that security should not depend on which open model, proprietary model, or harness is in use. That is a practical architecture point, not a branding point. Enterprises will keep changing models, frameworks, and infrastructure, so the durable control plane has to sit above those choices and enforce the same data policies everywhere. This is where open-source AI security becomes a governance issue rather than a model preference issue.

Practical implication: define policy at the data and identity layer so model swaps do not reset your security assumptions.

Threat narrative

Attacker objective: The attacker’s objective is to move sensitive enterprise data through an AI workflow in a way that becomes hard to trace, govern, or prove compliant.

1. Entry occurs when an AI workflow receives sensitive material in a prompt, file upload, or tool-mediated context.
2. Credential and data access expand as the agent invokes tools and reaches across systems to collect, transform, or relay information.
3. Impact follows when the workflow moves protected data into new destinations without a reliable custody trail or policy check.

Breaches seen in the wild

• Nx s1ngularity attack 2025: Attackers stole Nx's npm token via a GitHub Actions flaw and shipped malware that stole 2,349 secrets and abused developers' AI CLIs.
• LiteLLM PyPI package breach: LiteLLM PyPI supply chain attack, credentials stolen from users.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Data custody is becoming the defining security control for open-source AI. The main problem is no longer which model is in use, but whether sensitive data can be traced as it moves through prompts, tools, agents, and outputs. That makes lineage, policy persistence, and runtime visibility more important than model preference. Practitioners should treat custody as the security boundary that survives model churn.

Agentic AI introduces a non-human identity governance problem that most security stacks do not model cleanly. When an agent can access files, call APIs, and hand data to other systems, it behaves like a governed software identity rather than a passive model. That intersection matters for IAM and NHI programmes because access scope, ownership, and auditability now have to cover machine decision-making as well as machine access. Practitioners should align AI controls with NHI governance, not just AI safety.

Model-agnostic security is becoming the only durable posture for enterprise AI. Open-source and proprietary stacks will continue to coexist, and infrastructure will keep changing underneath them. That means organisations need controls that sit above the harness and enforce the same policy logic across endpoints, cloud, and business applications. Practitioners should design for portability in security controls, not just portability in model choice.

Open Secure AI Alliance participation signals a category shift from model governance to control-plane governance. The important question is no longer whether an organisation can run a model, but whether it can govern how data is authorised, transformed, and retained across the AI workflow. This is the point where AI governance and identity governance converge. Practitioners should evaluate whether their current controls can survive agentic workflows without losing accountability.

Chain-of-custody visibility is the named concept that security teams should carry forward from this article. It captures the need to prove where data came from, who or what touched it, how it changed, and where it ended up. In agentic environments, that proof becomes the basis for both compliance and containment. Practitioners should build controls around evidentiary custody, not around isolated prompts.

From our research library:

• 28% of secrets incidents now originate outside code repositories, in Slack, Jira, and Confluence, and are 13% more likely to be categorised as critical than code-based leaks, according to the State of Secrets Sprawl 2026.
• Read next: Agentic AI Security Guide

What this signals

Data custody will become a board-relevant control issue as agentic AI spreads. Security teams will be judged less on which models they allow and more on whether they can prove where sensitive data travelled and why. That shifts the operational question from model approval to evidentiary control.

Chain-of-custody visibility is the most useful design principle for open-source AI programmes. Once agents can query systems and pass data onward, the security boundary has to move from the model to the workflow. Controls that cannot survive model swaps will not survive enterprise AI adoption.

Secrets now move well beyond repositories, according to the State of Secrets Sprawl 2026, with 28% of secrets incidents originating in Slack, Jira, and Confluence and 13% more likely to be critical than code-based leaks.** That is a reminder that AI data custody and secret governance are converging in the same operational sprawl.

For practitioners

• Implement chain-of-custody visibility for AI workflows Track where sensitive data originated, which agent or system touched it, how it changed, and where it moved so policy decisions survive model and harness changes.
• Govern AI agents as non-human identities Assign ownership, task scope, and access boundaries to each agent so credentials, tool use, and data access are reviewable as part of NHI governance.
• Separate policy from model choice Write data-security rules once at the control plane and enforce them across open models, proprietary models, endpoints, cloud workloads, and business apps.
• Log tool use and downstream data movement Record each tool invocation, system query, and output destination so investigations can reconstruct the full agent path after a policy breach or misuse event.

Key takeaways

• Open-source AI security depends on controls that can trace data as it moves across prompts, tools, agents, and outputs.
• AI agents create a governance problem because they act as software identities with access, tool use, and data-handling authority.
• Enterprises need policy that survives model changes, harness changes, and infrastructure changes without losing custody evidence.

Key terms

• Chain of custody: A documented record that preserves the integrity of evidence from the moment an event is detected through investigation and response. In identity and data protection workflows, it helps prove what happened, when it happened, and which actor or session was involved.
• Agentic workflow: An agentic workflow is a sequence of tasks executed by an AI agent with some level of tool access and decision authority. In security terms, the workflow matters because it can span multiple systems, identities, and permissions, which makes attribution and revocation harder than with ordinary automation.
• Non-Human Identity (NHI): A digital identity assigned to a non-human entity such as a software application, service account, API key, bot, machine, or AI agent that enables it to authenticate and interact with systems without direct human involvement. NHIs now outnumber human identities in most enterprises by 25 to 50 times.
• Model-Agnostic Security Workflow: A model-agnostic security workflow accepts findings from any approved AI model without changing the underlying governance process. The point is consistency: one intake, one validation standard, one remediation path, so tool changes do not weaken accountability or introduce parallel exception handling.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It helps security practitioners translate identity controls into operational governance for agents, workloads, and access paths.