Open-source AI security depends on data custody across agents

At a glance

What this is: This is Cyberhaven’s argument that open-source AI security must be data-centric, with controls that track sensitive information across models, harnesses, agents, and infrastructure.

Why it matters: It matters to IAM practitioners because AI agents and adjacent workflows are becoming identity-relevant systems that access data, invoke tools, and expand privilege boundaries beyond traditional account governance.

👉 Read Cyberhaven's post on open-source AI security and the Open Secure AI Alliance

Context

Open-source AI security is increasingly a data-governance problem, not just a model-selection problem. As enterprises mix frontier models, open models, specialist models, and agent frameworks, the security question becomes whether sensitive data can be traced and controlled across prompts, tools, endpoints, cloud services, and business applications.

That creates a real intersection with identity and access governance. When an AI agent can access files, invoke tools, and pass information onward, the programme has to understand which identities, credentials, and authorisations enabled that path, and where the data-custody boundary was crossed.

Key questions

Q: How should organisations govern external tools used by AI agents?

A: Organisations should review external tools as security inputs, not convenience features. Each tool needs ownership, approval, metadata inspection, and ongoing monitoring for hidden instructions or unexpected behaviour. If an AI agent can act on a tool, then the tool’s provenance and control status should be governed like any other sensitive integration.

Q: Why do open-source AI environments create a data-governance challenge for security teams?

A: Because the security risk moves across models, harnesses, endpoints, cloud services, and business applications rather than staying inside one platform. The same sensitive data can be accessed, transformed, and forwarded by different components, so teams need a consistent control layer that follows the data wherever the workflow goes.

Q: What do security teams get wrong about prompt engineering for AI agents?

A: They often assume better wording is enough to create reliable control. In practice, prompt style can help, but it does not create a secure boundary when the agent is still free to reinterpret context. Real governance comes from structure, validation, and constrained action paths.

Q: How do you know if AI data trust controls are actually working?

A: Look for three signals: data is classified, access decisions are enforced where the data is touched, and non-human identities are visible in logs and reviews. If teams still need long manual approval loops to understand what an AI system can see, the control model is not working at runtime.

Technical breakdown

Chain of custody for AI data flows

Traditional security inspection often stops at a single interaction, such as a prompt, upload, or API call. In agentic workflows, that is too narrow because the same piece of sensitive data can move through multiple systems, be transformed, and be forwarded by another action. A chain-of-custody view reconstructs the origin, access path, transformation, and destination of the data so policy can be applied to the full workflow rather than one event. For open-source AI, that matters because the model and harness may change while the data risk remains constant.

Practical implication: map AI telemetry to the data path, not only to the model invocation.

Why agent identity changes the control problem

AI agents are not just text generators. They access files, invoke tools, query systems, and communicate with other agents, which means the security model must treat them as runtime entities with distinct permissions and behaviours. That does not make every agent autonomous, but it does mean authorisation, logging, and policy enforcement need to follow the agent at execution time. In IAM terms, the control boundary shifts from static account entitlements to task-scoped access and observable runtime behaviour.

Practical implication: bind agent actions to explicit identities, scoped credentials, and auditable tool permissions.

Model-agnostic controls and the governance baseline

The article’s central claim is that security should not depend on which model or harness is in use. That is the correct baseline for enterprises that will not standardise on a single AI stack. A model-agnostic approach separates policy from implementation, which helps maintain consistent handling of sensitive data even as infrastructure, frameworks, and vendors change. This is especially important where AI systems intersect with regulated data, because inconsistent enforcement creates blind spots that policy teams cannot reconcile after the fact.

Practical implication: define security policy once and enforce it across every AI runtime and integration path.

NHI Mgmt Group analysis

Data custody, not model novelty, is the core AI security problem. The article is right to shift attention away from the model layer and toward the movement of sensitive data across AI workflows. That is where governance failures become visible, because prompts, files, tool calls, and outputs can each widen exposure if they are not continuously correlated. For practitioners, the implication is that AI governance has to be built around data lineage and policy enforcement, not only model approval.

AI agents create an identity problem inside the security stack. Once an agent can read files, call tools, and pass work to other systems, it behaves like a runtime identity with delegated authority. That means conventional IAM thinking becomes relevant again, but only if teams recognize that access review alone is too static for live agent behaviour. The practical conclusion is to govern agents as identities with scoped permissions, lifecycle controls, and evidence of action.

Model-agnostic security is becoming a governance requirement rather than a design preference. Enterprises will keep changing models, harnesses, and execution environments, so policy has to outlast the implementation. This is where the article aligns with broader cloud and AI governance trends: security architectures that depend on a single platform create future control drift. The practitioner takeaway is to separate control intent from model choice so AI governance remains portable.

Open-source AI increases transparency, but transparency only helps if policy can consume it. More openness can improve inspection, evaluation, and incident response, yet the value is lost if telemetry is fragmented across tools and runtime layers. A usable governance model needs observable data movement, identity linkage, and clear accountability for agent-driven actions. For security teams, that means linking AI oversight to existing IAM, PAM, and data-protection processes rather than creating a disconnected AI-only programme.

Chain-of-custody controls are the named concept this topic should be read through. The article describes a need to reconstruct what data an AI system accessed, where it came from, how it changed, and where it went. That is a governance model for evidence, not just detection, and it is likely to become the deciding control pattern for enterprise AI oversight. Practitioners should treat chain of custody as the bridge between AI monitoring and enforceable policy.

What this signals

Chain-of-custody governance will become a practical requirement for AI programmes that cross model, endpoint, and cloud boundaries. Security teams should prepare to correlate data movement with identity events, because agent behaviour will increasingly produce evidence that conventional access reviews do not capture.

The control conversation is also moving closer to IAM and PAM. Once agents are allowed to invoke tools or reach business systems, their authorisations need lifecycle management, and the programme will need a way to prove that delegated access stayed within policy.

For teams building AI oversight, the next step is not a new dashboard. It is a control model that can survive model swaps, framework changes, and growing agent autonomy while preserving accountability for every sensitive-data path.

For practitioners

• Define AI chain-of-custody requirements Require every AI workflow to record the data source, access path, transformation steps, and destination so security teams can prove what happened to sensitive information.
• Bind agent actions to scoped identities Assign each agent a distinct runtime identity with least-privilege tool access, short-lived credentials, and logs that tie actions back to a specific execution context.
• Separate policy from model selection Write policy for data handling, authorisation, and redaction once, then enforce it across open models, proprietary models, and different harnesses.
• Extend incident response to AI data movement Add triage steps for agent-driven exfiltration paths, including file access review, tool invocation history, and downstream sharing events across cloud and endpoint environments.

Key takeaways

• Open-source AI security is fundamentally a data-custody problem, because risk follows the movement of sensitive information across agents and tools.
• AI agents behave like runtime identities, so IAM-style controls must extend into tool permissions, logging, and delegated access boundaries.
• Enterprises need model-agnostic policy enforcement, or governance will fragment every time the AI stack changes.

Key terms

• Chain of custody: A documented record that preserves the integrity of evidence from the moment an event is detected through investigation and response. In identity and data protection workflows, it helps prove what happened, when it happened, and which actor or session was involved.
• Runtime Identity: Runtime identity is the practice of making identity and authorization decisions at the moment an action occurs. For agents and workloads, it means access is validated against live context, not only against the identity state set during onboarding or provisioning. That makes accountability and scope enforcement possible inside fast-moving workflows.
• Identity-Centric Data Security: Identity-centric data security is the practice of governing sensitive data through the identities that can reach it, not only through storage controls. It connects entitlement, context, and auditability so organisations can explain and limit access across humans, machines, and AI agents.
• Model-Agnostic Control: Model-agnostic control means the security policy is independent of any single AI model or framework. This lets organisations keep the same governance rules in place as they switch models, adopt new harnesses, or move workloads across cloud, on-premises, and endpoint environments.

What's in the full article

Cyberhaven’s full post covers the operational detail this post intentionally leaves for the source:

• How its data-centric AI Security product reconstructs the complete chain of custody across prompts, files, tools, and outputs.
• What visibility and enforcement look like across open models, proprietary models, cloud infrastructure, endpoints, and on-premises systems.
• Why identity, isolation, evaluation, runtime visibility, and data protection are treated as the supporting security layers for open-source AI.
• How the vendor frames the alliance context around open source, AI infrastructure, and cybersecurity.

👉 Cyberhaven's full post covers the data-custody model, agent visibility, and the security layers behind its AI Security product.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management. It gives practitioners a practical framework for governing access, delegation, and lifecycle controls across identity-driven systems.