AI platform trust depends on seven controls for models and agents

At a glance

What this is: This is a Collibra analysis of seven non-negotiable controls for trusted AI platforms, arguing that inventory, context, runtime enforcement and evidence are what make model and agent governance real.

Why it matters: IAM and governance teams need this because AI trust now depends on whether controls survive runtime execution, not whether a platform can describe policies after the fact.

Context

An AI platform is only governable if you can see every model and agent, know what each is allowed to do, and verify those rules while it is running. That makes trust an identity and access problem as much as a data or model problem, because the decision boundary is now shared by models, agents and the controls around them.

The article argues that static governance breaks once an agent can act on live data before a human review happens. In that environment, inventory, context, runtime enforcement and audit evidence become the minimum architecture for accountable AI, not optional maturity features.

Key questions

Q: What breaks when model-level guardrails are treated as security controls for AI systems?

A: Model-level guardrails break down because they are probabilistic safety tendencies, not deterministic enforcement. An attacker can reframe malicious tasks as legitimate work, split them into harmless subtasks, and steer the model into compliance. Security teams should treat model safety as useful but insufficient, then enforce policy in the runtime path where identity, context, and authorization can actually be checked.

Q: Why do AI agents raise the governance bar for trusted platforms?

A: Because agents can act before a human review cycle catches up. That compresses the governance window and makes static approval, periodic certification and post-hoc monitoring insufficient on their own. Trust has to be enforced while the agent is running, not reconstructed afterwards.

Q: How can security teams tell whether AI context is trustworthy?

A: Look for freshness, source, and validation. Trustworthy context has a clear owner, a known observation time, and a live verification step before the system acts on it. If the agent cannot show where a fact came from or whether it still matches reality, the context should be treated as advisory only.

Q: What is the difference between AI observability and AI governance?

A: AI observability tells you what the system did. AI governance decides whether it should have been allowed to do it, who approved it, and what happens when it crosses a policy boundary. Observability is a data problem. Governance is an operating model that combines policy, ownership, evidence, and enforcement.

Technical breakdown

Single inventory and governed context for AI systems

A trusted AI platform starts with a complete inventory of models, use cases and agents, plus governed context that binds each system to definitions, relationships, lineage and quality signals. Without that pairing, organisations lose the ability to know what exists or whether the AI is grounded in the right meaning at runtime. The article's core point is that visibility without context still produces wrong decisions, while context without inventory leaves shadow systems outside governance.

Practical implication: build one authoritative register for models and agents, then tie each entry to governed context before you let it reach production data.

Runtime control over data, actions and agent guardrails

Runtime control is the difference between policy that exists on paper and policy that constrains an AI system while it operates. The article draws a hard line here: if access is reviewed at launch but not enforced at query time, the control is already bypassed once the AI is live. Guardrails matter only when they can pause an agent, block access or restrict actions in the moment the system reaches for data or tools.

Practical implication: verify that access decisions are enforced at execution time, not merely approved during onboarding or model release.

Observability, lineage and audit evidence as trust proof

Trust also requires evidence: decision traces, data-access events, end-to-end lineage and continuous audit records. These controls answer the questions regulators, incident responders and governance teams eventually ask: what happened, what data was touched and how can it be proven. The article treats observability and lineage as proof that the platform stayed within its intended boundary, rather than as monitoring extras.

Practical implication: require traceability from source data to agent action, and make audit evidence continuously available rather than reconstructed after an incident.

Threat narrative

Attacker objective: The objective is to make AI actions unauditable and uncontainable, so consequential decisions can occur outside governance without a reliable record.

1. Entry begins when a model or agent is introduced without complete inventory, which leaves shadow systems and orphaned AI outside governance.
2. Escalation occurs when governed context or runtime control is missing, allowing the AI to act on unverified data or exceed its intended scope while running.
3. Impact follows when decisions, actions and data touches cannot be traced back through lineage or audit evidence, so accountability is lost at the point of failure.

Breaches seen in the wild

• McKinsey AI platform breach: McKinsey AI platform hack exposed 46M chats and sensitive data.
• Hugging Face Spaces breach 2024: Unauthorised access to Hugging Face Spaces may have exposed secrets users stored for AI apps; tokens were revoked and org tokens removed.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Trust in AI platforms is an identity governance problem, not a branding claim. The article's seven controls map directly to the questions identity teams already ask of human and machine access: who exists, what they may reach, when policy is enforced and how proof is produced. Once agents can act on live data, trust becomes an architectural property, not a marketing label. Practitioners should treat AI trust as governed access plus evidence, not as a statement of intent.

Governed context is the missing control that separates useful AI from merely confident AI. Inventory tells you what is present, but context determines whether a model or agent is operating on trusted meaning at runtime. That makes context a governance layer, not a data enrichment feature. The practitioner takeaway is to treat context quality as part of access control for AI decisions, because ungoverned meaning is a control failure, not a model quirk.

Runtime control is where static governance assumptions break. Access review processes were designed for access that persists long enough to be reviewed; agents can make and consume decisions before that review cycle ever sees them. That assumption fails when the actor is autonomous because the relevant control point is execution time, not certification time. The implication is that governance must shift from periodic approval to continuous enforcement over action boundaries.

End-to-end lineage and continuous evidence define accountable AI operations. Without lineage, organisations cannot answer where a decision came from or what it touched, and without continuous evidence they cannot prove whether the platform stayed within policy. This is the same accountability challenge identity teams face in complex delegation chains, only compressed into agent runtime. Practitioners should view lineage as a governance record, not a reporting convenience.

AI platform trust now depends on guardrails that can intervene before harm is complete. The article correctly places the brake inside the platform, not around it, because a guardrail that cannot stop an agent at the moment of drift is only documentation. That shifts the market toward enforceable controls rather than claims of readiness. Security teams should judge AI platforms by whether they can stop, trace and explain an action, not by whether they can describe one.

What this signals

AI platform trust now hinges on runtime enforcement, not declared policy. Identity and governance teams should assume that review-time approval is insufficient once agents can query data and act continuously. The operating question changes from "was this approved?" to "was it still constrained when it executed?"

Accountability becomes the design requirement once AI systems can act on live data. The practical shift is toward inventory, governed context, lineage and continuous evidence as one control set. When those capabilities are missing, the organisation may still have AI, but it does not have governable AI.

For practitioners

• Establish one authoritative AI inventory Record every model, use case and agent at the source, with owner, risk tier and business purpose attached. Treat shadow agents and orphan models as governance defects rather than discovery noise.
• Bind governed context to runtime decisions Attach definitions, relationships, lineage and quality signals to the data path that the model or agent uses at execution time. If the context is only available in design documents, it is not governable.
• Enforce runtime limits on access and action Confirm that the platform can restrict what each AI system can reach and do while it is running, not just at approval time. Test that the control holds against live queries, tool calls and agent actions.
• Require traceable decision evidence Preserve decision traces, data-access events and end-to-end lineage so every consequential action can be explained after the fact. Make audit evidence continuous, not something assembled during incident response.
• Add pausable guardrails for agent drift Define the conditions under which an agent must be stopped or constrained immediately, and make that intervention available inside the platform. A trusted control set must be able to halt action before impact compounds.

Key takeaways

• Trusted AI platforms need more than policy statements, because runtime enforcement is what determines whether access and action stay controlled.
• The article's seven controls link inventory, context, execution, observability and evidence into a single accountability model for models and agents.
• For practitioners, the decision point is not whether AI is modern enough to deploy, but whether governance survives after the system starts acting.

Key terms

• Governed Context: The approved business meaning, labels, and boundaries that shape how an AI system interprets data before acting on it. When context is governed, the organisation is not only controlling access to information, but also controlling what the system can infer, combine, and operationalise from that information.
• Runtime control: Controls that enforce policy while an AI system is operating, rather than after the fact. For healthcare chatbots, runtime control includes data masking, output filtering, access scoping, and immutable logging so the organisation can defend the interaction itself.
• End-to-end Lineage: End-to-end lineage is the trace from source data to model input to agent action. It lets teams reconstruct how a decision was made and what it touched, which is essential for auditability, incident response, and accountability when AI outcomes are disputed.
• Continuous Audit Evidence: Continuous audit evidence is a persistent record of decisions, assessments and control outcomes captured as AI systems run. It reduces the need to reconstruct events under pressure and gives governance teams a durable proof trail for trust, compliance and incident review.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.