DASF shows why AI security must move from theory to operations

At a glance

What this is: This is an operational guide to the Databricks AI Security Framework that argues AI security must be managed across the full AI lifecycle, from design and training to monitoring and retirement.

Why it matters: It matters because AI systems now create governance gaps around data exposure, prompt abuse, and agent behaviour that existing application and infrastructure controls do not fully cover.

By the numbers:

• 27 days
• Only 44% of developers are reported to follow security best practices for secrets management, exposing a significant developer behaviour gap.

👉 Read Noma Security's guide to operationalising DASF for AI security

Context

AI security fails when teams treat models as static software and ignore the fact that training data, retrieval layers, prompts, tools, and agent behaviour can all become attack surfaces. In this article, the primary problem is not infrastructure compromise but the governance gap created when AI systems can be manipulated into revealing sensitive information or behaving outside intended bounds. That makes AI security a lifecycle issue, not just a deployment issue.

For identity and access teams, the intersection is real where AI systems consume internal knowledge, call tools, or act as agents. Those systems rely on credentials, permissions, and trust relationships that look increasingly like non-human identity problems, especially once RAG pipelines, APIs, and agent tools are introduced into production workflows.

Key questions

Q: How should security teams govern AI systems that use retrieval and internal knowledge bases?

A: They should treat retrieved content as a governed access surface, not just a data source. That means classifying what can be indexed, limiting which users and models can reach sensitive corpora, filtering outputs for disclosure risk, and logging retrieval paths for review. If the model can answer from internal documents, the retrieval layer needs the same discipline as any other access-controlled system.

Q: Why do sanctioned AI assistants create data exposure risk in collaboration platforms?

A: Sanctioned AI assistants inherit the permissions of the repositories they query, so any over-shared file or loosely governed workspace can become visible through the assistant interface. When access groups are too broad, AI can surface material that users were never meant to find through normal navigation. The control issue is least privilege, not model quality.

Q: What breaks when AI security is handled only at launch time?

A: Controls go stale as the system changes. New data sources, fine-tuning, prompt updates, and tool integrations can all expand the attack surface after go-live, which means the original approval no longer reflects the current risk. Continuous review is required because AI systems are operationally dynamic, not static software artifacts.

Q: Which frameworks should teams use to evaluate AI security controls and accountability?

A: Use NIST AI RMF for governance, OWASP guidance for common AI attack patterns, MITRE ATLAS for adversarial techniques, and ISO 27001 where enterprise control mapping is needed. The framework should help teams prioritise, test, and evidence controls, not replace validation against the actual model, data, and agent workflows.

Technical breakdown

Why RAG pipelines create a new data exposure path

Retrieval-augmented generation, or RAG, lets a model answer using external documents at inference time. That improves usefulness, but it also means indexed content becomes queryable through the model’s output layer. If confidential documents, policy files, or internal notes are retrievable, the model can expose them when prompts steer it toward disclosure. This is not a database breach in the traditional sense. It is a control failure where access, retrieval scope, and response generation are not sufficiently separated. In practice, the security boundary moves from storage to the retrieval and prompt layer.

Practical implication: classify and constrain retrievable corpora before production, not after the model is live.

How prompt manipulation turns harmless inputs into data leaks

Prompt injection works when an attacker crafts inputs that override or redirect the model’s intended behaviour. Because LLMs infer patterns rather than execute fixed rules, a malicious prompt can coax the system into ignoring safety constraints, surfacing hidden instructions, or disclosing content that should remain internal. The challenge is amplified when the model can access tools, memory, or confidential context. Security teams should think of this as runtime policy erosion, not just content abuse. The model is doing what it was designed to do, but the trust boundary around what it should reveal has been too broad.

Practical implication: separate user-facing prompts from privileged context and apply response filtering at the output boundary.

Why AI lifecycle controls have to cover agents, not just models

The article’s DASF framing is important because AI risk changes as systems move from training into deployment and then into agentic operation. Once an AI system can invoke tools, retrieve data, or chain actions, the security problem expands from model safety to delegated action control. That introduces governance questions similar to NHI management: who granted the capability, what is the scope, how is it monitored, and when is access revoked? If the organisation only secures the model and ignores the agent layer, it leaves the most operationally dangerous behaviour outside policy enforcement.

Practical implication: govern AI agents with explicit lifecycle ownership, scoped permissions, and continuous review of tool access.

Threat narrative

Attacker objective: The attacker wants to extract confidential internal knowledge from the AI system without breaching traditional network defences.

1. Entry occurs through crafted prompts aimed at a RAG-enabled AI assistant that has access to internal indexed documents and policy content.
2. Escalation happens when the model is steered into surfacing sensitive context that was never meant for general user disclosure.
3. Impact is the exposure of confidential business knowledge, including loan underwriting criteria and other protected internal material.

NHI Mgmt Group analysis

AI governance debt is now a material security issue: organisations that deploy AI without lifecycle controls accumulate hidden exposure across training data, retrieval layers, tool access, and output behaviour. The article’s core lesson is that AI systems cannot be secured with launch-time review alone because the attack surface changes after deployment. Practitioners should treat AI governance as an operating discipline, not a project deliverable.

RAG exposure is a retrieval governance problem, not just a model problem: confidential documents become attackable when they are indexed into a system that can answer open-ended questions. That means classification, indexing scope, and response filtering all matter together. For teams managing human identity and NHI programmes, the lesson is to control what the AI can retrieve with the same discipline used for access entitlements.

AI agents need NHI-style lifecycle controls: once a system can call tools or act on internal data, it behaves like a privileged non-human actor. That creates a governance pattern familiar to identity teams: scoped permissions, ownership, revocation, and continuous review. The practical conclusion is that agentic AI cannot be left outside identity governance just because it is model-driven.

Standards mapping matters only when it changes operation: the value of DASF comes from turning abstract AI risk into controls that can be mapped to NIST AI RMF, OWASP, MITRE ATLAS, and ISO 27001. Mapping alone is not assurance, but it gives security, audit, and risk teams a shared vocabulary for enforcement. Practitioners should use frameworks to prioritise controls, not to replace operational validation.

Named concept: AI attack surface drift: the article describes a control reality in which the AI system that was safe at launch is not necessarily safe after new data, integrations, or agent functions are added. That drift is the governance failure. Security teams should assume every AI update can widen the attack surface and require re-approval.

What this signals

AI attack surface drift: once an AI system starts changing through new data, prompt updates, or tool integrations, the security programme has to move from one-time approval to continuous governance. That is especially relevant for teams already managing privileged access and non-human identities, because agentic behaviour introduces a familiar ownership and revocation problem in a new form.

Security leaders should expect AI review boards to converge with identity governance, data protection, and model risk functions. The practical shift is from asking whether the model is accurate to asking whether its retrieval scope, action scope, and disclosure controls are still aligned to current business use.

Where AI systems can retrieve internal content or invoke tools, the boundary between model security and identity governance disappears. Teams should prepare for policy, audit, and access review processes that cover AI agents, their tool permissions, and the sensitive corpora they can reach.

For practitioners

• Define AI deployment boundaries before launch Inventory models, training sources, RAG stores, APIs, and tool integrations so the security scope matches the real system rather than the intended design. Use this inventory to decide which data, actions, and integrations sit inside policy enforcement.
• Classify and constrain retrievable content Remove confidential material from general retrieval paths unless there is a documented business need and explicit access control. Treat indexed knowledge as a governed resource, because anything retrievable by the model can become promptable by an attacker.
• Embed policy checks in AI workflows Place review, validation, and approval gates inside the deployment and orchestration workflow instead of relying on manual sign-off. That reduces the chance that teams bypass controls when retraining, fine-tuning, or changing prompts under time pressure.
• Monitor prompt and output behaviour continuously Log inference activity, unusual prompt patterns, retrieval anomalies, and unexpected disclosures so drift is visible before it becomes an incident. Pair detection with response playbooks for containment, rollback, and model or corpus changes.
• Assign ownership for agentic access For AI systems that invoke tools or call internal services, define who owns the permissions, how they are reviewed, and when they are revoked. This is the same discipline identity teams use for non-human identities with operational privilege.

Key takeaways

• AI security fails when organisations treat models as static software instead of mutable systems with changing data, prompts, and tool access.
• The real risk is not just model output quality but uncontrolled retrieval and disclosure from internal knowledge sources.
• Practitioners need continuous governance, lifecycle ownership, and identity-style controls for AI agents and their permissions.

Key terms

• Retrieval-augmented Generation: Retrieval-augmented generation is a pattern where an AI model pulls external information before generating output. The security challenge is that access rules can weaken when data is chunked, embedded, cached, or reused, so source permissions may not automatically follow the content into the model's context.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.
• AI attack surface drift: The expansion or change in an AI system’s risk profile after launch because of new data, prompts, integrations, tools, or model updates. It is the reason AI security must be monitored continuously rather than approved once and forgotten.
• Agentic AI: Autonomous AI systems capable of planning, deciding, and taking actions — including calling APIs, writing code, and orchestrating other agents — with minimal human oversight. Agentic AI introduces new NHI risks as agents must authenticate to external services.

What's in the full article

Noma Security's full blog covers the operational detail this post intentionally leaves for the source:

• A step-by-step walkthrough of the ten DASF implementation phases for AI security teams that need to operationalise controls.
• Examples of control selection and workflow integration across model serving, orchestration, and monitoring environments.
• Specific anti-patterns and cautionary examples for AI deployment, drift, prompt abuse, and continuous validation.
• How the source maps DASF controls to frameworks such as NIST AI RMF, MITRE ATLAS, OWASP, ISO 27001, and the EU AI Act.

👉 The full Noma Security post covers the DASF implementation steps, control mappings, and AI workflow examples.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, secrets management, and workload identity alongside the governance patterns that also matter for agentic AI. It gives practitioners a structured way to connect identity controls to broader security programmes that now include AI systems.