Why do foundation models increase security and governance risk in enterprise AI systems?

Why This Matters for Security Teams

Foundation models change the enterprise risk profile because they are rarely deployed as a single, isolated application. Instead, they become shared decision engines, content generators, or orchestration layers behind many business workflows. That means prompt exposure, unsafe output handling, weak governance, or model compromise can affect several systems at once, not just one team or one use case. Security leaders should treat them as high-blast-radius services that need explicit control ownership, logging, and review. The NIST Cybersecurity Framework 2.0 is useful here because it reinforces governance, risk management, and continuous monitoring rather than one-time approval.

The practical problem is that many enterprises adopt a foundation model first and only later discover how many applications now depend on it for access decisions, customer interactions, summarisation, or code generation. At that point, a single model update, policy change, or data leakage event can cascade across multiple environments. In practice, many security teams encounter the blast radius only after a shared model has already been embedded into production workflows rather than through intentional architectural review.

How It Works in Practice

Foundation models increase governance risk because their behaviour is shaped by pretraining, fine-tuning, retrieval content, system prompts, tool permissions, and user inputs. Each layer can introduce a different failure mode. A model may be technically accurate in isolation but still unsafe when connected to internal documents, external tools, or privileged workflows. That is why model security cannot be reduced to content filtering alone. Current guidance suggests treating the full AI stack as a control boundary, with separate attention to data provenance, prompt handling, access control, and output validation.

For enterprise deployment, security teams typically need to map the model lifecycle to operational controls:

• Define approved model use cases and prohibit unsupported ones before teams start integrating the model into business processes.
• Classify the data the model can see, store, infer, or return, including sensitive prompts and retrieval sources.
• Restrict tool access so the model can only call approved systems with tightly scoped permissions.
• Log prompts, outputs, policy decisions, and tool actions for investigation and governance review.
• Validate outputs before they reach users or downstream automation, especially for decisions that affect customers or regulated processes.

The most reliable pattern is to combine AI governance with standard security engineering: least privilege, change control, monitoring, incident response, and supplier review. The NIST AI 600-1 Generative AI Profile is especially relevant because it translates AI risk into operational activities such as mapping, measuring, and managing model behaviour. These controls tend to break down when foundation models are connected directly to broad SaaS permissions and real-time action tools because the model can execute high-impact actions faster than human review can intervene.

Common Variations and Edge Cases

Tighter model governance often increases deployment friction, requiring organisations to balance speed of adoption against review, access scoping, and validation overhead. That tradeoff becomes sharper when different teams want the same foundation model for very different purposes, such as customer support, coding assistance, and internal analytics. There is no universal standard for this yet, so best practice is evolving around shared controls, use-case-specific guardrails, and risk-tiered approval.

One common edge case is retrieval-augmented generation, where the model itself is stable but the connected knowledge base introduces leakage, poisoning, or stale content risk. Another is agentic AI, where the foundation model not only answers questions but also takes actions through tools and APIs. In those environments, model governance overlaps with non-human identity governance because the agent may need credentials, scoped entitlements, and revocation controls. The risk is not just a misleading answer, but an automated action taken with legitimate access. For that reason, security teams should separate model trust from action trust and review them independently.

Multimodal systems add another layer of uncertainty because images, audio, and documents can trigger behaviours that text-only testing may miss. Enterprises with strict regulatory obligations should also align AI policy to incident handling, supplier assurance, and evidence retention expectations, rather than assuming model vendor assurances are sufficient.

Related resources from NHI Mgmt Group

• Why do agentic AI systems increase risk for API security and governance?
• Why do AI helpdesks and security tools increase identity governance risk?
• Why do AI-enabled marketing systems increase privacy and security risk at the same time?
• Why do AI systems increase identity risk even when they improve security operations?