Azure OpenAI and Azure AI Foundry Security Guide: Securing Enterprise AI Workloads

Azure OpenAI and Azure AI Foundry deployments are most often exposed through API keys that bypass role-based access control, agents that inherit broad identities, and network and logging settings left at their defaults. Microsoft now documents the platform as Microsoft Foundry: Azure OpenAI models run inside a Foundry resource (resource type Microsoft.CognitiveServices/accounts), and the Azure AI built-in roles were renamed to Foundry roles. This guide covers Entra ID access and the Foundry roles, agent identity, key-based authentication, network isolation, content filtering, data handling, logging and Defender alerts. It is written for cloud security engineers, Azure platform teams and AI engineers running Foundry in production. For the other hyperscalers see the Amazon Bedrock Security Guide and the Google Vertex AI Security Guide.

Key takeaways

• API keys skip RBAC entirely. Microsoft states that key-based authentication grants full access without role restrictions, so set disableLocalAuth to true and enforce it with Azure Policy.
• Owner is not the dangerous data role, Foundry User is. Azure Owner and Contributor carry no data actions, while Foundry User (formerly Azure AI User) can build agents and call models; Owner can still grant itself any role.
• Every agent now has its own Entra identity. New Foundry agents get an agent identity under an agent identity blueprint, with no secrets and no permissions until you grant them, which makes least privilege practical.
• Private endpoints do not cover every path. Some agent tools such as Bing grounding and SharePoint use public endpoints, and an AI Gateway created from the Foundry portal is public until isolated separately.
• Logs and alerts need switching on. Resource logs need a diagnostic setting, content filter metrics need dashboards, and Defender for AI Services threat protection is enabled per subscription.

How identity and access work in Azure AI Foundry

Foundry separates the control plane (creating resources, projects and deployments through Azure Resource Manager) from the data plane (calling models, building and running agents, uploading files). Azure RBAC governs both, but only roles with data actions open the data plane. Callers authenticate in one of two ways:

The resource hierarchy matters for scoping. A Foundry resource holds model deployments, connections and shared settings; projects inside it hold agents, files and evaluations; agents can be addressed as their own scope (.../accounts/<account>/projects/<project>/agents/<agent>). Two managed identities are in play: the project's managed identity, which needs Foundry User on the Foundry resource to use its data and APIs, and the Foundry resource's managed identity, which reaches external resources such as a storage account (Storage Blob Data Reader) or Azure AI Search. See the Azure RBAC and Service Principals Guide.

Admin roles and privileged access

Microsoft renamed four roles without changing their IDs or permissions, so older documentation, Terraform and access reviews may still show the Azure AI names.

• Owner is an escalation path: it has no data actions, but it can assign Foundry User to itself or anyone. Treat subscription Owner as privileged and put it behind Privileged Identity Management. See the JIT Access Guide.
• Avoid legacy roles: Microsoft advises against Azure AI Developer for Foundry projects (it applies to Azure Machine Learning workspaces and hubs) and against built-in roles starting with "Cognitive Services", apart from Cognitive Services Usages Reader for quota visibility. Note that the managed compute documentation still asks for Cognitive Services Contributor and User; record that as a documented exception.
• Fine-tuning needs both planes: either Foundry Owner, or Foundry User combined with Foundry Account Owner. Prefer the split so no single standing account holds both.
• Agent-scope assignment: the portal assigns Foundry Agent Consumer only at account scope; use the CLI with the agent scope path to give a caller access to one agent rather than all of them.

Integrations, API tokens and service accounts

Agent identity

Foundry provisions a Microsoft Entra agent identity for each new agent: a special service principal created under an agent identity blueprint. The blueprint authenticates with a federated identity credential that Foundry manages, so there is no client secret or certificate to store or rotate. When an agent calls a tool, the Agent Service authenticates as the blueprint, obtains a token for the agent identity, exchanges it for a token scoped to the downstream audience (for example https://storage.azure.com), and the target checks its own RBAC. Agents can act application-only under their own permissions or on behalf of the signed-in user (delegated, for example AgentOnBehalfOfUser), which is still in preview.

• Starts with nothing: a new agent identity has no resource permissions. Grant roles to its object ID at resource or resource-group scope, never subscription scope.
• Legacy shared identity: older agents use a shared project-level identity, and azd legacy deployments give it Foundry User automatically. Migrating to per-agent identities changes the identity and role assignments do not transfer, so plan and test the grants.
• Hosted agents: hosted agent code cannot use the project managed identity; it receives tokens for its own agent identity and needs separate permissions for container registry and Application Insights.
• Governance in Entra: agent identities appear in the tenant inventory in the Microsoft Entra admin center alongside Copilot Studio agents, and can be targeted by Conditional Access and Identity Protection, with owners, sponsors and expiry. See the Agentic AI Identity Guide and the Securing Microsoft Copilot Studio Agents Guide.

API keys

Disable local authentication by setting disableLocalAuth: true on the Microsoft.CognitiveServices/accounts resource in Bicep, ARM or Terraform, or with Set-AzCognitiveServicesAccount -DisableLocalAuth $true, and assign the built-in policy "Azure AI Services resources should have key access disabled (disable local authentication)" at subscription or management group scope. Microsoft warns the change is not instant: the shared gateway can keep accepting previously valid keys for minutes to several hours while its cache refreshes, so do not rely on it as an emergency revocation. Confirm success when an old key returns HTTP 401 with "Access denied due to invalid subscription key or wrong API endpoint". Where keys must remain, store them in Key Vault, rotate them on a schedule, and restrict the network path so a stolen key does not work from the internet. See the API Key Management Guide.

The cost of not doing this is documented. In the Storm-2139 case Microsoft said the group used Azure OpenAI API keys exposed by customers and collected from public sources, altered the services to get round guardrails and resold the access. That is T1552.001 Credentials In Files followed by T1496.004 Cloud Service Hijacking. See the LLMjacking Guide.

Connections and tools

Connections to Azure AI Search, storage, Cosmos DB and external APIs should use Entra ID with the resource or agent identity rather than stored keys. Remember that authentication does not change a third-party tool's data handling: calls to non-Microsoft tools can send data outside Microsoft services. Review MCP and OpenAPI tools as integrations with an owner. See the MCP Security Guide.

Logging and detection

The activity log records control-plane operations automatically. Data-plane resource logs are not collected until you create a diagnostic setting on the Foundry resource and route the categories to Log Analytics, where they land in the AzureDiagnostics table and metrics in AzureMetrics.

Defender's threat protection for AI services covers Azure OpenAI and Azure AI Model Inference models, scans text tokens only, and is available in commercial Azure only. In applications, check finish_reason on every response: a filtered completion returns content_filter, a filtered prompt returns HTTP 400 with code content_filter, and if the filtering system is unavailable the request succeeds unfiltered with a content_filter_error in content_filter_results.

Metrics that show the controls work:

• Percentage of Foundry resources with disableLocalAuth true and compliant with the Azure Policy assignment.
• Percentage of Foundry resources with public network access disabled or limited to selected networks.
• Number of standing Foundry Owner, Foundry Account Owner and subscription Owner assignments, trended monthly.
• Percentage of agents on per-agent identities with resource-scoped grants rather than the shared project identity.
• Diagnostic setting coverage across Foundry resources (target 100% in production).
• Mean time from a token-volume or RAIRejectedRequests spike to investigation.

Hardening baseline

Network isolation

• Inbound: create a private endpoint for the Foundry resource (target sub-resource "account"), set public network access to Disabled or Selected networks, and confirm with nslookup that clients resolve the privatelink name to a private IP. Custom DNS must forward the privatelink zone to Azure DNS (168.63.129.16). Removing a private endpoint does not change public access, which is a separate setting.
• Outbound for agents: use Agent Service virtual network injection with a subnet delegated to Microsoft.App/environments (/27 or larger) and the standard setup, which brings your own Storage, AI Search and Cosmos DB so agent data stays in your tenant. Create private endpoints to those services yourself, since they are not created automatically. Outbound networking cannot be changed after deployment, so decide before you build.
• Know the gaps: Code Interpreter and function calling use the Microsoft backbone, Bing, web search and SharePoint tools use public endpoints, several tools (for example Browser Automation and Computer Use) are not supported with network isolation, and an AI Gateway created from the Foundry portal is public until you isolate it in API Management.

Content filtering and guardrails

• Default filters: all models except Whisper ship with default filters for hate, sexual, violence and self-harm content at four severity levels. Custom filters are configured under Guardrails + controls and attached per deployment.
• Prompt Shields: enable detection of user prompt attacks and indirect (document) attacks for any deployment that reads retrieved content, emails or tool output. Indirect attack detection depends on documents being embedded and formatted as the documentation describes. See the Prompt Injection Guide.
• Approval gate: turning completion filters off or to annotate-only requires Microsoft approval through the Modified Content Filters form. Track which resources hold that approval and why.
• Other detectors: protected material for text and code, groundedness (streaming only, limited regions), PII on outputs and Task Adherence for agents. Use blocklists for organisation-specific terms.

Data handling and encryption

• Training and abuse monitoring: Microsoft states prompts, completions and training data are not used to train foundation models without permission. Content flagged by abuse monitoring may be stored for human review in the resource's geography. Customers can apply for modified abuse monitoring; when approved, the resource shows the capability ContentLogging set to false.
• Stored data: files, vector stores, Responses API and thread history, stored completions, fine-tuned models and batch data are stored in your resource's geography, encrypted with AES-256 by default and deletable. Set retention and deletion jobs for them like any other data store.
• Deployment type: Standard (regional) processes within your geography, DataZone anywhere in a Microsoft-defined zone such as the EU, and Global in any geography where the model runs. Choose by residency requirement, not only by quota.
• Customer-managed keys: the key vault must be in the same region with soft delete and purge protection, the managed identity needs Key Vault Crypto User, keys are RSA 2048 or larger, and you cannot switch a project back to Microsoft-managed keys. A revoked key makes the data inaccessible.

Rollout order

1. Inventory: list Foundry and Azure OpenAI resources, deployments, projects, agents, keys in use and role assignments, including legacy Azure AI and Cognitive Services roles.
2. Turn on visibility: diagnostic settings, metric alerts, Defender for AI Services and budget alerts first, so you can see what breaks later.
3. Move callers to Entra ID: find key-based callers from logs, move them to managed identities, then assign the disable-local-auth policy in audit mode, then deny. Roll back by setting disableLocalAuth to false, remembering the same propagation delay.
4. Reduce roles: replace Foundry Owner with the user and account owner split, scope Foundry User to projects and use Foundry Agent Consumer for callers.
5. Isolate the network: pilot private endpoints on one non-production resource, check DNS and tool behaviour, then disable public access in production.
6. Tune guardrails: run Prompt Shields and custom filters in annotate mode where allowed, review hits, then block.

Small and large organisations: a small team can reach a good baseline with one Foundry resource, keys disabled, per-agent identities and Defender enabled. Large organisations should enforce the same settings with Azure Policy at management group level, separate resources per environment, and run agent identity grants through change control. See the Azure Policy and Management Groups Guide.

Common misconfigurations

• Keys left enabled "for one legacy client": every key holder has full data-plane access, so a single key in a public repository is enough, as Storm-2139 showed.
• Granting Foundry Owner to developers to make things work: it combines full build rights with the ability to grant roles, when Foundry User on the project would do.
• Assuming private endpoints cover agents: without virtual network injection and private endpoints to Storage, AI Search and Cosmos DB, agent data and tool traffic still use public paths.
• Skipping Prompt Shields for indirect attacks: agents that read email, documents or web content are exposed to injected instructions, the pattern behind EchoLeak in Microsoft 365 Copilot (OWASP LLM01:2025 Prompt Injection).
• Not migrating grants with agent identities: moving an agent from the shared project identity to its own identity silently drops its role assignments, and teams then over-grant to restore function.
• Ignoring content_filter_error: when filtering is unavailable, responses arrive unfiltered and applications that do not check for the error pass them straight to users.

Framework mapping

Practitioner checklist

• Set disableLocalAuth to true on every Foundry and Azure OpenAI resource and assign the matching Azure Policy, audit first, then deny.
• Replace legacy Azure AI and Cognitive Services role assignments with Foundry roles at the narrowest scope.
• Put subscription Owner, Foundry Owner and Foundry Account Owner behind PIM with approval.
• Give callers Foundry Agent Consumer at agent scope instead of Foundry User.
• Move agents to per-agent identities, grant roles to their object IDs at resource scope, and re-grant after migration.
• Disable public network access, add private endpoints, and use VNet injection with the standard agent setup.
• Create diagnostic settings for all log categories, alert on token and RAIRejectedRequests spikes, and enable Defender for AI Services.
• Enable Prompt Shields for user and indirect attacks on every agent deployment, and handle content_filter_error in code.
• Choose Standard, DataZone or Global deployments by residency requirement and document any modified abuse monitoring approval.
• Apply customer-managed keys where policy requires, with purge protection and monitored key vault logs.

Standards and references

• Microsoft Learn: Role-based access control for Microsoft Foundry
• Microsoft Learn: Agent identity concepts in Microsoft Foundry
• Microsoft Learn: Disable local authentication in Foundry Tools
• Microsoft Learn: Configure network isolation for Microsoft Foundry
• Microsoft Learn: Content filtering
• Microsoft Learn: Data, privacy and security for models sold by Azure
• Microsoft Learn: Azure OpenAI monitoring data reference
• OWASP: Top 10 for LLM Applications 2025

Related NHI Mgmt Group resources: Microsoft Azure Security Guide · Enterprise AI Copilot Security Guide · AI Infrastructure Workload Identity Guide · Entra ID Security Guide