AI Platforms Guide: Securing Enterprise AI Platforms

Enterprise AI platforms hand a model the reach of a user, an application or an agent, so every weakness in permissions, credentials and logging becomes something a prompt can find or trigger. Most organisations now run several platforms at once: a productivity copilot grounded on mail and documents, a chat assistant with connectors, a cloud model service behind internal applications, a coding assistant in the IDE and a model hub for open weights. Each has its own admin console, identity model and logging, and each changes monthly. This page is the overview of the whole AI Platforms category: what the field covers, the concepts that apply to every platform, the main threats, controls by maturity, who owns what, and the frameworks that apply. It then links every NHI Mgmt Group platform guide for the product-level detail. It is written for security architects, identity teams, cloud and data protection practitioners and the owners of AI platforms.

Key takeaways

Platforms inherit access, they rarely create it. Copilots ground answers on what the user can already open, so oversharing and stale group membership are the first thing to fix, as the Microsoft 365 Copilot and Gemini guides both show.
Every platform runs on non-human identities. API keys, service roles, OAuth apps, connector tokens and agent identities do the real work, and stolen keys are the root of LLMjacking.
Logging is often off or partial by default. Amazon Bedrock model invocation logging must be switched on, and audit coverage differs by product tier, so verify what is recorded before go-live.
Guardrails are a layer, not a boundary. Content filters and prompt classifiers were bypassed in EchoLeak and the Gemini calendar case, so least privilege on data and tools has to carry the weight.
Govern platforms as a portfolio. One inventory, one approval path, one logging standard and one set of data rules across SaaS assistants, cloud model services and self-hosted models beats a separate policy per product.

What the field covers

The AI Platforms category covers the commercial and open platforms through which an organisation consumes or builds AI, and the security settings each one exposes. It is about the platform layer, not about the model's internal behaviour (that is the AI Threats and Model Security category) or about autonomous agents as a class (the agent categories).

Platform family	Typical examples	Who uses it	Main security question
Productivity copilots	Microsoft 365 Copilot, Gemini for Google Workspace	All staff	What can the assistant ground on, and is that content correctly permissioned and labelled?
Enterprise chat assistants	ChatGPT Enterprise, Claude for Enterprise	Knowledge workers	Which connectors, data retention settings and workspace controls apply, and who administers them?
Cloud model services	Amazon Bedrock, Azure OpenAI and Azure AI Foundry, Google Vertex AI	Developers and application teams	Which roles can invoke which models, are guardrails enforced, and is invocation logged?
Coding assistants	GitHub Copilot	Engineers	What code and secrets reach the model, and how are suggestions and agent actions reviewed?
Model hubs and open weights	Hugging Face, self-hosted open-weight models	ML engineers	Are downloaded artefacts safe to load, and are access tokens and private repositories protected?
Automation and low-code AI	Workflow tools with AI steps	Makers and business teams	Which shared integration identities do AI steps use, and who approved them?

Core concepts

Shared responsibility: the provider secures the model hosting and service plane; the customer owns identities, data exposure, configuration, connectors and logging. Contract terms on training use and retention vary by tier and must be read for each product. See the Shared Responsibility Models Guide.
Grounding and retrieval: copilots and RAG applications retrieve content at question time and pass it to the model. The retrieval layer decides what can leak, which is why permission-aware retrieval matters. See the Permission-Aware RAG Guide.
Delegated versus application access: a connector that acts with the user's delegated OAuth token can only reach what that user can; a connector using an application permission or shared service account reaches everything it was granted, for every user.
Model access entitlement: in cloud model services, the right to call a model is an IAM permission (for example bedrock:InvokeModel on a foundation-model ARN). Allowlisting approved models by explicit deny is more reliable than relying on any enablement step.
Guardrails and content safety: provider filters for harmful content, prompt attack detection and sensitive data redaction. In Bedrock they apply only when a request names a guardrail unless enforcement is configured; the bedrock:GuardrailIdentifier condition key makes them mandatory.
Agents and actions: platforms increasingly let assistants call tools, send mail and change records. Once an assistant can write, prompt injection becomes a data movement and integrity risk, not only a disclosure risk.
Gateways: an AI gateway or LLM gateway sits between applications and model providers to centralise keys, quotas, logging and policy. See the AI Gateway and LLM Gateway Guide.

Main threats

The OWASP Top 10 for LLM Applications 2025 is the common vocabulary for application-level risk, and MITRE ATLAS gives technique IDs for detection and threat modelling. The table maps the threats that recur across platforms.

Threat	How it shows up on platforms	OWASP LLM 2025	MITRE ATLAS / ATT&CK
Oversharing surfaced by the assistant	Copilot answers from "everyone" shared sites and stale memberships	LLM02 Sensitive Information Disclosure	AML.T0057 LLM Data Leakage
Indirect prompt injection	Crafted email, invite or document steers the assistant (EchoLeak, Gemini calendar)	LLM01 Prompt Injection	AML.T0051.001 Indirect
Stolen API keys and cloud credentials	Attackers invoke models on the victim's bill (LLMjacking)	LLM10 Unbounded Consumption	AML.T0055 Unsecured Credentials, AML.T0034 Cost Harvesting, T1078.004 Cloud Accounts
Over-privileged connectors and agents	Application-permission connector or agent service role reaches far more than the use case	LLM06 Excessive Agency	AML.T0053 AI Agent Tool Invocation
Supply chain compromise	Malicious model files, poisoned SDKs or gateway packages (LiteLLM)	LLM03 Supply Chain	AML.T0010 AI Supply Chain Compromise, T1195.002
System prompt and configuration leakage	Instructions, keys or internal URLs in system prompts are extracted	LLM07 System Prompt Leakage	AML.T0056 Extract LLM System Prompt
Unsafe output handling	Rendered Markdown, links or generated code fetch attacker URLs or execute	LLM05 Improper Output Handling	AML.T0025 Exfiltration via Cyber Means
Secrets in code and prompts	Keys pasted into chats or committed code suggested back	LLM02	T1552.001 Credentials In Files

Three incidents illustrate the pattern. In EchoLeak (CVE-2025-32711) one crafted email could make Microsoft 365 Copilot place internal data in a link that the browser fetched; the classifier, link redaction and content security policy were each bypassed. In the Gemini calendar prompt injection a calendar invite made the assistant write a summary of private meetings into a new event. In LLMjacking, stolen cloud credentials and API keys were used to call Bedrock, Azure OpenAI and other services, with Sysdig estimating potential costs above $46,000 per day. For the threat side in depth, see the AI Threats and Model Security Guide and the Prompt Injection Guide.

Choosing a platform model

Most organisations end up with several of these. The decision is which workloads go where, not which single platform wins.

Option	Choose it when	Security trade-off
Productivity copilot in your existing suite	Value comes from mail, documents and meetings already in that suite	Inherits every permission problem in the tenant; data stays inside the suite's compliance boundary and admin tooling
Enterprise chat assistant	Staff need a general assistant with selected connectors	Another admin plane and SSO integration; connector scope and retention settings must be governed separately
Cloud model service in your cloud account	You build applications or agents and want IAM, network and key control	Most control, most responsibility: you own roles, guardrail enforcement, logging and quotas
Self-hosted or open-weight model	Data residency, cost or customisation requires it	Full stack is yours, including endpoint authentication, patching and model file safety. See the Self-Hosted LLM Security Guide.

Small organisations: standardise on one suite copilot and one chat assistant, turn on SSO and audit logging, restrict connectors to an approved list and block personal accounts at the proxy. Large organisations: add a model gateway for all application traffic, enforce model allowlists by organisation policy (for example AWS SCPs or Azure Policy), separate builder and logging-admin roles, and run platform onboarding through a formal AI governance review. See the AI Governance Guide.

Key controls by maturity

Area	Foundational	Managed	Advanced
Inventory	List of approved platforms, owners and licence counts	Inventory of connectors, agents, API keys and service roles per platform	Continuous discovery of shadow AI and unapproved model calls; see the Shadow AI Discovery Guide
Identity	SSO and MFA for every console; no shared admin accounts	Least-privilege roles, separate builder, operator and logging-admin duties	Short-lived credentials and workload identity federation in place of static keys
Data exposure	Oversharing clean-up before rollout; sensitive sites excluded	Sensitivity labels and DLP applied to AI interactions	Permission-aware retrieval for every custom RAG application
Model access	Only approved models callable	Explicit deny on unapproved model identifiers at organisation level	Gateway-enforced routing, quotas and per-application keys
Guardrails	Provider content filters on	Guardrails mandatory by policy condition, prompt attack detection enabled	Custom evaluations and red teaming before each major release
Actions and agents	Write actions disabled or limited to pilots	Human confirmation for sending, sharing and changing records	Per-agent identities with scoped tokens and kill switches
Logging	Admin audit logs retained and sent to SIEM	Prompt and response or invocation logging where lawful	Detections for injection signals, spend anomalies and logging changes

Rollout approach

Inventory and classify: list every platform in use, including trials and personal accounts seen in proxy logs, and assign an owner and data classification to each.
Prepare data: fix oversharing, remove "everyone" grants on sensitive sites, apply labels, and decide which repositories are excluded from grounding.
Harden identity: enforce SSO, MFA and conditional access on consoles; replace long-lived API keys with roles where the platform allows; store remaining keys in a secrets manager. See the LLMjacking Guide.
Turn on logging first: enable audit and invocation logging and confirm events arrive in the SIEM before any user is licensed.
Pilot ring: license a small group (often IT, security and a willing business unit) with connectors and write actions limited; review what the assistant surfaced and tune exclusions.
Expand in rings: widen access department by department, enabling connectors and agent features only after their owner and scope are approved.
Roll back cleanly: keep a documented path to remove licences, disable connectors and agent features, revoke API keys and service-role trust, and restrict sites from search, so a problem can be contained in hours rather than weeks.

Roles and ownership

Role	Owns
AI platform owner	Configuration, licensing, feature enablement, change log and roll-back plan for one platform
Identity and access team	SSO, admin roles, OAuth app consent, service accounts, API key lifecycle and access reviews
Data protection and records	Labels, DLP, retention, data residency and the lawful basis for prompt logging
Cloud security	IAM policies, organisation guardrails, network paths and key management for cloud model services
Application and ML engineering	RAG pipelines, system prompts, output handling, model selection and dependency hygiene
SOC and detection engineering	Log onboarding, detections and AI incident playbooks; see the AI Agent Incident Response Playbook
AI governance or risk committee	Approval of new platforms and high-risk use cases, policy and exceptions

Where identity fits

Every AI platform is an identity system in disguise. Copilots act as delegated identities for the user; connectors and agents act through OAuth apps, service principals and service roles; applications call models with API keys. The Bedrock service roles for agents and knowledge bases, Azure managed identities, Vertex AI service accounts, Hugging Face access tokens and GitHub Copilot's organisation policies are all machine identities that need an owner, least privilege, rotation and review. The LiteLLM PyPI breach shows the consequence: a poisoned CI scanner exposed a publishing token, and the malicious package harvested LLM provider API keys alongside cloud credentials. See the Agent Identity and Access Guide.

Detection and measurement

Log sources: suite audit logs for copilot interactions (Microsoft 365 records prompt injection and jailbreak detections as audit properties), cloud control plane logs (CloudTrail, Azure Activity Log, Cloud Audit Logs), model invocation logs, gateway logs and OAuth consent events.
High-value alerts: changes to logging configuration, first use of a model not on the allowlist, new API keys or long-lived keys, spend spikes, new connectors or agents published, and AI-generated links pointing outside the organisation.

Metric	What good looks like
Platforms with named owner and logging verified	100% of approved platforms
Long-lived AI API keys in use	Falling each quarter; none older than the rotation policy
Sensitive sites or repositories exposed to organisation-wide grounding	Zero after remediation, tracked weekly during rollout
Connectors and agents with approved scope and owner	100%, reviewed at least quarterly
Model calls through the gateway or approved endpoints	Above 95% of application traffic
Mean time to revoke a leaked AI key	Hours, not days

Common failure patterns

Licensing before permissions: rolling a copilot out to everyone, then discovering it answers from years of overshared content.
Static keys in code and CI: provider keys in repositories, environment variables and CI secrets readable by every job, the route used in LLMjacking and the LiteLLM compromise.
Advisory guardrails: filters configured but not enforced, so any application can call the model without them.
Logging assumed, not verified: invocation logging off, or audit events on a tier the organisation did not buy, discovered only during an incident.
Connectors with application permissions: a shared integration identity that lets every user's prompt reach every mailbox or record.
External content treated as trusted: inbound email, invites and shared documents grounded and acted on without confirmation, as in EchoLeak and the Gemini calendar case.

Standards and frameworks

Framework	Where AI platforms fit
NIST AI RMF 1.0 (January 2023) and NIST AI 600-1 Generative AI Profile (July 2024)	Govern, Map, Measure and Manage functions for AI risk; the profile adds generative AI risks such as information security and data privacy. See the NIST AI RMF Guide and NIST AI 600-1 Generative AI Profile Guide.
NIST CSF 2.0	ID.AM-02 inventories of software and services; PR.AA-05 access permissions with least privilege; DE.CM-09 monitoring of software and runtime environments; GV.SC-07 supplier risk
NIST SP 800-53 Rev. 5	AC-6 Least Privilege, AU-2 Event Logging, CM-8 System Component Inventory, SI-4 System Monitoring, SR-3 Supply Chain Controls and Processes, RA-3 Risk Assessment
ISO/IEC 42001:2023	AI management system requirements for organisations that provide or use AI; platform onboarding and supplier controls sit within it
NCSC and partners: Guidelines for secure AI system development (November 2023)	Secure design, development, deployment, and operation and maintenance, including logging and update management
OWASP Top 10 for LLM Applications 2025	Application risk vocabulary for anything built on the platforms; see the OWASP Top 10 for LLM Applications Guide
MITRE ATLAS	Technique IDs such as AML.T0051 LLM Prompt Injection and AML.T0034 Cost Harvesting for threat models and detections
ENISA Multilayer Framework for Good Cybersecurity Practices for AI (June 2023)	Cybersecurity foundations, AI-specific and sector-specific layers

Regulatory obligations such as the EU AI Act sit alongside these frameworks; see the EU AI Act Compliance Guide.

Where to go next (all guides in this category)

Vendor-neutral foundations

Enterprise AI Copilot Security Guide: the risk model for any assistant that searches company data and acts on it, with rollout and monitoring steps.

Productivity copilots

Microsoft 365 Copilot Security Deep Dive: oversharing, Restricted Content Discovery, Purview labels and DLP, agents and audit events.
Gemini for Google Workspace Security Guide: admin controls, data protection and how Gemini uses Workspace permissions.

Enterprise chat assistants

ChatGPT Enterprise Security Guide: workspace admin controls, data handling and connectors.
Claude for Enterprise Security Guide: SSO and provisioning, admin settings, data handling and integrations.

Cloud model services

Amazon Bedrock Security Guide: IAM for models, agents and knowledge bases, guardrail enforcement, API keys and invocation logging.
Azure OpenAI and Azure AI Foundry Security Guide: Entra ID roles, managed identities, API keys, network isolation and logging.
Google Vertex AI Security Guide: service accounts, VPC Service Controls, agents and data protection on Google Cloud.

Developer and model platforms

GitHub Copilot Security Guide: organisation policies, content exclusion, coding agent controls and audit.
Hugging Face Security Guide: model and dataset safety, Spaces, organisation settings and access tokens.

Related pillars: the AI Threats and Model Security Guide for attacks on models and data, the LLM Security Guide for application security, and the Agent Controls and Monitoring Guide for agents built on these platforms.

Practitioner checklist

Keep one inventory of every AI platform, connector, agent, API key and service role, each with a named owner.
Remediate oversharing and apply sensitivity labels before licensing a copilot beyond a pilot ring.
Enforce SSO and MFA on every AI admin console and separate builder, operator and logging-admin roles.
Replace long-lived AI API keys with roles or short-lived credentials; vault and rotate any that remain.
Allowlist approved models by explicit deny at organisation level and make guardrails mandatory by policy.
Turn on audit and invocation logging, confirm it reaches the SIEM, and alert on any change to it.
Require human confirmation for assistant actions that send, share or change data, especially with external content in context.
Restrict connectors to delegated access where possible and review application-permission integrations quarterly.
Set budgets and spend alerts on every model service to catch key abuse early.
Document a tested roll-back for each platform: licences, connectors, agent features, keys and trust policies.

AI Platforms Guide: Securing Enterprise AI Platforms, Models and Integrations