How should security teams discover shadow LLMs across the enterprise before they start building controls around them?

Start by inventorying where LLM use already exists, including cloud services, developer tools, and API integrations that touch prompts or retrieval data. Then trace application and resource usage to expose hidden dependencies. Discovery matters because you cannot protect what you cannot see. A complete inventory gives security teams a baseline for data scanning, policy enforcement, and governance decisions across the AI stack.

How to discover shadow LLMs before controls are built

Security teams should treat shadow LLM discovery as an inventory problem first, not a policy problem. The goal is to find where LLMs are already in use, who is using them, what data they touch, and which systems they can reach. That baseline lets you size exposure before you decide on guardrails, logging, approval flows, or data restrictions.

Discovery usually succeeds when teams look beyond obvious chat apps and examine developer tooling, SaaS features, API integrations, browser extensions, copilots, and automation platforms that can send prompts or retrieve content. In practice, the first pass should identify both sanctioned and unsanctioned usage, then trace the data paths that make a hidden model operationally relevant.

A useful working rule is that if a tool can see enterprise data, call external model APIs, or pass context into retrieval, it belongs in the discovery scope. That includes embedded LLM features inside larger products, because shadow use often hides in software procurement, plugin ecosystems, and loosely governed API keys rather than in standalone AI products.

Trace prompts, data paths, and dependencies

Once obvious entry points are listed, the next step is to trace application and resource usage so hidden dependencies become visible. That means following outbound API calls, reviewing connector inventories, checking cloud and identity logs, and mapping where prompts, embeddings, documents, or responses are stored. The objective is to understand which assets can influence the model, and which outputs can influence business systems.

Enterprise AI Copilot Security Guide is useful here because copilot deployments often expose the exact discovery problem security teams face, connectors, data oversharing, and weak governance around who can turn an AI feature into a data access path.

For enterprises using retrieval augmented generation, discovery must include the retrieval layer itself. A hidden model may not sit in a central platform at all, it may be embedded in search, ticketing, knowledge management, or document workflows where access to source data is effectively access to model output. That makes permission mapping, connector review, and data classification part of discovery, not a later hardening exercise.

Build the first inventory from observable evidence

The most reliable discovery sources are the ones that already record how software behaves. Start with cloud audit logs, proxy and DNS telemetry, CASB or SSE records, API gateway traffic, software inventory, browser extension inventories, SaaS admin consoles, and developer platform logs. Then correlate those signals with service accounts, user tokens, third-party integrations, and outbound traffic to known model providers or AI middleware.

AI Infrastructure Workload Identity Guide helps anchor that inventory in the identities behind AI platforms, including pipelines, notebooks, inference services, vector stores, and the cloud credentials that let those components run.

LLMjacking NHI credential abuse is a reminder that exposed AI credentials are not just a billing issue, they are an indicator that an unknown model path may already be live and consuming enterprise resources. Discovery should therefore look for spend anomalies, unusual API usage, and orphaned keys alongside the more obvious AI application list.

Risk and Threat Considerations

Shadow LLMs create exposure before any formal control exists, which means data leakage, unsafe access paths, and untracked third-party dependencies can accumulate silently. The biggest risk is not simply that a model exists, it is that a model can be reached through unmanaged prompts, connectors, or credentials that security teams never inventoried.

Failure mechanism: Hidden integrations allow prompts, retrieved documents, or responses to flow through systems that were never assessed for data classification, access control, retention, or external sharing. That can turn ordinary tooling into an unsanctioned enterprise AI path.

Impact: Sensitive data can be exposed, governed data can bypass review, and security teams may only discover the LLM after usage has already spread across teams, vendors, and automations.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

CIS Controls v8, NIST SP 800-53 Rev 5 and CSA Cloud Controls Matrix set the technical controls, while ISO/IEC 27001:2022 defines the regulatory obligations.

Framework	Control / Reference	Relevance
CIS Controls v8	CIS-1 — Inventory and Control of Enterprise Assets	Shadow LLM discovery starts with enterprise software and service inventory.
	CIS-3 — Data Protection	Discovery must identify where prompts, retrieval data, and outputs touch sensitive information.
	CIS-6 — Access Control Management	Hidden LLM paths often rely on unmanaged tokens, keys, or SaaS connectors.
Recommendation — Inventory AI-capable tools and integrations before enforcing controls. Map AI data flows so protection rules can be applied to the right assets. Review and revoke AI-related access paths that lack clear ownership or approval.
NIST SP 800-53 Rev 5	CM-8 — System Component Inventory	Discovery requires a current inventory of systems and services that expose LLM use.
	AU-6 — Audit Record Review, Analysis, and Reporting	Tracing shadow LLMs depends on reviewing logs for prompts, API calls, and unusual use.
	AC-4 — Information Flow Enforcement	Shadow LLM discovery is about finding uncontrolled data movement into external model paths.
Recommendation — Maintain an inventory of AI-enabled components, integrations, and dependencies. Analyze logs to identify unmanaged model usage and hidden integrations. Map and constrain data flows to AI services before they become ungoverned.
ISO/IEC 27001:2022	A.5.9 — Inventory of information and other associated assets	An inventory of AI tools, connectors, and services is central to finding shadow LLMs.
	A.5.15 — Access control	Unsanctioned LLM use often appears through uncontrolled access paths.
Recommendation — Register AI services and integrations as assets subject to governance. Apply access rules to AI tools, connectors, and retrieval sources.
CSA Cloud Controls Matrix	IAM — Identity & Access Management	AI discovery depends on identifying which identities and credentials can reach model services.
	DSP — Data Security & Privacy	Shadow LLMs matter because they can expose or move governed data.
Recommendation — Trace AI usage through identities, tokens, and connected services. Classify and protect the data that can be sent to or retrieved by AI systems.

Practitioner Guidance

What to prioritise: Start with systems that can move sensitive data into or out of an LLM, especially developer tools, copilots, browser extensions, workflow automations, and API-connected SaaS features. Those are the most likely places where shadow use becomes material.

What to verify: Confirm whether each discovered LLM path has an owner, an approved purpose, a known data set, and a traceable credential or integration record. If you cannot name those four things, the control gap is already operational, not theoretical.

Practitioner takeaway: Treat discovery as the prerequisite control, because once LLM usage is visible and mapped to real data flows, policy, logging, and governance become enforceable rather than aspirational.