LLM Discovery

LLM discovery is the process of locating where language models are used across applications, workflows, SaaS tools, and embedded integrations. It identifies model endpoints, retrieval layers, and connected tools so security teams can assign ownership, assess exposure, and enforce policy before testing or deployment.

Expanded Definition

LLM discovery is the inventorying and mapping of large language model usage across an organisation’s environment, including applications, SaaS services, workflows, retrieval components, and tool-connected agents. It goes beyond identifying a chat interface or API call. A useful discovery process also traces where prompts enter, where outputs flow, which model endpoint is invoked, what data is retrieved, and which identities or service accounts can trigger the interaction.

That distinction matters because LLMs are often embedded indirectly, such as through low-code platforms, customer support tools, developer copilots, or agentic workflows. In practice, discovery supports ownership assignment, policy scoping, risk classification, and control selection before a model is allowed to process sensitive information. This aligns closely with the governance intent of the NIST AI Risk Management Framework and the NIST AI 600-1 Generative AI Profile, both of which emphasise understanding AI system context, impact, and risk.

The most common misapplication is treating LLM discovery as a one-time software inventory task, which occurs when teams miss hidden integrations, shared API keys, or shadow AI embedded in business workflows.

Examples and Use Cases

Implementing LLM discovery rigorously often introduces visibility and coordination overhead, requiring organisations to weigh faster AI adoption against the cost of continuous tracking and owner validation.

• Mapping customer service platforms that use embedded LLMs for response drafting, summarisation, or retrieval from internal knowledge bases.
• Identifying developer tools and IDE extensions that send source code, secrets, or internal prompts to third-party models.
• Tracing agent workflows that chain an LLM to ticketing, email, or document systems through tool calls and plugin connectors, a pattern highlighted in the OWASP Top 10 for Agentic Applications 2026.
• Cataloguing retrieval-augmented generation pipelines to determine which indexes, documents, and permissions influence model outputs.
• Finding unsanctioned model endpoints in SaaS products where a business unit enabled generative features without security review.

Discovery also helps teams distinguish direct model use from adjacent AI services, such as summarisation features or embedded copilots, so security controls are applied at the right layer. Where agentic behaviour is involved, the OWASP Agentic AI Top 10 and the CSA MAESTRO agentic AI threat modeling framework are useful references for understanding how tool access expands the exposure surface.

Why It Matters for Security Teams

Security teams cannot govern what they cannot see. Without LLM discovery, organisations tend to miss data exposure paths, over-permissioned service accounts, uncontrolled retrieval sources, and model dependencies that sit outside normal application ownership. That creates blind spots for policy enforcement, incident response, data classification, and vendor risk review.

LLM discovery is especially important where identity and access intersect with AI usage. A model endpoint may be low risk in isolation, but the surrounding workflow may be driven by privileged credentials, long-lived API keys, or delegated access that should be treated as a non-human identity control problem. In those cases, discovery is the prerequisite for applying least privilege, boundary testing, and usage restrictions across the full AI interaction chain. The threat relevance is reflected in sources such as the MITRE ATLAS adversarial AI threat matrix and the Anthropic report on AI-orchestrated abuse, both of which show how hidden AI touchpoints can be operationalised by attackers.

Organisations typically encounter the true cost of poor LLM discovery only after an incident, when an exposed integration, sensitive prompt trail, or misrouted retrieval source makes model governance operationally unavoidable.

Related resources from NHI Mgmt Group

• How should security teams reduce false positives in LLM-assisted vulnerability discovery?
• Why is NHI discovery and inventory the primary goal of NHI security?
• Why is continuous discovery of AI agents important?
• When does secrets discovery become insufficient on its own?