AI security has become a crowded market, with products for discovering AI use, assessing AI posture, filtering prompts and outputs, testing models, securing agents and governing AI risk, and the labels overlap more than the datasheets admit. AI security posture management (AI-SPM), AI firewalls and guardrails, AI gateways, AI red teaming, AI governance and agent security are sold as separate categories, yet many products cover two or three of them. This vendor-neutral buyer's guide is for security architects, AI platform teams, IAM teams and procurement leads writing requirements or running an RFP. It explains the main categories and their limits, why identity decides the blast radius when an AI system is manipulated, and how to score products against your real AI estate with weighted criteria and a proof of concept.
Key takeaways
- Inventory before you shop: list sanctioned AI applications, shadow AI, in-house AI development, agents and the AI features inside your SaaS. The categories you need follow from that list.
- Guardrails reduce risk but cannot guarantee prevention: OWASP states that it is unclear whether fool-proof prevention of prompt injection exists. Pair runtime filtering with identity and access controls that limit what a manipulated model or agent can do.
- Identity is the blast-radius control: prefer products that discover and govern the API keys, service accounts, OAuth grants and agent identities AI systems use, not only the text they exchange.
- Weight the criteria: so a polished dashboard cannot outscore coverage, detection quality and data handling.
- Prove it on your own applications: run your RAG assistant, chatbot and agents through the product with your own attack set, and measure detection, false positives, latency and operating effort.
Define requirements
Start with how AI is actually used, then decide which risks you are buying controls for. Consuming public AI tools calls for discovery and data protection; building RAG applications and agents calls for runtime controls, testing and identity governance.
| AI use in your estate | Main risks | Categories to prioritise |
|---|---|---|
| Staff using public AI tools and browser AI | Sensitive data pasted into prompts; unsanctioned tools; OAuth-connected AI apps | Discovery and usage governance; data loss prevention for prompts |
| AI features inside SaaS and copilots | Oversharing through retrieval; indirect prompt injection in email and documents | Discovery; runtime guardrails; permission review of the data the copilot can reach |
| In-house RAG applications and chatbots | Prompt injection, sensitive information disclosure, vector store access | Runtime guardrails; AI gateway; red teaming; AI-SPM for the cloud services behind them |
| Agents with tools and MCP servers | Excessive agency, tool misuse, identity and privilege abuse | Agent security; identity and secrets tooling; red teaming of tool paths |
| Model training and hosting | Data and model poisoning, exposed model infrastructure, stolen provider keys | AI-SPM; supply chain and model scanning; secrets management |
- Scope and ownership: name who owns AI security (security, the AI platform team or a governance function) and who will run the tool day to day.
- Architecture constraints: record whether traffic to model providers can be routed through a proxy or gateway, whether SDKs can be added to applications, and which clouds and agent frameworks are in use.
- Regulatory drivers: if you provide or deploy high-risk AI systems under the EU AI Act, Article 12 requires those systems to allow automatic recording of events (logs) over their lifetime, and Article 15 requires resilience against attempts by unauthorised third parties to alter their use, outputs or performance. See the EU AI Act Compliance Guide.
Map requirements to frameworks
Ask vendors to map features to the frameworks you already report against. The NIST AI RMF organises activity into four functions (GOVERN, MAP, MEASURE and MANAGE), and the NCSC guidelines for secure AI system development split the life cycle into secure design, development, deployment, and operation and maintenance. The function column below is our judgement of where each requirement mainly sits; the identifiers in the other columns are taken from the framework texts.
| Requirement | OWASP LLM Top 10 2025 | OWASP Agentic Top 10 2026 | MITRE ATLAS | NIST AI RMF function |
|---|---|---|---|---|
| Detect and contain prompt injection, including indirect injection in retrieved content | LLM01 Prompt Injection | ASI01 Agent Goal Hijack | AML.T0051 LLM Prompt Injection | MEASURE, MANAGE |
| Detect jailbreak attempts | LLM01 Prompt Injection | Not mapped here | AML.T0054 LLM Jailbreak | MEASURE |
| Stop sensitive data leaving through prompts and outputs | LLM02 Sensitive Information Disclosure | Not mapped here | Not mapped here | MANAGE |
| Limit agent tools and permissions | LLM06 Excessive Agency | ASI02 Tool Misuse and Exploitation; ASI03 Identity and Privilege Abuse | Not mapped here | GOVERN, MANAGE |
| Rate-limit and meter model use | LLM10 Unbounded Consumption | Not mapped here | AML.T0034.002 Agentic Resource Consumption | MANAGE |
| Protect retrieval and vector stores | LLM08 Vector and Embedding Weaknesses | ASI06 Memory and Context Poisoning | Not mapped here | MAP, MANAGE |
See the OWASP Top 10 for LLM Applications Guide and the NIST AI RMF Guide.
Categories and capability areas
Score each shortlisted product against every category it claims, not only its headline one.
| Category | What it does | Limits |
|---|---|---|
| AI discovery and usage governance | Finds AI apps, browser use, OAuth-connected AI tools and AI API keys; enforces usage policy | Visibility into custom-built AI varies |
| AI security posture management (AI-SPM) | Inventories AI models, pipelines, data and services in cloud; finds misconfigurations and risky access | Mostly cloud-focused; limited runtime protection |
| Runtime guardrails / AI firewalls | Inspect prompts, retrieved content and outputs for injection, sensitive data and policy violations | Probabilistic detection; latency; can be bypassed |
| AI gateways | Centralise model access, authentication, rate limits, logging and routing | Only cover traffic routed through them; concentrate provider keys |
| AI red teaming and testing | Test models and applications for jailbreaks, injection and unsafe behaviour | Point-in-time unless continuous |
| Agent security | Agent inventory, identity, tool and MCP control, runtime monitoring | Fast-moving; coverage of agent platforms varies |
| AI governance and risk | Model inventory, risk assessments, policy and regulatory mapping | Process-focused; limited technical enforcement |
- Discovery: the starting point for most buyers, because you cannot protect AI use you cannot see. See the Shadow AI Discovery Guide.
- AI-SPM: strongest for cloud-hosted models, notebooks, training pipelines and their permissions. See the AI Security Posture Management (AI-SPM) Guide.
- Guardrails: filtering is one of several OWASP mitigations for prompt injection, alongside least privilege, human approval for high-risk actions and segregating untrusted content. EchoLeak showed why it is not enough on its own: according to BleepingComputer, the hidden prompt was phrased like a normal message to a human and bypassed Microsoft's cross-prompt injection classifier. See the AI Guardrails and AI Firewalls Guide and EchoLeak 2025.
- Gateways: a gateway holds every provider key behind it, so treat it as an identity boundary that must fail closed. See LiteLLM MCP Auth Bypass 2026.
- Red teaming: NIST AI 600-1 repeatedly recommends AI red-teaming among its suggested actions for generative AI. See the AI Red Teaming and Testing Tools Buyer's Guide.
- Agent security and governance: see the AI Agent Identity Security Buyer's Guide and the AI Governance Platform Buyer's Guide.
Where identity fits
Every AI system uses identities: API keys to model providers, service accounts for pipelines, OAuth grants for AI apps, and agent identities with tool access. These determine the blast radius when AI is misused. OWASP's mitigations for LLM06 Excessive Agency include minimising extension permissions, executing extensions in the user's context and enforcing authorisation in downstream systems rather than relying on the model to decide. The OWASP Agentic Top 10 goes further under ASI03: without a distinct, governed identity of its own, an agent operates in an attribution gap that makes true least privilege impossible. Evaluate whether products:
- Discover AI credentials: find model provider API keys, cloud access keys with model permissions and OAuth-connected AI apps, with owner and permissions. LLMjacking campaigns run on exactly these machine credentials. See the LLMjacking Guide and LLMjacking 2024 to 2026.
- Attribute activity: tie every request to a user, application or agent, not a shared key.
- Integrate: connect to your identity provider, secrets manager and NHI tooling rather than keeping a separate credential store. See the NHI Security Platform Buyer's Guide and Secrets Management Guide.
- Enforce least privilege for tools: scope agent tools per task, with short-lived credentials, as ASI03 recommends, rather than only filtering text. See the AI Agent Authorisation Guide.
- Understand MCP: the MCP security best practices warn that token passthrough creates a confused deputy and breaks audit trails, and recommend a minimal initial scope set with incremental elevation. See the MCP Security Guide.
Evaluation criteria
The weights below are a starting point for an organisation building its own AI applications and agents. Shift weight to discovery and data protection if most AI use is third-party and agree weights before demonstrations.
| Criterion | Weight % | What good looks like |
|---|---|---|
| Coverage | 20 | Supports the model providers, clouds, frameworks, SaaS AI features and agent platforms you run today, with a published roadmap for the rest |
| Detection quality | 20 | High detection of direct and indirect injection, jailbreaks and sensitive data on your test set, with a false positive rate you can live with in production |
| Identity and credential control | 15 | Discovers AI credentials and agent identities, attributes activity, scopes tool access and integrates with your IdP and secrets manager |
| Data handling | 10 | Clear processing location, configurable retention, no training on customer prompts, redaction before storage |
| Performance and resilience | 10 | Measured added latency under load; a documented, configurable fail-open or fail-closed choice per application |
| Deployment fit | 10 | Inline proxy, SDK, API, browser or agentless options that match your architecture without re-platforming |
| Logging and explainability | 10 | Clear reason for each block or finding; full logs exportable to your SIEM to support incident response and record-keeping |
| Framework mapping and reporting | 5 | Findings mapped to OWASP LLM and Agentic Top 10, MITRE ATLAS, NIST AI RMF and the EU AI Act where relevant |
Score each criterion from 0 to 5 in the proof of concept, multiply by the weight and keep the evidence. See the AI Agent Observability and Incident Response Guide.
Questions to ask vendors
- Show detection of an indirect prompt injection in a retrieved document for our RAG application, and show what happens to the output.
- How do you find AI API keys, cloud credentials with model access and OAuth-connected AI apps across our estate?
- What happens when your detection misses an injection? Which identity, authorisation or human-approval controls limit the damage?
- How do you secure agent tool calls and MCP servers, and can you detect token passthrough or over-broad scopes?
- Can every request be attributed to a user, application or agent identity, and can you revoke one agent's access without affecting others?
- What latency do you add at our expected volume, and do you fail open or fail closed if your service is unavailable?
- Where are our prompts and outputs processed and stored, for how long, and are they ever used to train models?
- How quickly do you update detections for new techniques?
- Which findings map to the OWASP LLM Top 10, OWASP Agentic Top 10 and MITRE ATLAS, and can we export them?
Red flags
- Claims to "prevent all prompt injection", which contradicts OWASP's own assessment.
- No answer on where prompts and outputs are processed, retained or used for training.
- No integration with identity or secrets tooling for AI credentials.
- Coverage limited to one model provider or one agent framework.
- A gateway or proxy that ships with a default admin key or fails open on authentication errors.
- Detection rates quoted only from the vendor's own benchmark.
- Agent "security" that filters text but cannot see or scope the credentials an agent uses.
Proof of concept
- Pick real use cases: two or three, such as an internal assistant with RAG, a customer-facing chatbot and an agent with tools. Agree pass criteria and weights before you start.
- Build a test set: direct and indirect injection, jailbreaks and sensitive data cases drawn from the OWASP LLM and Agentic Top 10 and MITRE ATLAS techniques, plus a larger set of normal traffic. Success: detection and false positive rates recorded per category.
- Test indirect injection end to end: plant instructions in a document, email or web form your application retrieves, as in ForcedLeak 2025. Success: injection flagged, and no data reaches an untrusted URL.
- Test discovery: seed shadow AI use, an unsanctioned OAuth-connected AI app and a planted test API key. Success: each found with owner and permissions within the agreed time.
- Test agent controls: attempt a tool call outside the agent's task scope and a privilege escalation through a second agent. Success: blocked by policy or held for human approval, and logged against the agent's identity.
- Measure performance: run production-like load. Success: added latency within your budget, and the fail-open or fail-closed behaviour matches your configuration.
- Check data handling and logs: confirm retention settings, redaction and SIEM export. Success: every block and finding is explainable from the logs alone.
- Score and decide: apply the weighted table, add operating effort (tuning hours, alert volume) and keep the metrics after go-live. See the Prompt Injection Guide.
Standards and references
- NIST: AI 100-1 Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- NIST: AI 600-1 Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
- NIST: AI 100-2 E2025 Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations
- EUR-Lex: Regulation (EU) 2024/1689 (Artificial Intelligence Act)
- OWASP: LLM01:2025 Prompt Injection
- OWASP: LLM06:2025 Excessive Agency
- OWASP: Top 10 for Agentic Applications for 2026
- MITRE: ATLAS (Adversarial Threat Landscape for AI Systems) data
- NCSC: Guidelines for secure AI system development
- Model Context Protocol: Security Best Practices
How NHI Mgmt Group can help
We provide independent AI security and agentic AI identity advisory, requirements, RFP and evaluation support. Browse vendors in our products directory or contact us.
Related NHI Mgmt Group resources: Agentic AI Security Guide · AI Agent Identity Security Buyer's Guide · Shadow AI Discovery Guide · Permission-Aware RAG Guide · Generative AI Data Loss Prevention Guide