Employees and their AI agents now use hundreds of generative AI services, browser extensions and copilots that security teams never approved, and each one can receive pasted source code, customer data or credentials. A growing market of AI usage control tools promises to discover that use and govern it in line, but products differ sharply in where they sit (network, browser, endpoint, SaaS API), what they can see and whether they can act before data leaves. This vendor-neutral buyer's guide covers requirements, capability areas, evaluation criteria, vendor questions, red flags and a proof of concept. It is for security, data protection and procurement teams, and for executives sponsoring an AI use policy. Building and running your own AI applications is a different purchase, covered in the AI Security Platform Buyer's Guide.
Key takeaways
- Discovery is only half the job. Knowing which AI apps are in use is useful; the value is in-line control at the moment a prompt, upload or extension install happens.
- Where the tool sits decides what it sees. Network controls see destinations, browser controls see prompts and uploads, endpoint agents see desktop apps and local tools, SaaS API connectors see AI features inside approved apps.
- Coach before you block. Blanket bans push use to personal devices; the better products steer users to an approved tool and only block sensitive data.
- Include agents and OAuth grants. Shadow AI is no longer just chat sites: it includes AI agents, MCP servers and AI apps granted OAuth access to corporate SaaS.
- Prompt content is personal data. Capturing prompts creates an employee monitoring and data retention obligation, so privacy design is an evaluation criterion, not an afterthought.
Define requirements
The Samsung ChatGPT source code leak showed the pattern: staff pasted source code and a meeting transcript into a public AI tool within weeks of it being allowed, with no attacker involved, and the company first capped prompt size and then restricted generative AI tools on company devices. Decide what outcome you want before shortlisting products.
- Policy first: which AI services are approved, tolerated or banned, and which data classes may go to each. Without this, a tool can only report. See the Shadow AI Policy Template.
- Coverage surfaces: managed browsers, unmanaged browsers, desktop AI apps, IDE assistants, mobile, AI features embedded in sanctioned SaaS, and agents or MCP servers run by developers.
- Control actions needed: monitor, warn and coach, require justification, redact, block upload or paste, block app, or redirect to the approved tool.
- Data to protect: source code, secrets and keys, customer personal data, regulated data, financial data, and documents with sensitivity labels.
- Privacy and works council constraints: whether prompt content may be captured at all, where it is stored, who can view it and for how long. Where GDPR applies, monitoring of this kind is likely to need a data protection impact assessment under Article 35. The AI Usage Monitoring Guide covers the privacy design in depth.
- Existing stack: secure web gateway, enterprise browser, DLP, CASB or SSPM tools you already own may cover part of the need; list them so you buy only the gap.
Capability areas
| Deployment point | What it can see | What it can do | Blind spots |
|---|---|---|---|
| Network (secure web gateway, SSE, DNS) | Destinations and app categories; content only with TLS inspection | Discover and block apps, apply coarse policy | Prompt context, desktop apps using pinned certificates, off-network devices |
| Browser extension or enterprise browser | Prompts, pastes, uploads, extensions installed, per-tab context | Warn, redact, block paste or upload, steer to approved tools | Desktop apps, unmanaged browsers, command-line tools |
| Endpoint agent | Desktop AI apps, local models, clipboard, file movement, developer tools | Block or log at the process and file level | Unmanaged and personal devices; deep prompt semantics vary |
| SaaS API connector | AI features enabled in sanctioned SaaS, OAuth grants to AI apps | Report and revoke grants, adjust tenant settings | Real-time prompt interception |
Most organisations need two of these combined: typically browser plus SaaS API, with endpoint coverage for developers. See the Browser Security Buyer's Guide and the DLP Buyer's Guide for adjacent purchases.
- Discovery and catalogue: continuously updated catalogue of AI apps with risk attributes (data retention, training on inputs, hosting region, certifications), and per-user and per-team usage.
- Prompt and upload DLP: detection of secrets, source code, personal data and labelled documents in prompts and files, with redaction or blocking before submission. See the Generative AI Data Loss Prevention Guide.
- Account context: distinguishing a corporate tenant of an AI service from a personal account of the same service, so policy can allow one and block the other.
- User coaching: in-context messages, justification prompts and redirection to the approved tool, with the response recorded.
- Agent and OAuth governance: discovery of AI apps and agents holding OAuth grants or API keys to corporate systems, and of unapproved MCP servers. See the Shadow AI Discovery Guide.
- Reporting and integration: events to the SIEM, tickets for exceptions, and evidence for audits of the AI use policy.
Evaluation criteria
| Criterion | What good looks like | Evidence to request |
|---|---|---|
| Coverage of your surfaces | Works on your browsers, operating systems and developer tools today | Supported platform matrix; live test on your builds |
| Detection accuracy | Finds secrets, code and personal data in prompts and files with low false positives | Results on a test corpus you supply |
| Corporate versus personal account | Reliably tells tenant from personal sign-in for major AI services | Demonstration with both account types |
| In-line action latency and user experience | Decisions in under a second, clear coaching text, no broken pages | Pilot user feedback; measured latency |
| Catalogue quality | Frequent updates, documented risk attributes, quick addition of new apps | Update cadence; time to add an app you request |
| Privacy controls | Configurable content capture (off, metadata, redacted, full), regional storage, role-based access to prompt records, retention limits | Admin console walkthrough; data processing terms |
| Agent and OAuth visibility | Inventory of AI apps with grants to your SaaS and the scopes held | Report from a pilot tenant |
| Tamper resistance | Users cannot disable the extension or agent without detection | Test of removal and the resulting alert |
| Integration | SIEM, identity provider groups, ticketing, existing DLP labels | Connector documentation |
Questions to ask vendors
- Which of our surfaces (browsers, desktop apps, IDEs, mobile, agents) do you cover today, and which only on the roadmap?
- How do you distinguish a corporate tenant from a personal account of the same AI service?
- Show detection of an API key, a block of source code and a customer record pasted into a prompt. What is redacted, and what does the user see?
- What prompt content do you store by default, where, for how long, and who in our organisation can read it? Can capture be metadata only?
- Do you use our prompts, files or telemetry to train models or improve detections?
- How quickly do new AI apps and browser extensions enter your catalogue, and how do you assess their data retention and training terms?
- Can you discover AI agents, MCP servers and AI apps holding OAuth grants to our SaaS tenants?
- What happens when a user disables the extension, uses another browser or works off network?
- What is the measured added latency for in-line inspection?
Red flags
- Discovery dashboards sold as control: lists of AI apps with no ability to act before data is submitted.
- Full prompt capture by default with no retention limits: creates a sensitive data store of its own and a privacy problem.
- Block-only policy models: no warn, coach or redirect options, which drives users to personal devices.
- No account context: cannot tell a sanctioned enterprise tenant from a personal account on the same domain.
- Easy bypass: extension removable without alert, or coverage limited to one browser.
- Agents ignored: nothing on OAuth grants, API keys or MCP servers, so the fastest-growing shadow AI goes unseen. See Top 10 Shadow AI Risks.
Proof of concept
- Agree policy and success criteria: approved and banned AI services, data classes, expected actions, latency and false positive targets.
- Pick a representative pilot: engineering, sales, legal and finance users on managed devices, 30 to 100 people, for three to four weeks.
- Start in monitor mode: collect a week of discovery to baseline which AI services and extensions are actually used.
- Run seeded tests: paste a fake API key, sample source code and synthetic personal data into approved and unapproved tools, through personal and corporate accounts; upload a labelled document.
- Turn on coaching and blocking: measure how many users switch to the approved tool after a coaching message, and how many justification overrides occur.
- Test bypass: disable the extension, use an unmanaged browser and a desktop app, and confirm alerts.
- Review privacy outputs: with your data protection officer, check what was stored and who could see it.
Measure success with a short list: share of AI traffic going to approved services, sensitive data events blocked or redacted per month, percentage of users reached by coaching who then use the approved tool, number of AI apps with OAuth grants reviewed, and time to add a newly discovered AI app to policy.
How NHI Mgmt Group can help
We provide independent market analysis, RFP strategy and evaluation support for NHI and agentic AI identity programmes, including AI usage control and shadow AI tooling. Browse vendors in our NHI and AI products directory or contact us about independent evaluation support.
Related NHI Mgmt Group resources: AI Usage Monitoring Guide · Shadow AI Discovery Guide · Generative AI Data Loss Prevention Guide · Shadow AI Policy Template · AI Security Platform Buyer's Guide