Embedded AI in SaaS: Governing Vendor AI Features

SaaS vendors are adding AI assistants, summaries, search answers and agents to applications you already approved, often inside existing licences and sometimes enabled for every user, without a new procurement or security review. The data these features read is the same data the application already holds, but the processing is new: prompts and content may go to a third-party model provider, be retained, cross a data residency boundary or be combined across apps. This guide explains how embedded AI differs from a standalone AI tool, which admin controls and data-use terms the major platforms document, how to build a review process that keeps up with vendor release cycles, and what to monitor. It is for SaaS owners, security architects, privacy teams and governance leads.

Key takeaways

Embedded AI is a change to an approved service. Treat a new AI feature as a material change in the supplier relationship that triggers review, not as a product update.
Look for the "future features" switch. Some platforms let admins decide whether new AI features are enabled automatically; Slack, for example, has an "Always allow access to new AI features" setting for Primary Owners.
Permission-trimmed is not the same as safe. Most assistants only show what the user can already access, which turns oversharing in the underlying app into an AI exposure problem.
Read the data-use terms per plan. Training and retention commitments often differ between enterprise and lower tiers, and between first-party and third-party models.
Govern the connections, not just the toggles. AI features that pull from other apps, the web or third-party agents widen the data flow; connected consumer AI apps were the entry point in the Vercel Context.ai breach.

What embedded AI is and why it is different

Embedded AI is any generative AI capability delivered inside a SaaS product the organisation already uses: a copilot side panel, meeting summaries, AI search answers across a workspace, generated content in a document editor, or an agent that can act inside the application. It differs from shadow AI (a tool staff sign up for themselves) in three ways.

It arrives through a trusted channel: the vendor is already approved, SSO is in place and the data is already there, so no proxy, CASB or procurement step sees a new service.
The processing chain changes: a vendor that stored and displayed your data now sends parts of it to a model, sometimes run by a subprocessor. Zoom, for example, states that for features using third-party model providers it shares relevant data with those providers.
Scope can widen silently: Atlassian documents that when AI is activated for any app on a site, Rovo and other AI features can access and use data from any app on that site. A decision taken for one product changes the reach for all of them.

The security questions are therefore supplier and data governance questions first: what data does the feature read, where is it processed, is it retained or used for training, who can use it, and can it act. See the SaaS Security Guide and the Third-Party AI Risk Guide.

Vendor admin controls: what the platforms document

The table summarises the admin controls named in each vendor's own documentation at the time of writing. Names change often, so confirm in your tenant.

Platform	Where AI is controlled	Scope options	Notable settings
Google Workspace (Gemini app)	Admin console: Menu, Generative AI, Gemini app (requires the Gemini Settings administrator privilege)	Organisational unit or configuration group; group settings override OU settings	Service status (On or Off for everyone); User access, including "Allow all users to access the Gemini app, regardless of license"; Gemini conversation history, on by default, retention 3, 18 or 36 months (default 18)
Microsoft 365 Copilot	Microsoft 365 admin center (Integrated apps for agents); privacy controls for optional connected experiences; Microsoft Purview for stored interactions	Per user licence and per agent assignment	Web search through Bing can be turned off as an optional connected experience; agents only available if the admin allows them; retention policies for Copilot interactions in Purview
Slack	Admin, Workspace settings (or Organization settings on Enterprise), Roles & permissions, Feature access, AI	Everyone or No one; on Enterprise, specific people and groups or everyone except them	Per-feature toggles for using files, third-party connectors in AI workflow steps and Slackbot web search; Future AI feature releases with "Always allow access to new AI features" (Primary Owners only, does not cover future Slackbot releases)
Atlassian (Rovo)	Atlassian Administration, AI and Rovo, Access, Add app, Block access	Per app; Jira-family shared features stay on while any Jira app on the site has Rovo enabled	Blocking an app disables current and upcoming AI features for it; Bitbucket Cloud uses Workspace settings, AI, AI features
Zoom (AI Companion)	Web portal Admin Center: Settings (account) or Users, Groups (group), Zoom AI section	Account, group or user; lock icon prevents user changes	Some features manageable only at account or group level; locked features appear greyed out to users
Notion AI	Workspace settings	Workspace	Models without zero data retention are off by default and need workspace owner opt-in

Two patterns stand out. First, defaults are rarely documented in one place: Google states some defaults explicitly, while Atlassian and Zoom's pages read for this guide do not state whether features start on or off. Second, the strongest control is often a forward-looking one. Slack's future-features setting and Atlassian's statement that blocking an app also disables upcoming AI features are the only ways to stop the next release from switching itself on.

Data-use terms: training, retention and residency

Vendor commitments are the core of the risk assessment, and they are not uniform. The statements below are from each vendor's primary pages, paraphrased.

Vendor	Training	Retention and processing	Caveats to check
Microsoft 365 Copilot	Prompts, responses and data accessed through Microsoft Graph are not used to train foundation LLMs	Interactions stored encrypted as Copilot activity history; manageable with Content search and Purview	EU traffic stays in the EU Data Boundary, but models from Anthropic as a subprocessor are currently excluded from it; third-party agents follow their own terms
Google Workspace Gemini	For qualifying Business, Enterprise and Education editions, content is not human reviewed or used for model training outside the domain without permission	Conversation retention configurable (3 to 36 months)	Users accessing Gemini as an additional service without a qualifying edition may have chats reviewed by humans and used to improve Google products
Slack AI	Customer data is not used to train third-party LLMs; retrieval augmented generation at inference time	Third-party LLMs run inside Slack's infrastructure	Feature set varies by plan; legacy plans without the add-on get fewer features
Salesforce	Trust Layer: prompts and responses not stored or used to train third-party LLMs (zero data retention)	Data masking replaces sensitive PII with tokens before the prompt is sent; dynamic grounding respects permissions	Confirm which features and models sit behind the Trust Layer in your org
Zoom	Does not use audio, video, chat, screen sharing, attachments or other communications-like content to train its or third-party models	Relevant data shared with third-party model providers when a feature uses them; may be processed in US data centres	Models may change as Zoom updates them
Atlassian	Not addressed on the admin page read	Encrypted in transit; some providers such as OpenAI may process data outside the EU	Cross-app data access once AI is on for any app on a site
Notion	By default neither Notion nor its AI providers train on customer data	Most models zero data retention; on Free, Plus and Business plans providers keep data for 30 days or fewer	Some features may need data-retaining models that admins enable

Plan tier matters: the Google and Notion terms both differ by edition or plan. A team on a lower tier, or a user on a personal licence, may be under different terms from the enterprise contract.
Subprocessor lists move: Notion states its provider list changes over time. Subscribe to subprocessor change notices and treat a new model provider as a review trigger.
Residency covers storage, not always inference: check where model calls are processed separately from where the data is stored, as the Microsoft EU Data Boundary caveat shows.

The real exposure: permissions and connected data

Every vendor read for this guide says its assistant respects existing access: Copilot only surfaces data the user has at least view permission to, Slack AI search will not return results regular search would not, and Salesforce grounding maintains permissions. That is necessary but it shifts the risk. Content that was technically shared with everyone but practically undiscoverable becomes one question away.

Fix oversharing before enabling: review "anyone in the organisation" links, public channels and spaces holding sensitive material, and stale group memberships. See the Enterprise AI Copilot Security Guide.
Use labels the AI honours: Microsoft states Copilot honours sensitivity labels and the usage rights granted to the user under Information Rights Management.
Control cross-source reach: turn off file use, web search and third-party connectors in AI features unless needed (Slack and Microsoft both expose such switches).
Treat agents and connectors as integrations: an AI agent or connector that reads from another app is an OAuth or API integration with its own scopes and token. Review it as one. See the SaaS and OAuth App Governance Guide.
Prompt injection travels with content: an assistant that reads email, documents or tickets can be steered by text an outsider wrote. OWASP lists this as LLM01:2025 Prompt Injection, and MITRE ATLAS as AML.T0051 LLM Prompt Injection. Limit what the assistant can do with what it reads, especially sending or writing data.

A review process that keeps pace with vendors

Build the inventory: for each approved SaaS application, record which AI features exist, which are on, for whom, which model providers are involved and which plan terms apply. Vendor release notes and admin consoles are the source; map this to your SaaS inventory.
Set a default posture per data class: applications holding regulated or highly confidential data start with AI features off and future features not auto-enabled; low-risk collaboration tools can allow approved features by default.
Switch off auto-enablement where possible: uncheck Slack's "Always allow access to new AI features", block upcoming features per app in Atlassian, and lock settings in Zoom so users cannot turn features back on.
Assess each feature: data read, processing location, retention, training use, subprocessors, ability to act, logging available, and whether it can be scoped to groups. Record a decision and an owner.
Pilot with a group: use OU, group or licence scoping to enable for a pilot ring, monitor usage and data access for a few weeks, then widen.
Update contracts: ask for written training and retention commitments, subprocessor change notice and AI feature change notice in renewals.
Roll back cleanly: know the switch for each feature, how long it takes to propagate (Google states up to 24 hours for Gemini app access changes), and how to delete stored AI interactions if required.
Re-review on change: new model provider, new data source, new action capability, or a change of terms each reopens the assessment.

Decision guidance

Enable, scope or block: enabling for everyone suits low-sensitivity tools with clean permissions; scoping to groups suits most cases; blocking fits apps with regulated data where the vendor cannot give acceptable terms or residency.
Read-only versus acting features: summaries and search answers carry disclosure risk; agents that send, create or change records also carry integrity risk and need stricter review and logging.
Bundled versus add-on: paid add-ons go through procurement and are easier to govern; features bundled into existing licences need the inventory and change triggers above because no purchase will flag them.
Small organisations: concentrate on the two or three platforms holding most data, turn off auto-enablement, and read the data-use page for your plan.
Large organisations: assign each SaaS owner responsibility for AI features in their app, integrate the review into the AI governance process, and pull AI usage and audit data into the SIEM. See the AI Governance Guide.

Detection and measurement

Admin changes: alert on AI feature setting changes in each admin console, including future-feature switches and per-feature toggles.
Usage: track which users and groups use AI features, against the approved scope.
Interactions: where the platform stores interactions (Copilot activity history in Purview, Gemini conversation history), apply retention and use eDiscovery and DLP over them. See the Generative AI Data Loss Prevention Guide.
New integrations: alert on new AI agents, connectors and OAuth grants with broad scopes.

Metric	Target
Approved SaaS apps with an AI feature inventory entry	100% of high and medium data class apps
AI features enabled without a recorded decision	0
Platforms with auto-enablement of new AI features turned off (where available)	100% for high data class apps
Time from vendor AI feature announcement to decision	Within the release notice period
AI-connected OAuth grants with broad scopes and no owner	0
Vendors with written training and retention terms on file	100% of vendors with AI features enabled

Common failure patterns

Features on before the review: a vendor release enables an assistant for every licensed user and the first security knows of it is a user question.
Terms checked for the wrong tier: the enterprise contract says no training, but a department or personal licence runs on different terms.
Oversharing amplified: an assistant faithfully respects permissions that were never right, and confidential files appear in answers.
Cross-app scope missed: AI is approved for one product and, by design, reads data from every app on the same site.
Consumer AI connected to corporate accounts: in the Vercel Context.ai OAuth Breach 2026, an employee signed up for a consumer AI office suite with a corporate Google account and granted it full access; after the AI vendor was compromised, the stolen OAuth token was used to take over that account and move into Vercel's environment.
Agents treated as features: an AI agent with write access is approved as a toggle rather than reviewed as a privileged integration with its own identity. See the Shadow AI Discovery Guide.

Framework mapping

Area	NIST SP 800-53 Rev. 5	NIST CSF 2.0	Threat references
Supplier review of AI features and terms	SA-9 External System Services; SA-4 Acquisition Process; SR-6 Supplier Assessments and Reviews	GV.SC-05; GV.SC-07	OWASP LLM02:2025 Sensitive Information Disclosure
Inventory of AI features and data	RA-3 Risk Assessment	ID.AM-02; ID.AM-07	Not applicable
Disable unneeded AI functions	CM-7 Least Functionality	PR.AA-05	OWASP LLM06:2025 Excessive Agency
Assistant surfaces repository data	AC-6 Least Privilege	PR.AA-05	ATT&CK T1213 Data from Information Repositories; ATLAS AML.T0057 LLM Data Leakage
Stolen tokens of AI integrations	AU-2 Event Logging	ID.AM-02	ATT&CK T1528 Steal Application Access Token; T1550.001 Application Access Token

For management systems, ISO/IEC 42001:2023 specifies requirements for an AI management system covering the use, not only development, of AI, and ISO/IEC 27001:2022 Annex A 5.19 Information security in supplier relationships and 5.23 Information security for use of cloud services apply directly. NIST AI 600-1, the Generative AI Profile of the AI Risk Management Framework, is a voluntary companion for generative AI risk. CIS Controls v8.1 Control 15 (Service Provider Management) covers the supplier side.

Practitioner checklist

List every approved SaaS application with AI features, which are enabled and for whom.
Turn off automatic enablement of future AI features where the platform allows it, and lock user-level settings.
Read and file the data-use terms for the plan you are actually on, including subprocessors and residency of inference.
Fix oversharing and apply sensitivity labels before enabling search answers or copilots.
Scope new AI features to a pilot group through OU, group or licence assignment before wider release.
Disable web search, file use and third-party connectors in AI features unless there is a recorded need.
Review AI agents and connectors as OAuth integrations with owners, scopes and expiry.
Apply retention and DLP to stored AI interactions.
Add AI feature and subprocessor change notice to vendor contracts.

Embedded AI in SaaS Guide: Governing AI Features Switched On by Vendors