A hosted model API reached over its public endpoint will answer anyone on the internet who holds a valid key or token, so a leaked credential becomes free inference and a data path out of your organisation. Private endpoints move that traffic onto a private IP inside your network, but they only change the route: identity, the public endpoint itself, DNS and the prompt data still need their own controls. This guide compares public and private exposure for managed AI services, with a threat model for each, the DNS and egress mistakes that quietly undo a private design, and a decision method based on data sensitivity. It is written for cloud architects, platform and AI engineering teams, and security reviewers approving model API access. Platforms are listed alphabetically: Amazon Bedrock, Azure OpenAI in Azure AI Foundry, and Google Cloud Vertex AI (documented by Google in 2026 under the Agent Platform name).
Key takeaways
- The real risk on a public endpoint is the credential, not the network. Stolen AI keys are tested within minutes and resold, as the LLMjacking and Storm-2139 cases show; network restriction limits where a stolen key works, identity-based authentication removes the key.
- A private endpoint is a path, not a lock. Google states that Private Service Connect and Private Google Access keep public reachability unless a VPC Service Controls perimeter is added, and Azure documents a separate firewall setting to block the public endpoint.
- DNS decides which path is used. If private DNS is missing or wrong, clients silently fall back to the public endpoint, or break and someone re-opens public access to fix them.
- Private endpoints do not protect prompt data from what the model, its logs or its tools do with it. Content controls, logging settings, data residency and agent tool permissions sit above the network layer.
- Choose by data sensitivity and caller location. Public endpoint plus strong identity suits low-sensitivity, internet-hosted callers; private endpoint plus disabled public access plus keyless identity suits regulated data and anything called from inside your network.
What changes between a public and a private model endpoint
Every managed model service publishes a regional HTTPS endpoint on a public DNS name. With a public endpoint, clients reach it over the internet or through a NAT gateway, and the service decides whether to answer based on the credential and, optionally, an IP allow list. With a private endpoint, the provider places a network interface with a private IP from your address range in your virtual network, and private DNS points the service name at that IP. Traffic then stays on the provider backbone. The Cloud Private Connectivity Guide explains the underlying PrivateLink, Private Link and Private Service Connect models and the producer and consumer roles; this page applies them to model APIs.
| Dimension | Public endpoint | Private endpoint |
|---|---|---|
| Who can reach the listener | Any internet host; the service authenticates every request | Hosts with a route to the private IP; public listener stays up unless you disable it |
| Where a stolen key works | Anywhere, unless IP rules or a perimeter restrict it | Only from inside the allowed network, once public access is disabled |
| Egress needed by callers | Internet or NAT egress to the provider domain | No internet egress for model calls; the endpoint is local |
| DNS | Public resolution, nothing to manage | Private zones or resolver rules for each service name in use |
| Identity-based network conditions | Source IP conditions on public ranges | Endpoint ID, VPC or organisation conditions on the request |
| Typical callers | SaaS apps, developer laptops, partner systems, prototypes | Workloads, agents and data pipelines in your cloud or data centre |
| Cost and effort | None beyond usage | Per-endpoint and data charges, DNS work, one endpoint per API family |
Threat model for public endpoints
The public endpoint threat model is dominated by credentials. An AI service key or a cloud access key is a bearer secret: whoever presents it is the caller. The LLMjacking campaigns tested harvested keys with quiet requests, checked whether prompt logging was on, enabled models the victim had never used and resold access through reverse proxies, with one exposed Amazon Bedrock key reported in about 75,000 invocations over two days. In the Azure OpenAI Storm-2139 abuse, scraped customer keys were used from outside the victim organisations to generate harmful content at their expense. In both, the lesson is customer-side: a key that works from anywhere is worth stealing.
- Key theft usable from anywhere: keys leak through code, notebooks, CI logs, container images and client-side apps (ATT&CK T1552.001 Credentials In Files); a public endpoint accepts them from any network. See the AI API key abuse guide.
- Cloud account takeover: stolen cloud credentials (T1078.004 Cloud Accounts) can call the model API and also change model access, logging and quotas.
- Cost and capacity abuse: OWASP lists this as LLM10:2025 Unbounded Consumption; without quotas, spend limits and anomaly alerts a stolen key runs up bills fast. See How to set spend limits on LLM API keys.
- Data exfiltration through the API: an insider or malware with a valid credential can send internal documents as prompts to a model account outside your control (T1567 Exfiltration Over Web Service). The public endpoint makes any account at the same provider reachable, including a personal one.
- Interception risk: low. Traffic is TLS to the provider's public certificate; the realistic exposure is TLS inspection proxies that log prompt bodies.
Threat model for private endpoints
A private endpoint shrinks the set of networks that can use a credential, but adds its own risks, mostly from configuration that looks finished and is not.
- Public endpoint left open: the most common gap. The private path works, so nobody notices that the public listener still accepts the same keys from the internet.
- Endpoint as an exfiltration path to other accounts: an interface endpoint for a model API can reach any account of that service in the region unless an endpoint policy or perimeter limits it. Amazon Bedrock documents that the default endpoint policy allows full access through the endpoint; narrow it to your principals and pair it with organisation conditions.
- Over-broad network trust: everything that can route to the endpoint subnet, including a compromised build agent or a peered partner network, can now try credentials against it. Segment who can reach the endpoint subnet.
- Trusted service exceptions: Azure lets trusted services such as Azure Machine Learning and Azure AI Search bypass network rules using managed identity. These paths are legitimate but must be reviewed like any allow rule.
- Producer-side internet egress: agents and pipelines hosted by the platform may need outbound internet for packages or tools. Google documents that inside a VPC Service Controls perimeter the producer's internet access is blocked and recommends a proxy in your own subnet with Cloud NAT; that proxy becomes a control point you must own.
DNS and egress pitfalls
Private endpoints depend on the client resolving the normal service name to a private IP. Each platform does this differently, and each has a failure mode that sends traffic back to the public path.
| Platform | Private connectivity object | DNS behaviour from the docs | Pitfall to test for |
|---|---|---|---|
| Amazon Bedrock | Interface VPC endpoints: bedrock, bedrock-runtime, bedrock-agent, bedrock-agent-runtime, bedrock-mantle (FIPS variants in some Regions) | With Enable Private DNS Name, default Regional names such as bedrock-runtime.region.amazonaws.com resolve to the endpoint; without it, clients must set the vpce endpoint URL | Runtime endpoint created but agent runtime or control plane calls still leave through NAT |
| Azure OpenAI in Azure AI Foundry | Private endpoint on the account subresource | Zones privatelink.openai.azure.com, privatelink.cognitiveservices.azure.com and privatelink.services.ai.azure.com; a Foundry resource can expose all three | Only one zone configured; clients calling the privatelink name directly, which Microsoft says not to do; custom DNS servers not forwarding the zone |
| Google Cloud Vertex AI | Private Service Connect for Google APIs or Private Google Access, plus a VPC Service Controls perimeter | Cloud DNS private zones map Google API names to the restricted VIP range 199.36.153.4 to 199.36.153.7 | Workbench kernels needing private.googleapis.com rather than restricted.googleapis.com; endpoints created before the project joined the perimeter |
- Silent public fallback: if the client's resolver does not see the private zone (on-premises hosts, a second VNet, a developer VPN), the name resolves publicly and the call succeeds over the internet while public access is still enabled. Disable public access first in test, so fallback fails loudly instead.
- NXDOMAIN for other resources: Microsoft warns that a private zone covering a resource type returns NXDOMAIN for public resources of that type with no private endpoint, and recommends the Fallback to Internet option on the virtual network link rather than manual A records.
- DNS is not access control: Microsoft notes that the privatelink CNAME chain resolves from anywhere; resolution proves a resource exists, it grants nothing. Treat the names as discoverable.
- Egress still open to the provider domain: if workloads can reach the public provider domain through NAT or a proxy, a private endpoint does nothing for data sent to someone else's model account. Block the provider's public API names at the egress layer for subnets that should use the endpoint. See Cloud Egress Filtering and the Agent Egress Control and Network Isolation Guide.
What private endpoints do not fix
Network placement answers one question: from where can this API be called. Several of the most important risks to model APIs sit elsewhere.
- Identity: inside the network, any holder of a key still gets full access to the account. Over-privileged workload roles and shared keys remain the main path to misuse.
- Prompt and response data: what you send is processed and may be logged by your own logging configuration, gateways and observability tools. OWASP LLM02:2025 Sensitive Information Disclosure applies whatever the network path.
- Prompt injection and agent tool use: an agent on a private network that reads untrusted content can still be steered into calling tools with its own permissions. See the Prompt Injection Guide.
- Cost abuse from inside: a compromised internal workload can exhaust quota as easily over a private IP.
- Feature trade-offs: perimeters change behaviour. Google lists, for example, that URL context live fetching is turned off and request and response logging is unavailable for Vertex AI projects inside VPC Service Controls; plan detection around those conditions.
Key-based versus identity-based authentication
The decision that matters as much as the network path is whether callers present a static key or a short-lived identity token. Each platform supports moving production callers off static keys.
| Platform | Static key option | Identity-based option | Control to enforce it |
|---|---|---|---|
| Amazon Bedrock | Long-term API keys, tied to an IAM user, which AWS recommends only for exploration | IAM roles with temporary credentials; short-term API keys valid up to 12 hours and inheriting the generating principal's permissions | Deny bedrock:CallWithBearerToken, or limit long-term keys with iam:ServiceSpecificCredentialAgeDays; target key types with bedrock:bearerTokenType |
| Azure OpenAI in Azure AI Foundry | Resource keys (local authentication) | Microsoft Entra ID tokens with managed identities for workloads | Set disableLocalAuth to true; built-in Azure Policy "Azure AI Services resources should have key access disabled (disable local authentication)" |
| Google Cloud Vertex AI | Service account keys for workloads outside Google Cloud | Attached service accounts and Workload Identity Federation; IAM is required on every API call | Organisation policy restricting service account key creation, plus VPC Service Controls access levels |
Two details from the vendor documentation affect rollout. Microsoft states that after disabling local authentication the gateway can keep accepting old keys until its cache refreshes, typically minutes and up to several hours, so verify with a data-plane call that returns HTTP 401 rather than assuming an instant cut-off. AWS notes that Bedrock API keys travel as bearer tokens and are not written to CloudTrail, while the calls themselves are logged. See Workload Identity Federation vs Static Cloud Keys and Service Principals vs Managed Identities for the identity side.
On AWS, network and identity can be combined in one policy: the aws:SourceVpce, aws:SourceVpc and aws:VpceOrgID condition keys are present only when a request arrives through a VPC endpoint, and AWS describes aws:VpceOrgID as the most scalable network perimeter key. Note that aws:SourceIp is absent for requests through a VPC endpoint, so IP-based conditions and endpoint-based conditions need separate statements.
Decision by data sensitivity
Classify the data that will appear in prompts, retrieved context and responses, then pick the tier. Caller location is the second axis: a SaaS product calling your model account from the internet cannot use your private endpoint at all.
| Data tier | Example use | Network exposure | Authentication | Additional controls |
|---|---|---|---|---|
| Public or test data | Prototypes, marketing copy, public documentation search | Public endpoint, IP rules where callers are fixed | Short-lived keys or identity tokens; no long-term keys in code | Spend limits, key scanning, per-key usage alerts |
| Internal | Employee assistants over internal wikis, coding help | Private endpoint for in-network callers, or public endpoint restricted to corporate egress IPs | Identity-based only; keys disabled | AI gateway for logging and quotas; egress block to other model accounts |
| Confidential or personal data | Customer support over case data, HR, contract analysis | Private endpoint, public access disabled, perimeter or endpoint policy limiting to your accounts | Managed or federated workload identity, least-privilege model permissions | DLP on prompts, logging retention rules, data residency checks |
| Regulated or highly sensitive | Health, payment, government or legally privileged data | Private only, perimeter enforced, no internet egress from workloads | Keyless identity, conditional access on admin roles | Separate accounts or projects per workload, change control on network rules and model access |
- Small organisations: start with keyless authentication and spend limits on the public endpoint; that removes most of the LLMjacking risk at almost no cost. Add a private endpoint when the first confidential workload arrives.
- Large organisations: centralise private DNS zones and endpoints in a hub, create them through policy as code, and route model traffic through an AI gateway so every team inherits logging and quotas. See the AI Gateway and LLM Gateway Guide.
- Third-party model APIs without a private option for your cloud: treat them as public endpoints. Put an egress proxy or AI gateway in front, hold the key there rather than in applications, and allow only that gateway to reach the provider.
Worked scenario: moving a claims assistant to private access
An insurer runs a claims summarisation service in a cloud VNet that sends claim notes, which include personal and health data, to a hosted model. Today it uses a resource key from an environment variable and calls the public endpoint through NAT.
- Inventory callers: query logs for every source and identity that called the model account in the last 30 days. The team finds the production service, two data scientists' laptops and a forgotten test function.
- Move to identity: give the service a managed identity with a data-plane user role, deploy, and confirm calls now carry tokens. Move the laptops to individual accounts with Entra ID sign-in through a separate development resource.
- Create the private endpoint and DNS: deploy the endpoint and all three privatelink zones the resource uses, linked to the hub and forwarded from on-premises resolvers. Test resolution from every caller subnet.
- Disable public access in staging first: set the network rule to selected networks and private endpoints with no IP rules. Any caller that was silently using the public path fails here, in staging, not in production.
- Disable keys: set disableLocalAuth, wait for the gateway cache, and verify the old key returns HTTP 401. Revoke the environment variable.
- Close egress: block the provider's public API domain at the firewall for the workload subnet, so the service cannot send claim notes to any other model account.
- Roll back safely: keep the change as code. If production breaks, re-enable selected public IPs for the service's egress address temporarily, never all networks, and fix DNS before removing them again.
The result: a leaked key is useless (keys are off), a stolen token works only from the VNet, and the workload cannot reach other model accounts. What remains is application risk, such as prompt injection in claim notes, which the team handles with content controls and least-privilege tool design.
Detection and measurement
- Log the control plane: alert on changes to public network access, network rules, endpoint policies, disableLocalAuth and perimeter membership. These are the settings an attacker or a frustrated engineer changes first.
- Log the data plane: invocation logs by identity and source; flag first-time model access, new regions and calls from outside expected endpoints.
- Watch DNS: resolver logs showing public addresses for model API names from workload subnets indicate fallback. See the DNS Logging and Detection Guide.
- Watch egress: connections from workload subnets to any model provider domain other than the approved endpoint or gateway.
| Metric | Target |
|---|---|
| Model accounts handling confidential data with public access disabled | 100% |
| Production model calls authenticated with keys rather than identity tokens | 0% |
| Long-term AI keys older than 90 days | 0, trending to none issued |
| Workload subnets able to reach unapproved model provider domains | 0 |
| Mean time to revoke a leaked AI key after detection | Under 1 hour |
Common failure patterns
- Private endpoint deployed, public access still on: the network diagram looks private, while stolen keys still work from the internet, which is exactly how the Storm-2139 keys were used.
- Keys kept "just for the private path": teams disable public access but keep static keys in code, so any foothold inside the network inherits full model access.
- Default endpoint policy: the endpoint allows calls to any account of the service, giving insiders a sanctioned-looking exfiltration route.
- Partial DNS: one privatelink zone configured while the application also calls another hostname of the same resource, so half the traffic goes public.
- Perimeter added after resources: Google warns that inference endpoints and agents created before the project joined a VPC Service Controls perimeter fail or remain publicly reachable; recreate them inside the perimeter.
- Treating the network as the AI security programme: no logging, no spend limits, no prompt-data rules, because "it is private".
Framework mapping
| Control area | NIST SP 800-53 Rev 5 | NIST CSF 2.0 | ATT&CK v20 technique addressed |
|---|---|---|---|
| Private endpoints, disabled public access, perimeters | SC-7 Boundary Protection, AC-4 Information Flow Enforcement | PR.IR-01 | T1567 Exfiltration Over Web Service |
| Keyless, identity-based authentication | IA-2, IA-5 Authenticator Management | PR.AA-01, PR.AA-03 | T1552.001 Credentials In Files, T1078.004 Cloud Accounts |
| Private DNS and resolver controls | SC-20, SC-21 | PR.IR-01 | Not technique specific |
| TLS to the endpoint | SC-8 Transmission Confidentiality and Integrity | PR.DS-02 | T1557 Adversary-in-the-Middle |
| Invocation and network monitoring | SI-4 System Monitoring | DE.CM-01 | Supports detection of all of the above |
Practitioner checklist
- Classify the data each model workload sends and record the chosen exposure tier.
- Disable key-based access for production model accounts and move callers to managed or federated identity.
- For confidential workloads, deploy private endpoints for every API family the workload calls, not only the runtime API.
- Disable public network access and confirm in staging that nothing silently fell back to the public path.
- Configure every private DNS zone the resource exposes and forward it from on-premises and VPN resolvers.
- Restrict endpoint policies or perimeters to your own organisation's accounts and principals.
- Block workload egress to unapproved model provider domains; route third-party model APIs through a gateway that holds the key.
- Alert on changes to network access, key settings and perimeter membership, and on first-time model use.
- Set spend limits and per-identity quotas, whatever the network path.
Standards and references
- AWS: Use interface VPC endpoints (AWS PrivateLink) with Amazon Bedrock
- AWS: Amazon Bedrock API keys
- AWS: IAM global condition context keys
- Microsoft Learn: Configure virtual networks for Foundry Tools
- Microsoft Learn: Disable local authentication in Foundry Tools
- Microsoft Learn: Azure Private Endpoint private DNS zone values
- Google Cloud: VPC Service Controls with Agent Platform
- OWASP: LLM10:2025 Unbounded Consumption
Related NHI Mgmt Group resources: Amazon Bedrock Security Guide · Azure OpenAI and Azure AI Foundry Security Guide · Google Vertex AI Security Guide · T1682 Query Public AI Services Defensive Guide · Azure Virtual Network Security Guide