Join our Newsletter — 33% off our NHI Course
Home› Guides› Private Endpoints vs Public Endpoints for AI Services:…
Guide AI Platforms

Private Endpoints vs Public Endpoints for AI Services: Choosing Network Exposure for Model APIs

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 14 November 2026 17 min read
On this page

A hosted model API reached over its public endpoint will answer anyone on the internet who holds a valid key or token, so a leaked credential becomes free inference and a data path out of your organisation. Private endpoints move that traffic onto a private IP inside your network, but they only change the route: identity, the public endpoint itself, DNS and the prompt data still need their own controls. This guide compares public and private exposure for managed AI services, with a threat model for each, the DNS and egress mistakes that quietly undo a private design, and a decision method based on data sensitivity. It is written for cloud architects, platform and AI engineering teams, and security reviewers approving model API access. Platforms are listed alphabetically: Amazon Bedrock, Azure OpenAI in Azure AI Foundry, and Google Cloud Vertex AI (documented by Google in 2026 under the Agent Platform name).

Key takeaways

  • The real risk on a public endpoint is the credential, not the network. Stolen AI keys are tested within minutes and resold, as the LLMjacking and Storm-2139 cases show; network restriction limits where a stolen key works, identity-based authentication removes the key.
  • A private endpoint is a path, not a lock. Google states that Private Service Connect and Private Google Access keep public reachability unless a VPC Service Controls perimeter is added, and Azure documents a separate firewall setting to block the public endpoint.
  • DNS decides which path is used. If private DNS is missing or wrong, clients silently fall back to the public endpoint, or break and someone re-opens public access to fix them.
  • Private endpoints do not protect prompt data from what the model, its logs or its tools do with it. Content controls, logging settings, data residency and agent tool permissions sit above the network layer.
  • Choose by data sensitivity and caller location. Public endpoint plus strong identity suits low-sensitivity, internet-hosted callers; private endpoint plus disabled public access plus keyless identity suits regulated data and anything called from inside your network.

What changes between a public and a private model endpoint

Every managed model service publishes a regional HTTPS endpoint on a public DNS name. With a public endpoint, clients reach it over the internet or through a NAT gateway, and the service decides whether to answer based on the credential and, optionally, an IP allow list. With a private endpoint, the provider places a network interface with a private IP from your address range in your virtual network, and private DNS points the service name at that IP. Traffic then stays on the provider backbone. The Cloud Private Connectivity Guide explains the underlying PrivateLink, Private Link and Private Service Connect models and the producer and consumer roles; this page applies them to model APIs.

DimensionPublic endpointPrivate endpoint
Who can reach the listenerAny internet host; the service authenticates every requestHosts with a route to the private IP; public listener stays up unless you disable it
Where a stolen key worksAnywhere, unless IP rules or a perimeter restrict itOnly from inside the allowed network, once public access is disabled
Egress needed by callersInternet or NAT egress to the provider domainNo internet egress for model calls; the endpoint is local
DNSPublic resolution, nothing to managePrivate zones or resolver rules for each service name in use
Identity-based network conditionsSource IP conditions on public rangesEndpoint ID, VPC or organisation conditions on the request
Typical callersSaaS apps, developer laptops, partner systems, prototypesWorkloads, agents and data pipelines in your cloud or data centre
Cost and effortNone beyond usagePer-endpoint and data charges, DNS work, one endpoint per API family

Threat model for public endpoints

The public endpoint threat model is dominated by credentials. An AI service key or a cloud access key is a bearer secret: whoever presents it is the caller. The LLMjacking campaigns tested harvested keys with quiet requests, checked whether prompt logging was on, enabled models the victim had never used and resold access through reverse proxies, with one exposed Amazon Bedrock key reported in about 75,000 invocations over two days. In the Azure OpenAI Storm-2139 abuse, scraped customer keys were used from outside the victim organisations to generate harmful content at their expense. In both, the lesson is customer-side: a key that works from anywhere is worth stealing.

  • Key theft usable from anywhere: keys leak through code, notebooks, CI logs, container images and client-side apps (ATT&CK T1552.001 Credentials In Files); a public endpoint accepts them from any network. See the AI API key abuse guide.
  • Cloud account takeover: stolen cloud credentials (T1078.004 Cloud Accounts) can call the model API and also change model access, logging and quotas.
  • Cost and capacity abuse: OWASP lists this as LLM10:2025 Unbounded Consumption; without quotas, spend limits and anomaly alerts a stolen key runs up bills fast. See How to set spend limits on LLM API keys.
  • Data exfiltration through the API: an insider or malware with a valid credential can send internal documents as prompts to a model account outside your control (T1567 Exfiltration Over Web Service). The public endpoint makes any account at the same provider reachable, including a personal one.
  • Interception risk: low. Traffic is TLS to the provider's public certificate; the realistic exposure is TLS inspection proxies that log prompt bodies.

Threat model for private endpoints

A private endpoint shrinks the set of networks that can use a credential, but adds its own risks, mostly from configuration that looks finished and is not.

  • Public endpoint left open: the most common gap. The private path works, so nobody notices that the public listener still accepts the same keys from the internet.
  • Endpoint as an exfiltration path to other accounts: an interface endpoint for a model API can reach any account of that service in the region unless an endpoint policy or perimeter limits it. Amazon Bedrock documents that the default endpoint policy allows full access through the endpoint; narrow it to your principals and pair it with organisation conditions.
  • Over-broad network trust: everything that can route to the endpoint subnet, including a compromised build agent or a peered partner network, can now try credentials against it. Segment who can reach the endpoint subnet.
  • Trusted service exceptions: Azure lets trusted services such as Azure Machine Learning and Azure AI Search bypass network rules using managed identity. These paths are legitimate but must be reviewed like any allow rule.
  • Producer-side internet egress: agents and pipelines hosted by the platform may need outbound internet for packages or tools. Google documents that inside a VPC Service Controls perimeter the producer's internet access is blocked and recommends a proxy in your own subnet with Cloud NAT; that proxy becomes a control point you must own.

DNS and egress pitfalls

Private endpoints depend on the client resolving the normal service name to a private IP. Each platform does this differently, and each has a failure mode that sends traffic back to the public path.

PlatformPrivate connectivity objectDNS behaviour from the docsPitfall to test for
Amazon BedrockInterface VPC endpoints: bedrock, bedrock-runtime, bedrock-agent, bedrock-agent-runtime, bedrock-mantle (FIPS variants in some Regions)With Enable Private DNS Name, default Regional names such as bedrock-runtime.region.amazonaws.com resolve to the endpoint; without it, clients must set the vpce endpoint URLRuntime endpoint created but agent runtime or control plane calls still leave through NAT
Azure OpenAI in Azure AI FoundryPrivate endpoint on the account subresourceZones privatelink.openai.azure.com, privatelink.cognitiveservices.azure.com and privatelink.services.ai.azure.com; a Foundry resource can expose all threeOnly one zone configured; clients calling the privatelink name directly, which Microsoft says not to do; custom DNS servers not forwarding the zone
Google Cloud Vertex AIPrivate Service Connect for Google APIs or Private Google Access, plus a VPC Service Controls perimeterCloud DNS private zones map Google API names to the restricted VIP range 199.36.153.4 to 199.36.153.7Workbench kernels needing private.googleapis.com rather than restricted.googleapis.com; endpoints created before the project joined the perimeter
  • Silent public fallback: if the client's resolver does not see the private zone (on-premises hosts, a second VNet, a developer VPN), the name resolves publicly and the call succeeds over the internet while public access is still enabled. Disable public access first in test, so fallback fails loudly instead.
  • NXDOMAIN for other resources: Microsoft warns that a private zone covering a resource type returns NXDOMAIN for public resources of that type with no private endpoint, and recommends the Fallback to Internet option on the virtual network link rather than manual A records.
  • DNS is not access control: Microsoft notes that the privatelink CNAME chain resolves from anywhere; resolution proves a resource exists, it grants nothing. Treat the names as discoverable.
  • Egress still open to the provider domain: if workloads can reach the public provider domain through NAT or a proxy, a private endpoint does nothing for data sent to someone else's model account. Block the provider's public API names at the egress layer for subnets that should use the endpoint. See Cloud Egress Filtering and the Agent Egress Control and Network Isolation Guide.

What private endpoints do not fix

Network placement answers one question: from where can this API be called. Several of the most important risks to model APIs sit elsewhere.

  • Identity: inside the network, any holder of a key still gets full access to the account. Over-privileged workload roles and shared keys remain the main path to misuse.
  • Prompt and response data: what you send is processed and may be logged by your own logging configuration, gateways and observability tools. OWASP LLM02:2025 Sensitive Information Disclosure applies whatever the network path.
  • Prompt injection and agent tool use: an agent on a private network that reads untrusted content can still be steered into calling tools with its own permissions. See the Prompt Injection Guide.
  • Cost abuse from inside: a compromised internal workload can exhaust quota as easily over a private IP.
  • Feature trade-offs: perimeters change behaviour. Google lists, for example, that URL context live fetching is turned off and request and response logging is unavailable for Vertex AI projects inside VPC Service Controls; plan detection around those conditions.

Key-based versus identity-based authentication

The decision that matters as much as the network path is whether callers present a static key or a short-lived identity token. Each platform supports moving production callers off static keys.

PlatformStatic key optionIdentity-based optionControl to enforce it
Amazon BedrockLong-term API keys, tied to an IAM user, which AWS recommends only for explorationIAM roles with temporary credentials; short-term API keys valid up to 12 hours and inheriting the generating principal's permissionsDeny bedrock:CallWithBearerToken, or limit long-term keys with iam:ServiceSpecificCredentialAgeDays; target key types with bedrock:bearerTokenType
Azure OpenAI in Azure AI FoundryResource keys (local authentication)Microsoft Entra ID tokens with managed identities for workloadsSet disableLocalAuth to true; built-in Azure Policy "Azure AI Services resources should have key access disabled (disable local authentication)"
Google Cloud Vertex AIService account keys for workloads outside Google CloudAttached service accounts and Workload Identity Federation; IAM is required on every API callOrganisation policy restricting service account key creation, plus VPC Service Controls access levels

Two details from the vendor documentation affect rollout. Microsoft states that after disabling local authentication the gateway can keep accepting old keys until its cache refreshes, typically minutes and up to several hours, so verify with a data-plane call that returns HTTP 401 rather than assuming an instant cut-off. AWS notes that Bedrock API keys travel as bearer tokens and are not written to CloudTrail, while the calls themselves are logged. See Workload Identity Federation vs Static Cloud Keys and Service Principals vs Managed Identities for the identity side.

On AWS, network and identity can be combined in one policy: the aws:SourceVpce, aws:SourceVpc and aws:VpceOrgID condition keys are present only when a request arrives through a VPC endpoint, and AWS describes aws:VpceOrgID as the most scalable network perimeter key. Note that aws:SourceIp is absent for requests through a VPC endpoint, so IP-based conditions and endpoint-based conditions need separate statements.

Decision by data sensitivity

Classify the data that will appear in prompts, retrieved context and responses, then pick the tier. Caller location is the second axis: a SaaS product calling your model account from the internet cannot use your private endpoint at all.

Data tierExample useNetwork exposureAuthenticationAdditional controls
Public or test dataPrototypes, marketing copy, public documentation searchPublic endpoint, IP rules where callers are fixedShort-lived keys or identity tokens; no long-term keys in codeSpend limits, key scanning, per-key usage alerts
InternalEmployee assistants over internal wikis, coding helpPrivate endpoint for in-network callers, or public endpoint restricted to corporate egress IPsIdentity-based only; keys disabledAI gateway for logging and quotas; egress block to other model accounts
Confidential or personal dataCustomer support over case data, HR, contract analysisPrivate endpoint, public access disabled, perimeter or endpoint policy limiting to your accountsManaged or federated workload identity, least-privilege model permissionsDLP on prompts, logging retention rules, data residency checks
Regulated or highly sensitiveHealth, payment, government or legally privileged dataPrivate only, perimeter enforced, no internet egress from workloadsKeyless identity, conditional access on admin rolesSeparate accounts or projects per workload, change control on network rules and model access
  • Small organisations: start with keyless authentication and spend limits on the public endpoint; that removes most of the LLMjacking risk at almost no cost. Add a private endpoint when the first confidential workload arrives.
  • Large organisations: centralise private DNS zones and endpoints in a hub, create them through policy as code, and route model traffic through an AI gateway so every team inherits logging and quotas. See the AI Gateway and LLM Gateway Guide.
  • Third-party model APIs without a private option for your cloud: treat them as public endpoints. Put an egress proxy or AI gateway in front, hold the key there rather than in applications, and allow only that gateway to reach the provider.

Worked scenario: moving a claims assistant to private access

An insurer runs a claims summarisation service in a cloud VNet that sends claim notes, which include personal and health data, to a hosted model. Today it uses a resource key from an environment variable and calls the public endpoint through NAT.

  1. Inventory callers: query logs for every source and identity that called the model account in the last 30 days. The team finds the production service, two data scientists' laptops and a forgotten test function.
  2. Move to identity: give the service a managed identity with a data-plane user role, deploy, and confirm calls now carry tokens. Move the laptops to individual accounts with Entra ID sign-in through a separate development resource.
  3. Create the private endpoint and DNS: deploy the endpoint and all three privatelink zones the resource uses, linked to the hub and forwarded from on-premises resolvers. Test resolution from every caller subnet.
  4. Disable public access in staging first: set the network rule to selected networks and private endpoints with no IP rules. Any caller that was silently using the public path fails here, in staging, not in production.
  5. Disable keys: set disableLocalAuth, wait for the gateway cache, and verify the old key returns HTTP 401. Revoke the environment variable.
  6. Close egress: block the provider's public API domain at the firewall for the workload subnet, so the service cannot send claim notes to any other model account.
  7. Roll back safely: keep the change as code. If production breaks, re-enable selected public IPs for the service's egress address temporarily, never all networks, and fix DNS before removing them again.

The result: a leaked key is useless (keys are off), a stolen token works only from the VNet, and the workload cannot reach other model accounts. What remains is application risk, such as prompt injection in claim notes, which the team handles with content controls and least-privilege tool design.

Detection and measurement

  • Log the control plane: alert on changes to public network access, network rules, endpoint policies, disableLocalAuth and perimeter membership. These are the settings an attacker or a frustrated engineer changes first.
  • Log the data plane: invocation logs by identity and source; flag first-time model access, new regions and calls from outside expected endpoints.
  • Watch DNS: resolver logs showing public addresses for model API names from workload subnets indicate fallback. See the DNS Logging and Detection Guide.
  • Watch egress: connections from workload subnets to any model provider domain other than the approved endpoint or gateway.
MetricTarget
Model accounts handling confidential data with public access disabled100%
Production model calls authenticated with keys rather than identity tokens0%
Long-term AI keys older than 90 days0, trending to none issued
Workload subnets able to reach unapproved model provider domains0
Mean time to revoke a leaked AI key after detectionUnder 1 hour

Common failure patterns

  • Private endpoint deployed, public access still on: the network diagram looks private, while stolen keys still work from the internet, which is exactly how the Storm-2139 keys were used.
  • Keys kept "just for the private path": teams disable public access but keep static keys in code, so any foothold inside the network inherits full model access.
  • Default endpoint policy: the endpoint allows calls to any account of the service, giving insiders a sanctioned-looking exfiltration route.
  • Partial DNS: one privatelink zone configured while the application also calls another hostname of the same resource, so half the traffic goes public.
  • Perimeter added after resources: Google warns that inference endpoints and agents created before the project joined a VPC Service Controls perimeter fail or remain publicly reachable; recreate them inside the perimeter.
  • Treating the network as the AI security programme: no logging, no spend limits, no prompt-data rules, because "it is private".

Framework mapping

Control areaNIST SP 800-53 Rev 5NIST CSF 2.0ATT&CK v20 technique addressed
Private endpoints, disabled public access, perimetersSC-7 Boundary Protection, AC-4 Information Flow EnforcementPR.IR-01T1567 Exfiltration Over Web Service
Keyless, identity-based authenticationIA-2, IA-5 Authenticator ManagementPR.AA-01, PR.AA-03T1552.001 Credentials In Files, T1078.004 Cloud Accounts
Private DNS and resolver controlsSC-20, SC-21PR.IR-01Not technique specific
TLS to the endpointSC-8 Transmission Confidentiality and IntegrityPR.DS-02T1557 Adversary-in-the-Middle
Invocation and network monitoringSI-4 System MonitoringDE.CM-01Supports detection of all of the above

Practitioner checklist

  • Classify the data each model workload sends and record the chosen exposure tier.
  • Disable key-based access for production model accounts and move callers to managed or federated identity.
  • For confidential workloads, deploy private endpoints for every API family the workload calls, not only the runtime API.
  • Disable public network access and confirm in staging that nothing silently fell back to the public path.
  • Configure every private DNS zone the resource exposes and forward it from on-premises and VPN resolvers.
  • Restrict endpoint policies or perimeters to your own organisation's accounts and principals.
  • Block workload egress to unapproved model provider domains; route third-party model APIs through a gateway that holds the key.
  • Alert on changes to network access, key settings and perimeter membership, and on first-time model use.
  • Set spend limits and per-identity quotas, whatever the network path.

Standards and references

Related NHI Mgmt Group resources: Amazon Bedrock Security Guide · Azure OpenAI and Azure AI Foundry Security Guide · Google Vertex AI Security Guide · T1682 Query Public AI Services Defensive Guide · Azure Virtual Network Security Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 14 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.