Unauthenticated llm proxies expose prompt leakage and abuse risk

At a glance

What this is: FireCompass found a publicly accessible LLM proxy that allowed unauthenticated access to backend AI services and exposed internal prompt logic.

Why it matters: This matters to IAM and security teams because AI proxies sit on the access-control boundary, where missing authentication, authorisation, and separation can turn model exposure into broader governance failure.

👉 Read FireCompass's analysis of the unauthenticated LLM proxy and prompt disclosure issue

Context

An LLM proxy is the access-control layer between users and the model backend. When that layer is exposed without authentication or policy enforcement, the problem is no longer just prompt quality. It becomes an application access issue, with consequences for cost, data handling, and control over system instructions. For teams running customer-facing AI features, this is a classic governance gap: the interface that should mediate access is itself the weak point.

The identity angle is real because the proxy behaves like a privileged service boundary for non-human access. If the proxy does not validate the caller, the application effectively grants model capability to anyone who can reach the endpoint. That places LLM endpoints in the same governance conversation as API keys, service accounts, and other non-human identities that need explicit scoping, monitoring, and separation of duties.

Key questions

Q: What breaks when an LLM proxy has no authentication or authorisation?

A: The proxy stops being a controlled access layer and becomes a public gateway to privileged model resources. That allows anonymous users to invoke backend AI services, probe behaviour, and potentially extract hidden instructions. The failure is architectural, because the organisation has exposed a non-human access path without the governance controls that should restrict it.

Q: Why do unauthenticated AI endpoints increase both disclosure and abuse risk?

A: Because the same missing boundary that allows model access also allows repeated probing, prompt injection attempts, and high-volume usage. Once attackers can interact with the backend freely, they can learn the system's constraints and consume resources at the same time. That makes confidentiality, integrity, and availability failures converge at one control point.

Q: How can security teams tell whether an LLM proxy is failing in practice?

A: Look for anonymous requests reaching the model, responses that reveal policy or instruction text, and traffic patterns that indicate repeated probing or runaway usage. If the proxy can be reached without a verified caller identity and the logs cannot explain who invoked it, the control is already failing.

Q: Should organisations treat AI gateways like privileged identity controls?

A: Yes. AI gateways mediate access to powerful non-human capabilities, so they need the same discipline used for sensitive service accounts and other privileged access paths. That means authentication, authorisation, logging, rate limiting, and explicit separation between user input and system instructions. Without those controls, the gateway becomes an unmanaged privilege surface.

Technical breakdown

Why unauthenticated LLM proxies are high-risk control points

An LLM proxy is not just a routing component. It often enforces authentication, rate limits, prompt filtering, model selection, and instruction separation before forwarding requests to the backend model. If those controls are missing, the proxy becomes a public execution path into a privileged AI service. Attackers do not need to break the model itself. They only need to reach the interface that already has model access and operational trust. That is why proxy exposure is a control-plane failure, not a simple application bug.

Practical implication: Treat the proxy as a privileged service and require authentication, authorisation, and abuse controls before any model call can execute.

How system prompt disclosure happens through context leakage

System prompts are the hidden instructions that shape model behaviour, constrain output, and encode operational logic. When user input is merged too directly into model context, the boundary between system, developer, and user instructions can blur. That makes prompt extraction possible through carefully crafted queries that ask the model to reveal its own context. The risk is not limited to a single secret phrase. Leaked instructions can expose workflow logic, safety assumptions, and response constraints that attackers can reuse.

Practical implication: Separate system instructions from user-controlled content and test whether prompt injection can elicit hidden context or policy text.

Why AI abuse controls must cover cost, rate, and behaviour

Unrestricted model access creates more than disclosure risk. It also creates resource abuse risk, because attackers can generate large volumes of inference requests, exhaust capacity, and inflate spend. Behavioural controls matter because a model that responds outside policy may still be technically functional while operationally unsafe. In practice, AI governance needs usage quotas, anomaly detection, response constraints, and logging that captures who invoked the model, what context was sent, and what was returned.

Practical implication: Instrument AI endpoints with quotas, monitoring, and audit trails so abuse, leakage, and unexpected model behaviour can be detected early.

Threat narrative

Attacker objective: The attacker aims to reach backend AI services without authorisation, extract hidden prompt logic, and use that knowledge to amplify abuse or bypass restrictions.

1. Entry occurred through a publicly accessible LLM proxy endpoint that accepted requests without authentication or session context.
2. Escalation followed once the attacker could invoke the backend model directly and probe how prompts were handled and constrained.
3. Impact came from unrestricted model use, potential resource abuse, and disclosure of system instructions that could help bypass future controls.

NHI Mgmt Group analysis

LLM proxies are becoming non-human identity control points. Once an AI application sits behind a proxy that can call a model, the proxy itself functions like a privileged service identity. If caller validation, session binding, and authorisation are weak, the organisation has not just exposed an endpoint. It has exposed a non-human access path that can be reused at scale. For practitioners, the control question is no longer whether the model is safe, but whether the proxy is governed like any other privileged machine identity.

Prompt disclosure is an identity and governance failure, not only an AI safety issue. Hidden instructions often encode policy, workflow, and control decisions that should never be available to unauthorised users. When those instructions are recoverable through ordinary requests, the organisation has blurred the separation between system, developer, and user context. That creates governance debt because the attack surface now includes the policy layer itself. Practitioners should treat prompt separation as part of access governance, not just content moderation.

Cost abuse is the first operational symptom of weak AI boundary control. An unauthenticated proxy can be used to drive unlimited inference, exhaust budget, and degrade availability long before a breach is obvious. That shifts AI security from a purely confidentiality problem to a resilience problem. The named concept here is LLM proxy exposure, meaning a public model gateway that lacks the controls needed to mediate privileged AI access. For teams, the lesson is to govern the gateway as infrastructure with access policy, logging, and limits.

Traditional application testing misses the control failures that matter in AI systems. This finding was not just about a missing header or a bad response code. It was about the combination of automated discovery, behavioural validation, and context probing needed to prove the proxy could be used as an attack path. That aligns with broader AI governance thinking in NIST AI RMF and OWASP agentic AI guidance. Practitioners should test the boundary between application logic and model access, not only the model output.

The market is moving toward AI attack-surface governance, not isolated model hardening. As customer-facing AI features multiply, the controlling question becomes whether organisations can inventory, authenticate, monitor, and constrain every LLM entry point. That is an identity and security architecture problem spanning application teams, cloud teams, and IAM owners. Teams that still treat model endpoints as experimental features will keep missing the real risk, which sits at the access boundary.

What this signals

LLM proxy exposure is an early warning for broader AI governance debt. As more products route user traffic through model gateways, teams will need asset inventory, access policy, and monitoring for every AI entry point, not just the model itself. The practical shift is toward treating AI gateways as privileged infrastructure with explicit control ownership and audit expectations.

The next failure pattern is likely to be a mismatch between what the proxy can do and what the organisation can explain in logs. If caller identity, prompt content, and model output are not linked in a forensic trail, teams will struggle to investigate prompt disclosure, cost abuse, or policy bypass. That is why the control conversation should now include traceability, not only detection.

For practitioners

• Inventory every exposed AI entry point Map all LLM proxies, model gateways, and API routes that can reach production models. Classify each one by authentication requirement, session binding, and whether it can reach backend services without explicit authorisation.
• Require strong caller validation at the proxy layer Enforce authentication and authorisation before any request is forwarded to a model. Bind requests to a known user, service, or workload identity, and reject anonymous access even for internal testing paths.
• Separate system instructions from user input Keep hidden prompts, policy text, and developer instructions outside user-controlled context wherever possible. Test for prompt injection and disclosure with controlled probes that attempt to reveal hidden instructions.
• Apply usage limits and abuse detection Set rate limits, quotas, and anomaly detection on AI endpoints so repeated prompt submission, unexpected request bursts, and unusual output patterns trigger review before costs or availability are affected.
• Log model access with audit-ready detail Capture who called the proxy, what identity was asserted, what model context was used, and what response was returned. Use those logs to support investigation, abuse detection, and control validation.

Key takeaways

• Unauthenticated LLM proxies turn AI access into an unmanaged non-human identity problem.
• Prompt disclosure, cost abuse, and behaviour drift all start at the same weak control boundary.
• Teams need authenticated AI gateways, separation of instructions, and audit-ready logging before production usage expands.

Key terms

• LLM Proxy Gateway: An LLM proxy gateway is an intermediary service that routes application traffic to one or more AI providers. It often centralises API keys, routing logic, logging, and policy controls, which makes it operationally useful but also highly sensitive if compromised because it can expose multiple credentials at once.
• System Prompt Disclosure: System prompt disclosure occurs when hidden instructions meant to guide model behaviour become visible to an unauthorised user. The risk is not only leakage of text, but exposure of business logic, safety rules, and operational constraints that can help attackers craft better prompts or bypass controls.
• AI Gateway: A control point that sits between AI applications and the models, tools, or data they call. In practice, it can authenticate requests, enforce policy, inspect runtime behaviour, and stop unsafe actions before they spread into connected systems.
• Prompt Injection (Agentic): An attack where malicious instructions are embedded in content that an AI agent reads — causing the agent to execute unintended actions using its own legitimate credentials. A primary vector for agent goal hijacking and identity abuse.

What's in the full article

FireCompass's full article covers the operational detail this post intentionally leaves for the source:

• Step-by-step request and response traces from the proxy validation workflow
• The exact prompt probes used to confirm system prompt disclosure behaviour
• How the assessment distinguished real security findings from expected model output
• Operational lessons for organisations building LLM-backed customer-facing services

👉 FireCompass's full post includes the request traces, disclosure evidence, and validation steps behind the finding.

Deepen your knowledge

The NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance, machine identity security, and secrets management in practical terms. It helps security and IAM practitioners build control models that apply cleanly to AI gateways and other privileged non-human access paths.