A LiteLLM Proxy holds the API keys for every model provider an organisation uses, so whoever obtains its master key, or abuses an unpatched release, inherits the organisation's entire AI spend and every credential stored behind it. Securing it comes down to customer-side settings: unique secrets, scoped keys, a private network and current, pinned releases. This guide covers how authentication works in LiteLLM Proxy, the master key and salt key, virtual keys and roles, MCP server credentials, logging and audit, a hardening baseline and the supply chain controls specific to running LiteLLM. It is for platform, AI engineering and security teams who self-host the proxy. The architectural case for AI gateways in general is in the AI Gateway and LLM Gateway Guide; this page goes deep on one implementation.
This is an independent guide from NHI Mgmt Group, based on LiteLLM's public documentation as of November 2026. It is not sponsored or reviewed by LiteLLM. Product features change, so confirm settings against LiteLLM's current documentation.
Key takeaways
- The master key is a root credential. It is the Proxy Admin key and the Admin UI password if UI_PASSWORD is unset; never use the documented example value.
- Set LITELLM_SALT_KEY before adding models and never change it. It encrypts the provider credentials stored in the database; with it set, the master key becomes a pure authentication secret that can be swapped safely.
- Give every caller its own virtual key. Scope keys with models, allowed_routes, max_budget, duration and rate limits, and block or regenerate them individually.
- Grant MCP access deliberately. Assign MCP servers per key or team, block MCP test endpoints at the edge unless used, and apply LiteLLM security updates promptly.
- Treat the proxy as a supply chain dependency with secrets inside. Pin exact versions and image digests, and rotate everything it held if you ever ran an unverified build.
How identity and access work in LiteLLM Proxy
LiteLLM Proxy sits between applications and model providers. Callers authenticate to the proxy with a LiteLLM key; the proxy then calls OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Vertex AI or other providers with the provider credentials it stores. That makes three distinct credential layers, each with different owners and failure modes.
| Credential | Set by | What it does | If it leaks |
|---|---|---|---|
Master key (LITELLM_MASTER_KEY or general_settings.master_key, must start with sk-) | Operator | Proxy Admin on every API; Admin UI login when UI_PASSWORD is unset | Full control: create keys, read spend, add pass-through routes, change models |
Salt key (LITELLM_SALT_KEY) | Operator | Encrypts provider credentials, environment variables and MCP credentials stored in the database | With a database dump, stored provider keys can be decrypted |
Virtual keys (/key/generate) | Admins, team admins or internal users | Per-caller access with budgets, models and route limits; stored as hashes | Spend and data access within that key's scope until blocked |
| Provider credentials | Config file, environment, database or secret manager | Upstream API keys or cloud roles used by the proxy | Direct provider abuse, bypassing the proxy (LLMjacking) |
| Upstream MCP credentials | Config or per-user variables | API keys, bearer tokens, OAuth clients or SigV4 for tool servers | Access to the tools and data behind each MCP server |
Database URL (DATABASE_URL, Postgres) | Operator | Holds keys, teams, spend logs and encrypted credentials | Read access to spend logs and encrypted secrets; write access to keys |
- Database required for real access control: virtual keys, teams, budgets and rotation need a Postgres database. A proxy running with only a master key has one shared identity for every caller.
- Virtual keys are hashed, not encrypted: the master key rotation documentation states this, which is why virtual keys survive a master key change. Spend logs record the SHA-256 hash (
user_api_key_hash), not the key. - Headers: LLM routes use
Authorization: Bearerwith a LiteLLM key; the MCP endpoint usesx-litellm-api-key; upstream MCP credentials are passed per server asx-mcp-{server_alias}-{header_name}. Be careful withextra_headers, which can forward a client's Authorization header to an MCP server if you list it.
For provider key abuse patterns and detection, see the LLMjacking Guide.
Admin roles and privileged access
LiteLLM separates global proxy roles from team and organisation roles. Team and organisation administration requires an Enterprise licence; on the open source build most organisations rely on proxy_admin plus internal_user.
| Role | Can do | Cannot do | Assign to |
|---|---|---|---|
proxy_admin | Everything: organisations, teams, users, any key, all spend, budgets, models | Nothing is out of scope | Two or three platform operators |
proxy_admin_viewer | Sign in, view all keys and spend | Create or delete keys, add users, change settings | Finance, audit and security reviewers |
org_admin (Enterprise) | Create teams and users in its organisation, create keys for its users, view organisation spend | Change organisation budgets, rate limits or allowed models | Business unit leads |
team_admin (Enterprise) | Manage members, member budgets and keys within editable fields enabled by a proxy admin | Create teams; raise max_budget unless allowed; add global models | Application owners |
internal_user | Create keys where team permissions allow, delete its own keys, view own spend | Add users; see others' items | Developers |
Team member (user) | View key information and health; further key routes per team member permissions | Act outside its team | Default for team members |
- Stop using environment credential login: by default the Admin UI at
/uiacceptsUI_USERNAME(defaultadmin) andUI_PASSWORD, or the master key, and anyone signing in that way is a proxy admin. Create a namedproxy_admindatabase user, then setdisable_env_credential_login: true. Keep the/fallback/loginbreak-glass account documented and vaulted. - Use SSO for people: configure SSO for the Admin UI and set
disable_password_login_when_sso_enabledonce it works, so administrators inherit the IdP's MFA and leaver process. - Login rate limiting: the UI blocks a source address for five minutes after more than 10 failed sign-ins in 60 seconds. Behind a load balancer set
trusted_proxy_rangesso the real client address is used, and do not setLITELLM_DISABLE_LOGIN_RATE_LIMIT. - No UI where none is needed: set
DISABLE_ADMIN_UI="True"on data-plane instances that only serve API traffic. - Least privilege for low-privilege keys matters: give internal-user keys only the routes they need and keep the proxy on a current release, since role checks are enforced by the version you run.
Integrations, API tokens and service accounts
Master key and salt key handling
- Generate a long random master key (prefix
sk-) and a separate long random salt key. Store both in a secret manager, inject them as environment variables, and never commit them toconfig.yaml. - Set
LITELLM_SALT_KEYbefore the first model is added to the database. The production documentation warns not to change it afterwards, because stored credentials become unreadable. - If a proxy is already running without a salt key, the master key is also the encryption key. The documented path is to set
LITELLM_SALT_KEYto the current master key value first, so later master key rotations become simple swaps. - To rotate the master key with a salt key set: replace it in the environment and restart all instances. Without a salt key: back up the database, call
POST /key/regeneratewithnew_master_key, update every place the key is stored, and restart. Either way, the Admin UI logs everyone out.
Virtual key design
- One key per workload: issue a key per application, agent or pipeline with
key_aliasnaming the owner (enforce akey_alias_pattern),team_idandmetadatafor ownership. - Scope tightly:
modelslimits which models a key can call;allowed_routeslimits which proxy routes it can reach (keep LLM keys off management and MCP routes unless needed);max_budgetwithbudget_durationcaps spend;tpm_limit,rpm_limitandmax_parallel_requestscap throughput. - Expire and rotate: set
durationso keys expire. Regeneration with agrace_periodand scheduled rotation (auto_rotate,rotation_interval,LITELLM_KEY_ROTATION_ENABLED) are Enterprise features; on the open source build, rotate by issuing a new key and blocking the old one. - Revoke fast:
POST /key/blockdisables a key immediately and/key/unblockrestores it.
Provider and MCP credentials
- Secret manager integration: LiteLLM can read provider credentials from AWS Secrets Manager, AWS KMS, Azure Key Vault, Google Secret Manager, Google KMS, HashiCorp Vault and CyberArk Conjur through
general_settings.key_management_system. Inconfig.yaml, reference secrets asos.environ/VAR_NAMErather than literal values. - Prefer workload identity for cloud providers: where the proxy runs in the same cloud as Bedrock or Vertex AI, use the platform's role-based identity instead of static keys, and restrict the role to model invocation.
- MCP servers: avoid
allow_all_keys, which exposes a server to every key; grant MCP servers per key, team or organisation. Prefer OAuth 2.0 client credentials or token exchange over staticauth_valuestrings. LeaveLITELLM_ENABLE_MCP_STDIOunset unless you need local stdio servers, because stdio means the proxy spawns processes. See the MCP Gateway and MCP Server Governance Guide. - Log content is a credential store too: tokens embedded in logged request or response bodies end up in the logs, so restrict who can read them and keep log stores off the internet.
Logging and detection
LiteLLM produces three useful record types: spend logs per request (with key hash, model, tokens and cost), application logs (set LITELLM_LOG="ERROR" and json_logs: true in production, with set_verbose: False), and audit logs of changes to keys, teams, users and models (store_audit_logs: true, Enterprise, viewable under Logs > Audit Logs and exportable to object storage). Decide deliberately whether to log prompt and response bodies: they help investigations but can contain personal data and secrets.
| Signal | Source | What it can indicate | Mapping |
|---|---|---|---|
| Requests with an unknown or malformed Authorization header, SQL error spikes on key verification | Proxy and reverse proxy logs | Authentication probing | T1190 Exploit Public-Facing Application |
Calls to /mcp/ from unexpected clients, or with very short bearer tokens | Reverse proxy logs | Authentication bypass attempts | T1190 |
Calls to /mcp-rest/test/connection or /mcp-rest/test/tools/list | Reverse proxy logs | Command execution attempts | T1059 Command and Scripting Interpreter |
| New keys, users, models or pass-through routes created outside change control | Audit logs | Persistence by someone holding admin access | T1098 Account Manipulation |
| Spend spike per key, new model families, off-hours bursts | Spend logs, budget alerts | Stolen virtual key or LLMjacking | T1496 Resource Hijacking; OWASP LLM10:2025 Unbounded Consumption |
| Proxy process spawning shells or miners; outbound traffic to non-provider hosts | EDR, container runtime, egress logs | Host compromise after exploitation | T1059, T1496 |
| Provider console showing usage from addresses other than the proxy | Provider logs | Provider key copied off the proxy | T1552 Unsecured Credentials |
Metrics that show the controls are working:
- Shared key ratio: share of traffic made with the master key or shared keys rather than per-workload virtual keys (target near zero).
- Key hygiene: percentage of virtual keys with an owner alias, a budget and an expiry.
- Version currency: days behind the latest patched release, and days to apply each LiteLLM security update.
- Exposure: count of proxy instances, Admin UIs and MCP routes reachable from the internet (target zero without an identity-aware proxy).
- Spend anomaly time to block: minutes from a budget or anomaly alert to the key being blocked.
Hardening baseline
- Apply security updates promptly: watch the LiteLLM security advisories and move to fixed releases quickly. NHI Mgmt Group recommends testing each release in staging against your config, then rolling it out by pinned digest.
- Keep it private: serve the proxy on a private network, behind an identity-aware proxy or API gateway, with TLS. Block
/mcp/and/mcp-rest/test/*at the edge unless used. Do not enableallow_requests_on_db_unavailableoutside a VPC that the internet cannot reach. - No default secrets: unique master key and salt key from a secret manager; check every deployment for the documentation example key.
- Run as non-root: set
LITELLM_NON_ROOT="true", and in KubernetesrunAsNonRoot: true, a fixed non-root UID,readOnlyRootFilesystem: truewith only the writable paths LiteLLM needs, and all capabilities dropped. - Pin what you run: use an exact version tag, never
:latest, and preferably the image digest; for pip installs, pin with hashes. This means you only run builds you have reviewed. - Restrict the proxy's own identity: its cloud role should only invoke models; block instance metadata access from workloads that do not need it; egress only to configured providers and MCP servers.
- Reduce error and body logging:
disable_error_logs: trueand a deliberate decision on request and response body logging. - Disable unused features:
DISABLE_ADMIN_UIon data-plane pods, stdio MCP off, no pass-through endpoints that are not reviewed.
Rolling it out
- Inventory: find every LiteLLM deployment, including developer laptops and notebooks running the proxy, with version and exposure. See the Unsanctioned AI Agents and MCP Servers Guide.
- Emergency items first: patch, remove internet exposure, replace default master keys, rotate provider keys on any instance that was exposed or vulnerable.
- Introduce virtual keys in a pilot team, with budgets in monitoring mode (soft budgets and alerts) before hard limits, so you learn real usage.
- Migrate callers off the master key application by application; keep the old shared key for a defined cut-over window, then block it.
- Enforce: hard budgets, disabled environment login, SSO only. Roll back by unblocking keys or raising budgets, not by re-sharing the master key.
Decision guidance
- Open source or Enterprise: the open source build covers master key, salt key, virtual keys, budgets and blocking. Audit logs, team and organisation admin roles, scheduled key rotation and storing virtual keys in a secret manager require an Enterprise licence. Regulated or multi-team estates usually need the audit logs; a single team may manage with change-controlled config and reverse proxy logs.
- Database-stored or config-file models:
store_model_in_dblets admins change models at run time and syncs pods, but puts encrypted credentials in Postgres and makes the salt key critical. Config-file models withos.environ/references keep secrets in the secret manager and changes in version control. - Self-hosted or managed gateway: self-hosting puts patch speed and supply chain verification on you, which 2026 showed is a real operational cost.
Common misconfigurations
- Documentation example key in production: a gateway reachable from the internet that accepts the example key is fully open; replace it before exposing any endpoint.
- Everyone shares the master key: no attribution, no per-caller budget and no way to revoke one caller without breaking all of them.
- Changing the salt key: rotating
LITELLM_SALT_KEYafter models are stored makes the stored provider credentials unreadable and causes an outage; it is not a rotation mechanism. - Floating image tags: deployments that pull
latestormain-stablerun whatever was published last, without review. Pin versions, and rotate every credential the proxy held if an unreviewed build ran. - Internal deployment exposed to the internet: logs and admin routes on an internal deployment become reachable from outside; keep internal instances on private networks.
- Provider keys usable from anywhere: if a provider key works outside the proxy, a stolen key bypasses every budget. Use provider-side network restrictions or workload identity where available.
Framework mapping
| Control | OWASP Top 10 for LLM Applications 2025 | NIST SP 800-53 Rev. 5 | NIST CSF 2.0 | ATT&CK / ATLAS |
|---|---|---|---|---|
| Master key, salt key and provider keys in a secret manager | LLM02 Sensitive Information Disclosure | IA-5, SC-28 | PR.AA-01, PR.DS-01 | T1552, AML.T0055 Unsecured Credentials |
| Per-workload virtual keys, roles, allowed_routes | LLM06 Excessive Agency | AC-6, AC-2 | PR.AA-05 | T1078 Valid Accounts |
| Budgets and rate limits per key | LLM10 Unbounded Consumption | SC-5 | PR.IR-04 | T1496 Resource Hijacking |
| Version pinning, digest verification, patching | LLM03 Supply Chain | SI-2, SR-3, SI-7 | ID.RA-01, GV.SC-07 | T1195.002 Compromise Software Supply Chain |
| Private exposure, MCP route blocking | LLM06 Excessive Agency | SC-7, CM-7 | PR.IR-01 | T1190 |
| Spend, audit and proxy logs to SIEM | LLM10 Unbounded Consumption | AU-2, AU-12, SI-4 | PR.PS-04, DE.CM-09 | T1098 |
Practitioner checklist
- Inventory every LiteLLM Proxy and its version; apply the latest LiteLLM security updates.
- Replace any example or shared master key with a unique secret from a secret manager; set LITELLM_SALT_KEY once and never change it.
- Issue one virtual key per workload with owner alias, models, allowed_routes, budget, rate limits and expiry.
- Create named proxy_admin users, enable SSO, set disable_env_credential_login, and vault the break-glass account.
- Keep the proxy and Admin UI off the internet; block /mcp/ and /mcp-rest/test/* where unused; leave MCP stdio disabled.
- Avoid allow_all_keys on MCP servers and prefer OAuth or token exchange to static upstream tokens.
- Run as non-root with a read-only root file system; restrict the proxy's cloud role and egress.
- Pin image digests or hashed pip requirements; never deploy floating tags.
- Forward spend, audit and proxy logs to the SIEM; alert on spend spikes, new admin objects and MCP test endpoint calls.
- Keep a rotation runbook covering master key, provider keys, MCP credentials and database credentials.
Standards and references
- LiteLLM: Virtual Keys
- LiteLLM: Best Practices for Production
- LiteLLM: Rotating Master Key
- LiteLLM: Role-based Access Controls
- LiteLLM: MCP Overview
- BerriAI: LiteLLM Security Advisories
- OWASP: Top 10 for LLM Applications 2025
Related NHI Mgmt Group resources: AI Gateway and LLM Gateway Guide · LLMjacking Guide · MCP Security Guide · Software Supply Chain Security Guide