AWS Lambda environment variables are the easiest place to put a database password or API key, and anyone who can read the function's configuration can usually read the secret. AWS encrypts them at rest, but with the default key any principal allowed to call GetFunctionConfiguration sees the plaintext, and every published version keeps its own frozen copy. This runbook shows how to inventory environment variables across every function, version and Region, identify which values are secrets, move them to AWS Secrets Manager with the Parameters and Secrets Lambda Extension or the SDK, and clean up old versions so the secret is really gone. It is for cloud security and platform engineers and for serverless teams. For the general case against environment variables as a secret store, see Environment Variables and .env Files.
This is an independent guide from NHI Mgmt Group, based on AWS's public documentation as of November 2026. It is not sponsored or reviewed by AWS. Product features change, so confirm settings against AWS's current documentation.
Key takeaways
- AWS itself says not to. The Lambda guide recommends Secrets Manager instead of environment variables for database credentials, API keys and authorisation tokens.
- Published versions freeze the secret. Environment variables are locked into each version when it is published, so removing a secret from
$LATESTleaves it in every older version until those are deleted. - Read access is the exposure. With the default AWS managed key no KMS permission is needed to view variables; only a customer managed key lets you gate viewing with
kms:Decrypt. - Discovery is read-only and sensitive.
list-functions --function-version ALLandget-function-configurationreturn the values; CloudTrail omits them, but your export file does not, so handle it as a secret. - Rotate before you clean. A secret that sat in configuration readable by many people is exposed; issue a new one at the source first, then move and delete.
When you need this
- Secret sprawl reviews: building an inventory of where credentials live, as required by NIST SP 800-53 IA-5 Authenticator Management.
- After a developer or CI compromise: any principal with Lambda read access, including CI roles, monitoring tools and contractors, could have copied every variable. The Codecov breach showed how routinely attackers harvest environment variables once they have a foothold.
- Before broadening read access: granting a team, a posture tool or an AI agent read-only access to Lambda also grants it every plaintext variable.
- Migration to Secrets Manager: the inventory is the work list for the move. See the Secrets manager migration playbook.
How Lambda stores and exposes environment variables
| Property | What AWS documents | Consequence for secrets |
|---|---|---|
| Encryption at rest | Always on with a KMS key; by default an AWS managed key at no charge; optionally a customer managed key | Protects storage, not access through the API |
| Who can view values | No KMS permission is needed with the default key; with a customer managed key, viewing needs kms:Decrypt | Default setup: Lambda read access equals secret access |
| Encryption helpers | Optional client-side encryption in transit with a customer managed key; the function decrypts with kms:Decrypt | Hides values in the console but adds code and key management; Secrets Manager is usually simpler |
| Versions | Variables are set on the unpublished version and locked into each published version | Old versions keep old secrets indefinitely |
| Size and names | 4 KB total; reserved names such as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN are set by the runtime from the execution role | Those reserved values are temporary role credentials, not findings; anything else that looks like a key is |
| Audit logging | The Environment structure and Variables map are omitted from CloudTrail logs | You can see who read configuration, not what they saw |
Code running in the function, its layers and any extension can also read the process environment, and runtime errors or debug logging that dump the environment send values to CloudWatch Logs. The AWS Lambda Security Guide covers the wider function threat model.
Before you start
- Permissions: a read-only audit role with
lambda:ListFunctionsandlambda:GetFunctionConfigurationin each account, pluskms:Decrypton customer managed keys if you need values from functions that use one. Remediation needslambda:UpdateFunctionConfiguration,lambda:PublishVersion,lambda:UpdateAlias,lambda:DeleteFunctionand Secrets Manager rights, held by a separate role. - Handle output as a secret: run discovery on a controlled host, write output to an encrypted location, never to a ticket or chat, and delete raw exports when the review ends. Where possible, report key names and a hash or masked prefix rather than full values.
- Scope: every enabled Region in every account. Lambda@Edge replicas are listed with
--master-region. - Owners: agree who rotates each kind of secret at its source (database, SaaS provider, internal API) before you start, because rotation is the step that takes longest.
Step-by-step
Steps 1 to 4 are read-only. Steps 5 and 6 change functions and are reversible through aliases. Step 7 deletes old versions and cannot be undone.
- List every function and version in each Region.
aws lambda list-functions --function-version ALL --region <region> \ --query "Functions[].[FunctionArn,Version,KMSKeyArn]" --output textALLincludes every published version as well as$LATEST. Lambda returns up to 50 functions per call and the CLI paginates for you. - List variable names first, values only where needed. AWS's own example returns only the keys:
Key names find most secrets on their own. Flag names containingaws lambda get-function-configuration --function-name <name> --qualifier <version> \ --query "{Version: Version, Role: Role, KMSKeyArn: KMSKeyArn, EnvVarKeys: keys(Environment.Variables || \`{}\`)}"PASSWORD,PASSWD,SECRET,TOKEN,API_KEY,APIKEY,PRIVATE_KEY,CLIENT_SECRET,CONNECTION_STRING,DATABASE_URL,DSNandWEBHOOK. - Inspect values for the rest. For variables whose names are not conclusive, retrieve
Environment.Variablesand look for value patterns: strings beginningAKIA(long-term AWS access key IDs), PEM headers such as-----BEGIN, URLs with a user and password before the host, JSON Web Tokens, provider token prefixes your secret scanner recognises, and long high-entropy strings. Feed the export to the same scanner you use for repositories so the rules match. See Secret Scanner Rule Coverage Comparison Guide. - Classify and rank. For each finding record function, version, alias pointing at it, Region, key name, secret type, owner, the encryption key (AWS managed or customer managed) and whether the value also appears in infrastructure as code or deployment pipelines, because those are a second copy. Rank cloud and database credentials with write access first, then third-party API tokens, then low-value configuration such as public keys.
- Rotate at the source. Issue a new credential in the database, provider or IAM, and store it in Secrets Manager (or Parameter Store
SecureStringfor simple values). Grant the function's execution rolesecretsmanager:GetSecretValueon that secret ARN only. Do not revoke the old credential yet. - Change the function to read the secret at runtime. Either call Secrets Manager through the SDK, caching the value, or add the AWS layer
AWS-Parameters-and-Secrets-Lambda-Extension, which serves a local endpoint on port 2773 at/secretsmanager/get?secretId=, requires theX-Aws-Parameters-Secrets-Tokenheader with the session token, and caches values for up to 300 seconds by default (SECRETS_MANAGER_TTL). Then remove the variable.update-function-configuration --environmentreplaces the wholeVariablesmap, so pass every variable you want to keep:
Storing the secret's ARN or name in a variable is fine; it is a reference, not the secret. Revoke the old credential at the source once the new version is healthy.aws lambda update-function-configuration --function-name <name> \ --environment "Variables={LOG_LEVEL=INFO,DB_SECRET_ARN=<secret-arn>}" aws lambda publish-version --function-name <name> aws lambda update-alias --function-name <name> --name <alias> --function-version <new-version> - Delete versions that still hold the secret. Once no alias, event source mapping or caller refers to an old version, delete it:
Without the qualifier this command deletes the entire function, so check the command before you run it. If old versions must be kept for audit, the rotation in step 5 is what makes them harmless.aws lambda delete-function --function-name <name> --qualifier <old-version>
Verify it worked
- Re-scan: steps 1 to 3 return no flagged keys or values for the remediated functions in any version.
- Old credential dead: the source system shows the old credential revoked, and no successful use of it after the cut-over date.
- Runtime reads: CloudTrail shows
GetSecretValuecalls from the execution role, at a rate consistent with caching rather than one per invocation. - Health: function error rate and duration in CloudWatch unchanged after the alias moved; a cold-start increase from the extension is expected and small.
Rollback and common errors
| Situation | Fix |
|---|---|
| New version fails | update-alias back to the previous version; it still has the old variable, which works only while the old credential is not yet revoked |
| Other variables vanished after the update | --environment replaced the whole map; re-apply the full set from the previous version's configuration |
AccessDeniedException on GetSecretValue | Execution role lacks the permission, or the secret uses a customer managed key the role cannot decrypt |
| Timeouts reading the secret from a function in a VPC | AWS notes functions in a VPC need a Secrets Manager VPC endpoint or another route to the service |
| Stale value after rotation | Lower SECRETS_MANAGER_TTL or request versionStage=AWSCURRENT |
| Secret back after the next deploy | The template or pipeline still sets it; fix the source and add a pipeline check |
Choosing the right approach
| Option | Use when | Trade-off |
|---|---|---|
| Secrets Manager with the extension | Most runtimes; secrets that rotate; teams wanting no SDK code | Extra layer and a local HTTP call; cache TTL decides how fast rotation reaches the function |
| Secrets Manager through the SDK with caching | Teams already using the SDK; fine control over refresh | Code change in every function |
Parameter Store SecureString | Simple, rarely changing values | No built-in rotation workflow |
| Keep in environment variable with a customer managed key | Short-term exception while migrating | Gates viewing on kms:Decrypt but the value still sits in every version |
| Remove the secret altogether | The secret is an AWS access key or a credential for another AWS service | Best outcome: grant the execution role the permission instead of storing a key |
A small team with a handful of functions can run the steps by hand in an afternoon. An organisation with thousands of functions should run discovery centrally on a schedule, report key names and masked values to owners, and block new secret-looking variables in the deployment pipeline. Also limit who holds lambda:GetFunctionConfiguration: review read-only roles, posture tools and agent integrations, and consider a customer managed key with a kms:Decrypt deny for principals that need to manage but not read configuration, the pattern AWS describes.
Detection and monitoring
- Configuration reads: CloudTrail records
GetFunctionConfigurationandGetFunctioncalls without the variable values. Alert on bulk reads across many functions by one principal, and on reads by principals outside the owning team or approved tooling. - Configuration changes: alert on
UpdateFunctionConfigurationoutside the deployment pipeline, which is how secrets get added by hand and how attackers change a function's behaviour. - Logs: run the secret scanner over CloudWatch Logs groups for functions that held secrets, since error handlers and debug output may have written values there; shorten retention or purge where you find them.
- Drift: schedule the discovery steps weekly and treat any new secret-like key as a pipeline defect, not just a finding.
Where identity fits
Most secrets in Lambda variables are credentials for other non-human identities: database logins, SaaS API tokens, and too often long-term IAM access keys. The last are always avoidable because the function already has an execution role; give the role the permission and delete the key. Each secret you find should be recorded against the identity it belongs to, with an owner and a rotation date. See AWS Secrets Manager and, for any leaked AWS keys, the Cloud Access Key Leak Playbook.
Common failure patterns
- Cleaning
$LATESTonly: every published version still serves the old secret to anyone with read access. - Moving without rotating: the secret was readable for months; moving it to Secrets Manager does not make the old value unknown.
- Pasting the export into a ticket: discovery output contains plaintext values and becomes a new leak.
- Forgetting the second copy: templates, pipeline variables and state files hold the same value; the Indian Government breach shows how exposed configuration files leak credentials in bulk.
- Debug logging the environment: a print of the environment on error sends every secret to CloudWatch Logs, which many more people can read.
Metrics
| Metric | Target |
|---|---|
| Functions (all versions) with secret-like variables | Falling to zero outside documented exceptions |
| Long-term AWS access keys found in variables | Zero |
| Findings rotated at source within SLA | 100% |
Principals with lambda:GetFunctionConfiguration outside the owning teams | Known and falling |
| Deployments blocked for new secret-like variables | Tracked |
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | IA-5 Authenticator Management; SC-28 Protection of Information at Rest; AC-6 Least Privilege | Protect and rotate authenticators; limit who can read them |
| NIST CSF 2.0 | PR.AA-01; PR.DS-01 | Manage service credentials; protect data at rest |
| MITRE ATT&CK | T1552 Unsecured Credentials; T1078.004 Cloud Accounts | Credentials stored where readers can collect them, then reused |
| MITRE CWE | CWE-526 | Cleartext storage of sensitive information in an environment variable |
Practitioner checklist
- List every function with
--function-version ALLin every Region and account. - Scan key names first, then values, with your repository secret scanner rules.
- Treat the export as a secret and report masked values only.
- Rotate each secret at its source before moving it.
- Read secrets at runtime from Secrets Manager; keep only references in variables.
- Publish, move aliases, then delete old versions that hold the secret.
- Replace stored AWS keys with execution role permissions.
- Restrict
GetFunctionConfigurationand block new secret-like variables in the pipeline.
Standards and references
- AWS: Working with Lambda environment variables
- AWS: Securing Lambda environment variables
- AWS: Use Secrets Manager secrets in Lambda functions
- AWS CLI Reference: lambda list-functions
- AWS CLI Reference: lambda get-function-configuration
- AWS: Get a Secrets Manager secret in Lambda
- MITRE: CWE-526 Cleartext Storage of Sensitive Information in an Environment Variable
Related NHI Mgmt Group resources: Environment Variables and .env Files · AWS Lambda Security Guide · AWS Secrets Manager · Cloud Secrets Managers Guide