Notifications
Clear all
Topic starter
17/08/2026 4:39 pm
TL;DR: Centralized secrets handling can reduce secret scattering and cloud lock-in for Azure Databricks DataOps and MLOps workflows, while supporting Azure AD authentication and retrieval across Python, Scala, R, and SQL, according to Akeyless. The governance issue is that workload credentials still need lifecycle control, not just convenient access paths.
NHIMG editorial — based on content published by Akeyless: Cloud-agnostic secrets management for Azure Databricks workloads
By the numbers:
- Only 44% of organisations are currently using a dedicated secrets management system.
- Secrets management is a top five cybersecurity priority for only 33% of organisations, behind cloud security 45%, API security 42%, and endpoint security 36%.
Questions worth separating out
Q: How should security teams govern secrets in Azure Databricks workloads?
A: Treat the notebook, workspace, and secret store as one access chain.
Q: Why do cloud-specific secret vaults create governance problems for data platforms?
A: Cloud-specific vaults can tie access to one provider or one workspace, which makes migration, cloning, and multi-cloud operation harder to control.
Q: What breaks when secrets are passed through notebooks and Spark configuration?
A: The main failure is visibility.
Practitioner guidance
- Centralise workload secret retrieval Move Databricks credentials into a governed repository and require runtime lookup rather than embedding values in notebooks, jobs, or configuration files.
- Bind access to workspace-specific identity Use managed identity or equivalent workload identity to tie secret retrieval to a specific workspace and avoid reusable shared credentials across environments.
- Eliminate duplicate secret copies Search for secrets replicated in code, job parameters, Spark configs, and documentation, then remove redundant copies before the pipeline expands further.
What's in the full article
Akeyless's full blog post covers the operational detail this post intentionally leaves for the source:
- Step-by-step Databricks notebook code for authenticating to Akeyless with Azure cloud ID and managed identity.
- Python, Scala, and R retrieval patterns for passing secrets through Databricks runtime without hardcoding values.
- Example API request and Spark table write flow that shows how the retrieved secret is used in an actual workload.
- Implementation sequence for setting up Azure AD authentication and workspace-specific access paths.
👉 Read Akeyless's analysis of secrets management for Azure Databricks workloads →
Azure Databricks secrets management: what IAM teams need to know?
Explore further
View Full Forum → | NHI Foundation Course → | Our Services →
17/08/2026 4:45 pm
Cloud-agnostic secrets management is now a workload identity problem, not just a vault problem. Databricks users are not only choosing where to store a secret, they are defining how a workload proves who it is across notebook runtimes, languages, and cloud boundaries. That makes the control surface broader than a single vault or SDK. Practitioners should treat retrieval policy, runtime identity, and secret lifecycle as one governed system.
A few things that frame the scale:
- 54% of organisations are dissatisfied with their current secrets management solution because not all secrets are secured, and 43% cite lack of central management, according to The 2024 State of Secrets Management Survey.
- 88% of security professionals are concerned about secrets sprawl, with 49% of those in larger organisations described as very concerned.
A question worth separating out:
Q: Should managed identity replace all secrets in Databricks?
A: No. Managed identity reduces the need to embed static credentials, but it does not eliminate access governance. Teams still need least privilege, workload separation, and review of which identities can reach which secrets. A better test is whether the identity is bounded tightly enough to keep blast radius small.
👉 Read our full editorial: Cloud-agnostic secrets management for Azure Databricks workloads
