Effective access breaks down because the identity provider cannot fully describe permissions that are created inside workspaces and Unity Catalog. Teams may certify group membership while missing direct grants, nested inheritance and object-level access that still allow data, compute or model reach. The result is a false sense of control over who can actually use the platform.
Why the identity provider is only the starting point
Databricks governance becomes incomplete when teams treat the identity provider as the full source of truth for access. The IdP can tell you who belongs to a group, but it cannot fully express what happens inside the platform after that group lands in workspace permissions, Unity Catalog grants, inherited privileges, service principals, or direct object access. That gap is where false confidence starts.
In practice, the access model splits across identity, workspace, and data-plane layers. A clean group review may still miss direct grants to users or service principals, nested role inheritance, and object-level permissions on data, compute, notebooks, models, and shares. The question is not whether the IdP is wrong, but whether it is too shallow to represent the full authorisation state you actually need to govern.
That is why platform governance has to follow the effective access path, not just the directory record. If a permission can be granted, inherited, or bypassed inside Databricks, then it belongs in the review scope even when the upstream identity record looks clean. The governance control must reflect the platform’s own permission graph, not only the organisation chart in the IdP.
For teams formalising this boundary, the identity platform itself is only one part of the access-control problem, which is why an Identity Provider and SSO Security Guide is useful for the upstream layer while platform-native permission review handles the downstream layer. The same applies to the broader identity stack described in the IAM and Identity Provider Buyer’s Guide, where federation and lifecycle are only part of the control story.
Where Databricks permissions escape IdP visibility
The main failure mode is not exotic. It is the accumulation of access that exists outside the directory, then becomes invisible to certification because it is not tied cleanly back to a group. Direct entitlements, inherited catalog permissions, and workspace-specific grants can all create reach that persists after a user should have lost it.
That same pattern is amplified when service accounts, automation, or model-serving identities are involved. The IdP may still show the right human population, while the platform continues to expose data or compute to non-human actors, shared access paths, or stale grants that never pass through the same review cycle.
In a governed environment, the control problem is therefore not just “who is in the group?” but “what can that subject actually do inside the workspace and catalog?” When those two answers diverge, recertification is no longer evidence of least privilege. It is evidence that the review process is incomplete.
What good governance has to review inside the platform
Effective governance needs a permission inventory that can reconcile the IdP with Databricks-native access objects. That means reviewing direct grants, inherited access, workspace entitlements, catalog permissions, compute permissions, and any object-level access that can override a simple group-based view.
The control lens should also include ownership and revocation mechanics. If a permission cannot be traced to a named owner, a business justification, and a revocation path, it will tend to outlive the access need that created it. A directory review alone rarely exposes that drift.
This is also where lifecycle discipline matters. The access pattern must be reviewed after joiner, mover, and leaver events, after workspace changes, and after new data or model objects are published. The governance question is not whether a user still belongs to a group, but whether the entitlement still matches the current business use.
For lifecycle and entitlement control, the NHI Lifecycle Management Guide is a useful reference for thinking about provisioning, rotation, offboarding, and visibility, while the Top 10 NHI Issues highlights the common ways permissions, ownership, and stale access drift away from policy.
Risk and Threat Considerations
When governance stops at the identity provider, the main risk is hidden access: controls appear clean at the directory layer while the platform still allows data, compute, or model reach through direct, inherited, or object-level grants. That creates an audit blind spot and a real exposure gap if privileged or stale access remains active.
Failure mechanism: The review process certifies group membership instead of effective permissions, so access paths created inside Databricks are never revalidated, revoked, or challenged during recertification.
Impact: Users, service principals, or shared workflows can retain access to sensitive assets after business need changes, increasing the chance of unauthorised use, lateral reach, and delayed detection of overexposure.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10 addresses the attack and risk surface, while NIST SP 800-53 Rev 5 sets the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | AC-2 — Account Management | Databricks access must be inventoried and reviewed beyond directory groups. |
| AC-6 — Least Privilege | Direct grants and inherited permissions can exceed intended access. | |
| AC-16 — Security and Privacy Attributes | Object-level and inherited permissions depend on attributes attached to resources and subjects. | |
| Recommendation — Review platform-effective access, not just IdP group membership. Remove any Databricks entitlement that exceeds the user's needed access. Map Databricks object access rules to enforced security attributes. | ||
| OWASP Non-Human Identity Top 10 | NHI-05 — Overprivileged NHI | Non-human identities in Databricks can retain excessive platform access. |
| NHI-01 — Improper Offboarding | Stale grants can persist after people or automation should lose access. | |
| Recommendation — Audit service principals and automation for excess Databricks privileges. Revoke Databricks access paths at leaver and decommission events. | ||
Practitioner Guidance
What to verify: Prove that every high-value workspace, catalog, compute, and model entitlement can be traced back to an identity, an owner, and a revocation process. If the answer depends on manual exception handling, the control is already too weak to trust as a governance boundary.
Decision rule: If the IdP review and the platform permission graph do not match, treat the platform as the authoritative record for effective access and use the directory only as an input, not the control conclusion.
What practitioners underestimate: Recertification can look successful while still missing direct grants and inherited access, so the most important failure is usually not a technical break but a false conclusion that access has been governed when it has only been sampled.
Practitioner takeaway: The governance objective is effective access, not directory cleanliness, and that means reviewing the platform’s actual permission model whenever the IdP cannot fully express it.