Offline Conditional Access analysis exposes policy overlap gaps

At a glance

What this is: NHI Mgmt Group's analysis, drawing on SpecterOps's blog post: Offline Conditional Access analysis can separate definitive policy application from signal-dependent enforcement and expose where overlap creates gaps.

Why it matters: IAM teams need this distinction because Conditional Access design fails when policy intent, sign-in signals, and effective enforcement do not line up under real authentication conditions.

Context

Conditional Access is a policy engine, not a simple allow-or-deny switch. When policies overlap, the real governance problem is determining which rules apply, which ones depend on runtime signals, and where enforcement changes once sign-in context is known. Conditional Access only works cleanly when teams can reason about those dependencies before a user reaches production access paths.

For Entra ID programmes, the challenge is not visibility alone. Teams also need a repeatable way to test policy combinations without creating tenant noise, especially when MFA, device state, and client context influence the final decision.

Key questions

Q: How should teams test overlapping Conditional Access policies without touching production sign-ins?

A: Teams should test overlapping Conditional Access policies in a non-production tenant or a cloned policy set, then compare the simulated outcome for a target user, group, device state, location and app. Use a policy evaluation tool or offline ruleset export to see which grant and block controls would apply, without creating live sign-in prompts or lockouts.

Q: Why do Conditional Access policies fail when MFA depends on sign-in context?

A: Conditional Access policies fail when MFA depends on sign-in context because the policy engine may not yet have the device, location, risk or authentication strength data needed to decide whether to require or satisfy MFA. If the context is missing, stale or evaluated in the wrong order, the policy can block access, skip MFA, or trigger an unexpected challenge.

Q: How do you build Conditional Access policies in Microsoft Entra ID?

A: Build Conditional Access policies by defining the users, cloud apps, conditions, and grant controls you want to enforce, then test each policy in report-only mode before turning it on. Keep policies small and purpose-built, document exceptions, and monitor sign-in results so you can see which controls are blocking legitimate access and which are missing coverage.

Q: What is the difference between offline policy analysis and live What If testing?

A: Offline policy analysis evaluates access rules against stored policy data and a known user, device and app profile, without contacting the live sign-in service. Live What If testing runs the same decision logic against the current tenant state and real-time context, so it is closer to production but can still be affected by transient signals, policy changes and session state.

Technical breakdown

How Conditional Access policy evaluation actually compounds

Conditional Access policies are evaluated together, not in isolation. Each sign-in accumulates context about the user, application, device, client app, and available risk signals, then every applicable policy contributes to the final decision. A single policy can block access, but more often the outcome depends on the combined requirements of several policies. That is why overlap matters: a policy that looks permissive on its own may still fail once another enabled rule adds MFA, device compliance, or blocking conditions. Offline analysis has to preserve that cumulative logic or it will misstate the effective control surface. See the Conditional Access Guide.

Practical implication: Model policy sets as a combined enforcement surface, not as individual rules with independent outcomes.

Why offline what-if analysis needs signal dependency handling

Offline evaluation is useful only if it distinguishes what can be known from what is only resolved during sign-in. Some conditions, such as risk signals, authentication flow details, or device context, are not fully available until the runtime session exists. That means an offline engine must classify outcomes as definitive or signal-dependent rather than pretending to know the answer in advance. Normalisation also matters because Entra ID maps names, identifiers, and grouped resources into canonical forms that raw policy exports do not show directly. Without that modelling step, offline results drift away from actual enforcement. See the Active Directory and Entra ID Hardening Guide.

Practical implication: Separate deterministic policy matches from contingent ones so reviewers do not mistake incomplete inputs for enforcement certainty.

How exclusions and app groupings distort policy scope

Conditional Access scope is not just a list of assignments. Exclusions invert logic, special values such as All and None do not always behave like plain wildcards, and cloud app groupings compress many underlying resources into a single target. That makes offline reasoning harder because the apparent policy target may not match the operational target Entra ID evaluates. If a tool cannot account for grouped applications, exclusions, and special-set semantics, it will miss some policies and overstate others. The result is false confidence in what a sign-in would actually trigger. See the Microsoft 365 Security Guide.

Practical implication: Test scope resolution against grouped apps and exclusion logic before trusting any offline decision.

Threat narrative

Attacker objective: The objective is to find a viable Entra ID access path that avoids the controls defenders assume will enforce MFA or block the sign-in.

1. Entry occurs when an assessor reaches Entra ID-backed resources and starts enumerating users, groups, applications, and service principals to understand possible access paths.
2. Credential or policy access is then examined through Conditional Access, where the evaluator looks for MFA bypass conditions, excluded scopes, and signal-dependent rules that may not hold at runtime.
3. Escalation happens when overlapping policies and unresolved sign-in signals create a mismatch between expected enforcement and actual enforcement, allowing access paths that were assumed closed.
4. Impact is the ability to reason about or reach a protected resource without triggering the control the defenders believed would stop the sign-in.

Breaches seen in the wild

• Commvault Metallic breach 2025: A nation-state actor exploited a Commvault zero-day in Azure and may have taken app secrets that open customers' Microsoft 365 tenants.
• Storm-1283 OAuth apps abuse 2023: Storm-1283 added secrets to new and existing OAuth apps in Entra ID and used them to run Azure cryptomining VMs, costing victims up to $1.5 million.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Conditional Access fails as a governance object when teams treat runtime signals as if they were static policy inputs. Offline analysis exposes the gap between written intent and effective enforcement. The real issue is not whether a policy exists but whether it is decidable before sign-in. Practitioners should treat signal dependency as a control-design problem, not an administrative nuisance.

Policy overlap is the operational reality, not an edge case. Entra ID evaluates all applicable policies together, so one permissive rule does not cancel a stricter one. That means access governance has to be reasoned about as a cumulative decision surface, especially when exclusions, special values, and device filters interact. Teams that review policies one at a time are missing the governing model.

Offline analysis changes Conditional Access from intuition-driven review to testable control behaviour. That matters because administrators and red teams often need to ask the same question from different angles: where does enforcement depend on assumptions that are only true during live sign-in? The strongest CAP programmes are the ones that can answer that question before production users or testers do.

Policy normalisation is a hidden source of governance error. Entra ID resolves app groupings, resource identifiers, and client values into forms that do not always match the raw export. When policy reviews ignore that translation layer, they misread scope and miss the controls that actually apply. The practitioner lesson is to validate policy meaning, not just policy text.

Signal-dependent enforcement should be treated as a named design concept. Runtime enforcement uncertainty: the policy appears understandable on paper but only becomes fully decidable when Entra ID sees live sign-in context. That is the point where governance teams must stop assuming offline certainty and start measuring what the control really depends on.

What this signals

Runtime enforcement uncertainty: Conditional Access programmes should be designed around the fact that some decisions are only fully knowable at sign-in time. That means governance teams need to distinguish policy intent from policy decidability, or they will keep mistaking incomplete offline evidence for real enforcement coverage.

Offline analysis becomes most valuable when it is tied to change management. Each policy update should be checked for overlap, exclusion drift, and app grouping effects so that the access surface stays explainable as the environment grows.

The practical benchmark is whether a team can predict the enforced result for a representative sign-in before the event happens. If not, the programme is still relying on human interpretation instead of control design.

For practitioners

• Build a policy overlap map Group all enabled Conditional Access policies by user, resource, platform, client app, and exclusion logic so you can see where cumulative enforcement changes the effective outcome.
• Classify definitive versus contingent outcomes Mark each offline result as either fully enforced or dependent on missing sign-in signals such as risk, device context, or authentication flow.
• Validate cloud app groupings separately Test policies against grouped resources and not only single application identifiers, because Entra ID can resolve Office-style groupings to multiple underlying services.
• Reassess exclusion paths before deployment Review exclusions, All, and None conditions together so that exceptions do not quietly erase the control you thought would apply.
• Use offline scenarios to compare enforcement states Simulate common sign-in combinations for user, device, platform, and client app to identify where MFA or blocking requirements appear only under certain conditions.

Key takeaways

• Conditional Access can look coherent on paper while still producing ambiguous enforcement once overlapping policies and runtime signals are combined.
• The main failure mode is not missing policy volume but hidden decision dependency, especially where MFA, device state, and app grouping affect the final result.
• Offline analysis gives defenders and testers a way to validate policy behaviour before sign-in, which makes access governance more repeatable and defensible.

Related NHI Mgmt Group resources: Conditional Access Guide · Active Directory and Entra ID Hardening Guide · Zero Trust Identity Guide · Microsoft 365 Security Guide

Key terms

• Conditional Access: Conditional access is a policy model that decides whether an action should proceed based on context such as posture, resource sensitivity, timing, and scope. For AI agents, it must be evaluated at request time so a valid credential does not automatically equal permitted behaviour.
• Sign-In Risk: Sign-In Risk is the assessed likelihood that a login attempt is malicious or abnormal based on signals such as location, device, behaviour, network, and credential state. Security teams use it to drive step-up authentication, block suspicious access, and investigate anomalies, while tuning thresholds to reduce false positives and missed attacks.
• Policy overlap: Policy overlap is the condition where two or more access policies apply to the same user, app, device or session at the same time. In practice, overlapping rules can combine, conflict or short-circuit depending on the policy engine, which makes the effective access decision harder to predict and test.
• Cloud app grouping: Cloud app grouping is the classification of cloud applications into a named set so they can be managed with one policy, control or report. Administrators use the group as a targeting unit for access rules, security settings, monitoring and entitlement assignment across multiple apps.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity lifecycle are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an IAM programme, it is worth exploring.