Join our Newsletter — 33% off our NHI Course
Home› Guides› Identity control validation and continuous assurance Guide
Guide Identity Visibility, Posture & Threat Detection

Identity control validation and continuous assurance Guide

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 14 November 2026 13 min read
On this page

Identity controls are usually tested once, when they are built, and then assumed to keep working while group memberships, app registrations, exclusions and role assignments change underneath them. Continuous identity control validation replaces that assumption with tests: for each preventive control, state the outcome it must produce, then check it on a schedule and after every change. This guide covers how to write those expected outcomes, how to test conditional access without real sign-ins, how to find MFA enforcement gaps, how to prove privileged and just-in-time access actually expires, and how to report the results as assurance evidence. It is written for identity engineers, security assurance teams and auditors. Detection testing for identity attacks is in ITDR validation, and endpoint and network control simulation is in the Breach and Attack Simulation (BAS) Guide; this page covers the preventive identity layer that neither tests directly.

Key takeaways

  • Write the outcome, not the setting. "Every interactive sign-in by a privileged role from outside a compliant device is blocked" is testable; "CA policy 12 is on" is not.
  • Simulate first, observe second. Policy simulators answer "would this be enforced"; sign-in and activation logs answer "was it enforced". Assurance needs both, because simulators skip conditions they are not given.
  • Drift is the main failure mode. Exclusions added during incidents, new apps outside policy scope and permanent role assignments created "temporarily" are what break controls, not the original design.
  • Coverage gaps hide in definitions. A user who is "MFA capable" may still sign in with a single factor if no policy requires MFA for that app, client or location.
  • Every failed assertion is a ticket. A result with no owner and no due date is a finding, not assurance.

Validation, posture management and detection testing

Three practices look similar and are often bought or staffed as one. Keeping them distinct avoids gaps.

PracticeQuestion it answersTypical evidenceWhere covered
Identity control validation (this guide)Does each preventive control produce the outcome we require, today, for every identity in scope?Pass or fail per assertion, simulator results, sign-in log samplesThis page
Identity security posture managementWhich misconfigurations and risky identities exist?Findings list, risk scoresISPM Guide
Detection validationWhen an identity attack happens, does an alert fire and reach someone?Detection rate per technique, time to alertITDR validation
Breach and attack simulationDo endpoint, network and email controls stop emulated techniques?Prevented, detected, missed per scenarioBreach and Attack Simulation (BAS) Guide

NIST SP 800-137 frames the goal: continuous monitoring gives ongoing assurance that controls remain effective and aligned with risk tolerance. In SP 800-53 terms this is CA-7 Continuous Monitoring fed by CA-2 Control Assessments, rather than a once-a-year audit sample.

Writing expected outcomes as assertions

An assertion has four parts: the identity population, the access path, the expected result and the evidence source. Write them from your policy and from an external baseline so you are not only testing what you already built. CISA's SCuBA Microsoft Entra ID baseline is a useful starting set because its policies are already phrased as outcomes:

Control areaExample assertionExternal anchorPrimary test method
Legacy authenticationAny sign-in using a legacy client by any user is blockedSCuBA MS.AAD.1.1v1Simulator with legacy client app; sign-in log query for legacy clients with success
Risk-based blockingHigh-risk users and high-risk sign-ins are blockedMS.AAD.2.1v1, MS.AAD.2.3v1Simulator with sign-in risk High; review of risky sign-ins that succeeded
MFA for everyoneEvery user, including guests, must complete MFA for every cloud appMS.AAD.3.2v2Coverage query on sign-ins completed with a single factor
Phishing-resistant MFAHighly privileged roles can only sign in with phishing-resistant methodsMS.AAD.3.6v1Simulator per privileged role; sign-in method sample
Role populationBetween two and eight Global AdministratorsMS.AAD.7.1v1Directory count, compared each run
No standing privilegeNo permanent active assignments to highly privileged rolesMS.AAD.7.4v1Role assignment inventory
Privilege only through PAMNo highly privileged role is assigned outside the PAM systemMS.AAD.7.5v1Audit log of role assignments not made by the PAM workflow
Activation controlGlobal Administrator activation requires approval and raises an alertMS.AAD.7.6v1, MS.AAD.7.8v1Role setting read plus a scheduled test activation

Add assertions the baseline does not cover: service principals and workload identities that should be blocked from unexpected locations, break-glass accounts that must stay excluded from blocking policies but must alert on every use, and guest access to admin portals. Non-human identities often fall outside user-centred policies; see How to Find Accounts Excluded From MFA and Conditional Access and the Break-Glass Account Guide.

Testing conditional access without real sign-ins

Policy simulation is the fastest feedback loop because it needs no test accounts signing in. Microsoft Entra ID's worked example shows both the value and the traps.

  • What If tool and API: simulates a sign-in for a user, an agent identity (in preview) or a single tenant service principal against policies that are enabled or in report-only mode. Identity, target resource, device platform and client app are required; other conditions default to none. Results list policies that apply, with the grant and session controls required, and policies that do not apply, with the first unmet condition as the reason. The same engine is exposed as the What If Evaluation API in Microsoft Graph, so assertions can run in a pipeline.
  • Complete inputs: Microsoft's example shows a policy scoped to location and high sign-in risk returning "Does not apply" until all of application, location and sign-in risk are supplied. A test that omits conditions passes for the wrong reason, so each assertion should fix every condition the policy uses.
  • App groups and dependencies: groups such as Office 365 or Microsoft Admin Portals do not match in the simulator, which needs individual app IDs, and service dependencies are not tested. Test the specific apps your assertion names.
  • Report-only before enforcement: report-only policies are evaluated at real sign-ins and logged as Report-only: Success, Failure, User action required or Not applied, without prompting users. User Actions scope cannot be evaluated this way. Microsoft warns that report-only policies requiring a compliant device can repeatedly prompt macOS, iOS and Android users to pick a certificate, so exclude those platforms from report-only compliance policies.

Other platforms offer comparable simulation for authorisation policy. The AWS IAM policy simulator evaluates identity policies, permissions boundaries, service control policies and supplied resource policies and returns allowed or denied per action, without making a real request. AWS notes results can differ from live behaviour for VPC endpoint policies, role chaining or multiple resource-based policies, and that resource control policies are not supported. Whatever the platform, treat simulator output as a fast pre-check and confirm with observed outcomes. See the Conditional Access Guide for policy design.

Detecting policy drift

Drift validation compares today's control state with the last approved state and re-runs assertions whenever it changes.

  1. Snapshot policy as code: export conditional access policies, authentication method policies and PIM role settings (Microsoft Graph exposes PIM rules through unifiedRoleManagementPolicy) to version control on each run.
  2. Diff and classify: flag new exclusions, state changes from On to Report-only or Off, removed grant controls, narrowed app scope and new named locations. Exclusions are the highest-risk change because they are invisible in a policy's headline.
  3. Re-run affected assertions: a membership change to an excluded group should trigger every assertion that group touches, not wait for the weekly run.
  4. Baseline scanners on a schedule: CISA's ScubaGear queries Microsoft 365 APIs with PowerShell and evaluates the settings against Rego policies written from the SCuBA baselines using Open Policy Agent, producing HTML, JSON and CSV. The open source Maester framework runs Pester-based PowerShell tests and emits HTML, JSON and JUnit output for pipelines, and skips checks with a reason code when a licence or service is missing, which you should report rather than silently accept.

Broader configuration drift practice is in the Configuration Drift and Baseline Enforcement Guide.

Finding MFA enforcement gaps

MFA coverage has two halves that are often confused: registration (can the user do MFA) and enforcement (was MFA required on this sign-in). Microsoft's registration details report defines "MFA capable" as registered for a strong method that policy also allows for MFA; it lags by up to 36 hours and omits disabled and soft-deleted users. A user can be MFA capable and still sign in with a password alone if no policy requires MFA for that path.

  • Enforcement test: query successful sign-ins that completed with a single factor, grouped by application, client type and policy result. Each group is either a documented exception or a gap.
  • Path inventory: list every way to authenticate (IdP, legacy protocols, VPN and remote access portals, local application accounts, API keys) and assert MFA or an approved alternative for each. The Change Healthcare breach began with a stolen credential on a remote access portal that did not require MFA.
  • Forgotten tenants and test environments: include non-production tenants in scope. The Midnight Blizzard breach started with a password spray against a legacy test tenant account without MFA.
  • Method strength: for privileged roles, assert the method used was phishing resistant, not only that MFA happened. See MFA Coverage Measurement for the metric design.

Proving privileged and just-in-time access expires

JIT controls fail silently: an activation that lasts longer than intended or an eligible assignment that never ends looks normal in day to day use. Test the settings and the behaviour.

AssertionSetting to read (Entra PIM example)Behavioural test
Activations end within the approved windowActivation maximum duration (1 to 24 hours)Activate a test role and confirm the assignment is removed at expiry
Activation requires strong authenticationOn activation, require MFA, or require a Conditional Access authentication contextActivate from a session without fresh strong authentication and confirm a prompt
High-impact roles need a second personRequire approval to activate, with named approversSubmit a test activation and confirm it waits for approval
Eligibility itself endsExpire eligible assignment after, rather than Allow permanent eligible assignmentReport eligible assignments with no end date
No standing active privilegeExpire active assignment after, rather than Allow permanent active assignmentReport active assignments to privileged roles with no end date
Activation is visibleNotifications for activation of privileged rolesConfirm the alert arrives in the monitored channel, not only an inbox

Two documented limits shape these tests. Microsoft states the ticket number on activation is not validated against any ticketing system, so check it against your ITSM records yourself. Requiring MFA on active assignment applies when an administrator creates the assignment; PIM cannot enforce MFA when that assignment is later used, so the sign-in path still needs its own conditional access assertion. The same pattern applies to cloud PAM and secrets brokers that issue time-bound credentials: assert on issued lifetime and test that the credential stops working. See the JIT Access Guide and the Privileged Identity Management (PIM) Guide.

Implementation roadmap

  1. Weeks 1 to 2, catalogue: list preventive identity controls, owners and the outcome each must produce. Map them to the SCuBA policies or your own standard.
  2. Weeks 3 to 4, read-only checks: run baseline scanners and settings reads with a read-only identity. Record the starting pass rate; do not fix anything yet so the baseline is honest.
  3. Month 2, simulation suite: encode conditional access assertions as simulator calls with every condition supplied, run them in a pipeline on each policy change and nightly.
  4. Month 3, observed outcomes: add sign-in and activation log queries that confirm what the simulator predicted, and scheduled test activations for JIT roles using dedicated, named test identities with expiry dates.
  5. Ongoing, change-triggered runs: trigger affected assertions on policy, group and role changes. Failed assertions open tickets with the control owner.
  6. Rollback and safety: validation jobs must never change policy. If a test identity or job credential is misused, disable it; nothing else should depend on it.

The validation service's own credential is a sensitive non-human identity: it can read every policy and role assignment. Give it read-only scopes, keep it in a secrets store, monitor its sign-ins and review it with other privileged workload identities.

Measuring and reporting assurance

  • Assertion pass rate: passing assertions over total, by control area, trended weekly.
  • Time to detect drift: hours between a control-weakening change and the failed assertion; target same day.
  • Time to remediate: days from failed assertion to pass, by severity.
  • Single-factor sign-in rate: successful single-factor sign-ins not covered by a documented exception, trending to zero.
  • Standing privilege count: permanent active and permanent eligible assignments to privileged roles.
  • Skipped checks: assertions not evaluated because of missing licence, permission or data; these are coverage gaps, not passes.

Report results as evidence for audits: each assertion with its last run, result and linked ticket. For the board, roll up to trend lines on the pass rate and standing privilege.

Common failure patterns

  • Testing the policy you wrote, not the paths that exist: assertions only cover applications already in scope, so new apps and legacy portals never get tested.
  • Incomplete simulator inputs: omitting location or risk makes a policy "not apply" and the assertion silently passes or fails for the wrong reason.
  • Exclusions that outlive the incident: a group excluded during an outage stays excluded and grows; no assertion checks exclusion membership.
  • Registration mistaken for enforcement: high "MFA capable" percentages reported as MFA coverage while single-factor sign-ins continue.
  • JIT settings checked, expiry never observed: settings look right but nobody has watched an activation end or an eligibility lapse.
  • Skipped checks counted as passes: licence-dependent checks that did not run are reported green.

Framework mapping

FrameworkReferenceRelevance
NIST SP 800-53 Rev 5CA-2 Control Assessments; CA-7 Continuous Monitoring; CA-7(4) Risk MonitoringScheduled and change-triggered assertion runs as ongoing assessment
NIST SP 800-53 Rev 5AC-2(2) Automated Temporary and Emergency Account Management; AC-6(7) Review of User PrivilegesJIT expiry and standing privilege assertions
NIST CSF 2.0PR.AA-05; GV.OV-03; ID.IM-02Access permissions enforced and reviewed; performance evaluated; improvements from tests
CISA SCuBAMS.AAD.1.1v1, MS.AAD.3.2v2, MS.AAD.3.6v1, MS.AAD.7.4v1, MS.AAD.7.5v1Outcome statements used as assertions
MITRE ATT&CK v20T1078.004 Cloud Accounts; T1556.006 Multi-Factor Authentication; T1098 Account ManipulationTechniques that exploit or create the gaps these assertions look for

Practitioner checklist

  • Write each preventive identity control as an assertion with population, path, expected result and evidence source.
  • Start from the SCuBA Entra ID baseline or your equivalent standard, then add NHI and break-glass assertions.
  • Supply every condition a policy uses when calling a simulator, and test named apps rather than app groups.
  • Confirm simulator predictions against sign-in logs before treating them as evidence.
  • Export identity policy to version control and alert on new exclusions and state changes.
  • Measure single-factor sign-ins, not just MFA registration.
  • Run scheduled test activations to prove privileged roles expire and require approval.
  • Report skipped checks as gaps and give every failed assertion an owner and due date.
  • Keep the validation job's identity read-only and monitored.

Standards and references

Related NHI Mgmt Group resources: Breach and Attack Simulation (BAS) Guide · ITDR validation · ISPM Guide · Conditional Access Guide · JIT Access Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 14 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.