CISA SCuBA Baselines Guide: Running ScubaGear and ScubaGoggles as a Programme

Most Microsoft 365 and Google Workspace tenants drift away from a secure configuration one admin change at a time, and few organisations can say on any given day which settings fail a published baseline. CISA's Secure Cloud Business Applications (SCuBA) project gives a public, testable answer: secure configuration baselines for Microsoft 365 and Google Workspace, and two open source assessment tools, ScubaGear and ScubaGoggles, that check a tenant against them. This guide explains how the baselines are structured, how the tools work and what access they need, how to record exceptions properly, how BOD 25-01 uses SCuBA for US federal agencies, and how any organisation can run SCuBA as a continuous programme. It is for cloud security engineers, identity teams and GRC staff. Setting-by-setting hardening advice is in the Microsoft 365 Security Guide and the Google Workspace Security Guide; this page covers the baselines and tooling themselves.

Key takeaways

• SCuBA is a testable minimum, not a wish list. Each policy has a stable ID such as MS.AAD.3.1v1 or GWS.COMMONCONTROLS.1.1v1, a SHALL or SHOULD criticality, a rationale, and mappings to NIST SP 800-53 and MITRE ATT&CK.
• The tools read, compare and report. ScubaGear (PowerShell) and ScubaGoggles (Python) export tenant settings, evaluate them with Open Policy Agent Rego rules and produce HTML, JSON and CSV results.
• The YAML configuration file is the audit trail. Exclusions, annotations and omissions with rationale and expiry dates are how you document accepted risk; without the file, results reflect default assumptions.
• The assessment identity is privileged. ScubaGear's non-interactive service principal and ScubaGoggles' domain-wide delegated service account can read most of the tenant, and some required permissions include write capability.
• Federal deadlines show the operating model. BOD 25-01 required agencies to inventory tenants, run ScubaGear and implement mandatory policies in 2025, with continuous reporting after that, a cadence any organisation can copy.

What SCuBA provides

CISA developed the SCuBA secure configuration baselines with the Federal CIO Council. CISA's project page records that the Microsoft 365 baselines were first published in October 2022 and that the project also produced a Technical Reference Architecture (finalised June 2023), the extensible Visibility Reference Framework (eVRF) with workbooks for Microsoft 365 and Google Workspace, and a Hybrid Identity Solutions Architecture. The baselines themselves are maintained in the ScubaGear and ScubaGoggles GitHub repositories, which is where the current text lives.

Both projects are released under the Creative Commons Zero licence. CISA also publishes ScubaConnect, cloud-native infrastructure that automates running both tools. The baseline documents state that, for non-federal users, they are provided for informational purposes and that some settings may not be available in every licence.

How a baseline policy is written

Each policy uses RFC 2119 keywords. SHALL marks a mandatory control, SHOULD a recommended one. Badges in the baseline documents say whether a policy is a BOD 25-01 requirement, whether ScubaGear checks it automatically, whether it is configurable or requires configuration in the YAML file, or whether it must be verified manually. Each entry gives a rationale, a last modified date, NIST SP 800-53 Rev. 5 mappings (limited to the FedRAMP High baseline) and ATT&CK technique mappings.

Representative policies

The Entra ID baseline names the roles CISA treats as highly privileged: Global Administrator, Privileged Role Administrator, User Administrator, SharePoint Administrator, Exchange Administrator, Hybrid Identity Administrator, Application Administrator and Cloud Application Administrator. The baseline assumes E3 or G3 licensing and flags controls that need more, such as Entra ID P2 for risk-based policies. Version suffixes change when CISA revises a policy (MS.AAD.3.2v2, MS.EXO.2.2v3), so record the version you assess against.

How ScubaGear and ScubaGoggles work

1. Export: ScubaGear queries Microsoft 365 APIs through PowerShell; ScubaGoggles uses the Google Admin SDK and other Google APIs to collect settings, logs and role data.
2. Verify: Both pass the exported JSON to Open Policy Agent, which evaluates Rego policies written from the baseline documents.
3. Report: ScubaGear writes a time-stamped folder with BaselineReports.html, per-product reports, ScubaResults_{UUID}.json, ScubaResults.csv and ActionPlan.csv, which lists failing SHALL controls with fields for reasons and remediation dates. ScubaGoggles reports in HTML and JSON.
4. Compare: Invoke-SCuBADiff -BeforePath ... -AfterPath ... compares two ScubaGear result files offline and writes DiffResults.json, DiffResults.csv and DiffReport.html. ScubaGoggles documents an equivalent comparison of two runs.

The ScubaGear quick start is Install-Module -Name ScubaGear, Install-ScubaDependencies, then Invoke-SCuBA -ProductNames *. The documentation recommends a first run without a configuration file to see current posture, then repeated runs with a reviewed YAML file passed through -ConfigFilePath. Non-federal users add -SilenceBODWarnings. Update-ScubaGear updates the module, and Start-ScubaConfigApp opens a Windows configuration UI; on macOS and Linux, New-SCuBAConfig generates a template file.

Where identity fits: the assessment identity

A tool that reads every security setting in a tenant needs broad access, and its credential is a high-value non-human identity. Treat it with the same care as an administrator account.

The Microsoft Midnight Blizzard breach is the cautionary case: a forgotten test tenant without MFA led to a legacy OAuth application with elevated permissions. An assessment app that is left behind after a project ends, with a long-lived certificate and tenant-wide read rights, is the same kind of risk. Note too that BOD 25-01 excludes development and test tenants from its scope, but attackers do not: assess them as well.

Recording exceptions: exclusions, annotations and omissions

• Exclusions: Under the Aad key, CapExclusions lists the users, groups, applications or guest user types that a Conditional Access policy deliberately excludes, for example emergency access accounts. RoleExclusions covers the privileged role policies in section 7, and AgentExclusions lists service principal object IDs excluded from MS.AAD.9.1v1. CISA warns that exclusions "can introduce grave risks" and must be approved through risk acceptance. See the Break-Glass Account Guide.
• Annotations: AnnotatePolicy adds a comment to a result in the report. A comment is required when a control is marked incorrect, so reviewers see the justification beside the finding.
• Omissions: OmitPolicy removes a policy from evaluation and shows it as "Omitted". Each entry needs a Rationale and can carry an Expiration date in yyyy-mm-dd format. Use it when a third-party product meets the requirement (CISA's assumptions page gives this example for controls that need licences beyond E3 or G3) or when an operational deviation is approved.
• Organisation fields: OrgName and OrgUnitName record who owns the tenant, which matters when results flow to a central team.

Keep the YAML file in version control, require review for changes to any exclusion or omission, and set expiry dates so that exceptions return to the report for re-approval.

BOD 25-01 and the federal requirement

CISA Binding Operational Directive 25-01, Implementing Secure Practices for Cloud Services, and its implementation guidance were posted on 17 December 2024. The implementation guidance page, last updated 27 May 2025 when checked, sets out the following for Federal Civilian Executive Branch agencies:

The guidance limits scope to production SaaS tenants within federal information systems and excludes development and test tenants. Only SHALL policies are mandatory, although CISA strongly advises SHOULD policies too. Agencies report through CyberScope at least quarterly or through ScubaConnect, and document deviations as omitted policies with rationale, approved through their risk management process. At the May 2025 update the guidance said only the Microsoft 365 baselines were final and the Google Workspace baselines were in draft; the current ScubaGoggles baseline files carry BOD 25-01 Requirement markers, so check CISA's Required Configurations page for the authoritative list before relying on either statement. See the CISA Binding Operational Directives Guide.

Running SCuBA as a programme

1. Inventory tenants (week 1): List every Microsoft 365 and Google Workspace tenant, including test, legacy and acquired tenants, with an owner for each.
2. Set up the assessment identity (week 1 to 2): Create a dedicated app or service account with the documented least permissions, a certificate or key in a vault, and alerting on its sign-ins.
3. Baseline run (week 2): Run without a configuration file, store the JSON result as the starting point, and export ActionPlan.csv as the remediation backlog.
4. Triage (weeks 2 to 4): For each failing SHALL policy decide: fix, compensate with a documented omission, or accept with an annotation and expiry. Assign owners and dates.
5. Remediate in rings (weeks 4 to 12): Use report-only Conditional Access policies and pilot groups for authentication changes; change sharing and mail settings for a pilot organisational unit first; keep emergency access accounts excluded and documented. Roll back by reverting the individual setting, which the diff report will show.
6. Automate (month 3 onward): Schedule runs (weekly, and after change windows) through a pipeline or ScubaConnect, feed JSON results to the SIEM or GRC platform, and run Invoke-SCuBADiff to surface regressions.
7. Track baseline releases: Watch the repositories and the removed policies list; a new policy version can change a pass into a fail without any tenant change.

Choosing how far to go

• SHALL only or SHALL plus SHOULD: Federal agencies must meet SHALL policies. Non-federal organisations can treat SHALL as the floor and adopt SHOULD policies such as device code blocking (MS.AAD.3.9v1) where their threat model includes phishing of that flow.
• Interactive or automated: A small organisation can run ScubaGear interactively each month from an admin workstation. Larger or multi-tenant estates should automate with a certificate-based service principal and central result storage.
• SCuBA or another benchmark: SCuBA overlaps with CIS benchmarks for Microsoft 365 and Google Workspace. SCuBA's advantages are free tooling, stable policy IDs and government maintenance; organisations already audited against CIS can map one to the other rather than run two programmes.

Detection and measurement

• SHALL pass rate: Percentage of applicable SHALL policies passing per tenant, trended over time.
• Open action items: Count and age of rows in ActionPlan.csv, with the oldest item highlighted.
• Exception hygiene: Number of omissions and exclusions, and the share with an owner and an expiry date (target 100 per cent).
• Regression rate: Number of policies moving from pass to fail between consecutive runs.
• Coverage: Percentage of known tenants assessed in the last 30 days.
• Assessment identity health: Certificate or key age and any sign-in from outside the expected runner.

SCuBA checks configuration, not activity. Pair it with monitoring of the changes it would catch late: new consent grants (ATT&CK T1528 Steal Application Access Token), new role assignments (T1098.003 Additional Cloud Roles), new forwarding rules (T1114.003 Email Forwarding Rule) and sign-ins to cloud accounts (T1078.004 Cloud Accounts). See the Entra ID Sign-In and Audit Log Detection Guide and the Google Workspace Audit Log Detection Guide.

Framework mapping

Each baseline policy carries its own 800-53 and ATT&CK mapping, and ScubaGear publishes a consolidated SCuBA to NIST SP 800-53 Rev. 5 FedRAMP High mapping file, which helps teams reuse results in an authorisation package. See the FedRAMP Guide.

Common failure patterns

• Running once: A single assessment before an audit, with no schedule, misses drift and baseline version changes.
• Exceptions without expiry: Omissions and Conditional Access exclusions added to get a green report, with no rationale review, quietly become permanent blind spots.
• Overpowered, forgotten runner: A service principal with Sites.FullControl.All and a two-year certificate created for a pilot and never removed.
• Skipping non-production tenants: Test and legacy tenants left unassessed because a directive does not require them, despite the Midnight Blizzard precedent.
• Licence gaps reported as failures: Teams chase controls that need Entra ID P2 or Defender for Office 365 without recording a compensating third-party control as an omission, so reports stay red and lose credibility.
• Treating a pass as security: A tenant that passes every SHALL policy can still be compromised through consented apps or stolen tokens; configuration assessment complements detection, it does not replace it.

Practitioner checklist

• Inventory every Microsoft 365 and Google Workspace tenant, including test and legacy tenants, with a named owner.
• Create a dedicated, least-privilege assessment identity with a vaulted certificate or key and sign-in alerting.
• Run ScubaGear or ScubaGoggles once without configuration to capture current posture, and keep the JSON result.
• Build the YAML configuration in version control, with every exclusion and omission carrying a rationale, owner and expiry.
• Work the ActionPlan.csv backlog in pilot rings, using report-only Conditional Access and pilot organisational units.
• Schedule weekly runs and after change windows, compare results with Invoke-SCuBADiff, and alert on regressions.
• Track baseline releases and policy version changes in the repositories.
• Report SHALL pass rate, exception counts and coverage to leadership monthly.

Standards and references

• CISA: Secure Cloud Business Applications (SCuBA) Project
• CISA: BOD 25-01 Implementation Guidance, Implementing Secure Practices for Cloud Services
• CISA: ScubaGear (GitHub)
• CISA: ScubaGoggles (GitHub)
• CISA: M365 Secure Configuration Baseline for Microsoft Entra ID
• CISA: Google Workspace Common Controls Baseline
• CISA: ScubaGear Configuration File Documentation
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Microsoft 365 Security Guide · Google Workspace Security Guide · Entra ID Security Guide · SaaS and OAuth App Governance Guide · CISA Binding Operational Directives Guide