IaC Scanning Buyer's Guide: Evaluation Criteria

Most infrastructure-as-code (IaC) scanning purchases are judged on the number of built-in checks, when what decides their value is whether findings reach the right engineer before merge, survive module and variable resolution, and block the few misconfigurations that matter without drowning teams in noise. IaC scanners read Terraform, OpenTofu, CloudFormation, ARM and Bicep templates, Kubernetes manifests, Helm charts and Dockerfiles, and flag insecure settings before anything is deployed. The category ranges from free open source engines to commercial platforms that add policy management, drift detection, fix pull requests and links to runtime cloud posture. This vendor-neutral buyer's guide is for cloud security, platform engineering and AppSec teams writing requirements or running an RFP. It covers what to define first, capability areas, scoring criteria, questions that expose weak answers, red flags and a proof of concept.

Key takeaways

Coverage of your real code, not the datasheet list, is the first filter. Count your Terraform modules, variable files, Helm charts and templates, then test whether the product resolves modules, variables and loops before evaluating a rule. Unresolved values are where false negatives hide.
Plan scanning and source scanning answer different questions. Source scanning gives line-level feedback in a pull request; scanning the Terraform plan JSON sees computed values and module output. Strong programmes use both, and the plan file needs protecting because Terraform states that secrets in configuration are stored "in its state and plan files".
Custom policy is where the product earns its cost. Built-in checks cover generic benchmarks. Your tagging, region, encryption key and identity rules need policy-as-code you can write, test and version, ideally in a language your teams already use, such as Rego.
Suppression governance decides whether the tool is trusted. Inline skips with a reason, expiry and approver, plus a baseline for legacy findings, keep the gate enforceable. Silent, permanent suppressions turn the scanner into theatre.
The scanner holds privileged access. Commercial platforms need repository read access, often write access for fix pull requests, and sometimes cloud read roles for drift. Treat each integration as a non-human identity with an owner and least privilege.

Define requirements

Languages and frameworks: list every IaC format in use (Terraform and OpenTofu, CloudFormation and AWS SAM, ARM and Bicep, Kubernetes YAML, Helm, Kustomize, Ansible, Dockerfiles, CDK output) and the share of infrastructure each one manages. Products differ sharply outside Terraform and Kubernetes.
Where it must run: developer workstation and pre-commit, pull request checks, CI pipelines, the Terraform run platform (for example a run task in a hosted Terraform service or a pull request automation server), and on a schedule against default branches. See the Atlantis Security guide and the Terraform Cloud and HCP Security Guide.
Policy sources: which benchmarks you must evidence (CIS Benchmarks for your clouds, internal standards, regulatory baselines) and which organisation rules need custom policies. Write the top 20 rules you would enforce on day one.
Enforcement model: warn only, block on severity, or block on named policies. Decide who can approve an exception and for how long.
Connection to runtime: whether you need findings linked to deployed cloud resources and drift detection, which usually means the IaC scanner is part of a CNAPP or posture platform. See the CNAPP Buyer's Guide.
Existing tools: you may already run an open source engine in CI. Decide whether you are buying policy management and reporting on top of it, or replacing it. The Checkov and IaC Scanners Guide describes what the free engines already do.

Size changes the answer. A small team on one cloud and Terraform often gets most of the value from an open source scanner in CI, a short list of custom policies and SARIF output into the code host. A large organisation with several clouds, hundreds of repositories and a platform team publishing shared modules needs central policy distribution, exception workflow, module-level scanning, reporting by business unit and an API for evidence.

Capability areas

Where a scan runs determines what it can see. Score products on each point you need, not on one.

Scan point	What it sees	What it misses	Best used for
Editor and pre-commit	Raw files as written, instant feedback	Remote modules, variable values from pipelines, anything not on the laptop	Teaching developers, catching obvious issues early
Pull request (source scan)	The diff and the repository, with line numbers for inline comments	Values computed at plan time, data sources, some module outputs	The main enforcement gate for most teams
Plan scan (`terraform show -json`)	Resolved resource attributes in `resource_changes`, prior state and `resource_drift`	Line numbers unless mapped back to source; requires a plan run with credentials	Catching misconfigurations hidden behind variables and modules
Scheduled default branch scan	Everything merged, including code that pre-dates the tool	Changes made outside IaC	Baselines, reporting, new rules applied to old code
Drift and runtime correlation	Deployed resources compared with code	Depends on cloud read access and coverage of resource types	Finding console changes and mapping runtime findings back to the owning file

Parsing depth: module resolution (local, registry and Git sources), variable and tfvars evaluation, count and for_each expansion, and graph analysis that understands relationships, such as a bucket and its separate public access block resource. Context-aware graph checks reduce false positives from rules that look at one resource in isolation.
Policy engine: built-in checks mapped to benchmarks, custom policies in a general policy language such as Rego (the Open Policy Agent language) or in YAML or Python, policy unit tests, versioned policy bundles and the ability to scope policies by repository, environment or tag.
Secrets and identity checks: detection of hard-coded credentials in providers, variables and user data, and identity rules such as wildcard IAM actions, trust policies without conditions and long-lived access keys created in code. See IaC identity policy checks.
Developer experience: inline pull request comments, a clear fix for each finding, auto-fix pull requests where safe, and SARIF output so results land in the code host's own security view. SARIF 2.1.0 is an OASIS Standard, with each result carrying a ruleId, level, locations, suppressions and baselineState.
Exceptions and baselines: inline suppressions that require a reason, central exceptions with expiry and approver, and baselining of existing findings so only new issues block a merge.
Reporting and evidence: findings by repository, team, policy and severity over time, mapping to frameworks you report against, and export through an API.

Evaluation criteria

Weight these to your requirements. An illustrative split is 25% coverage and parsing accuracy, 20% custom policy and testing, 20% developer workflow and enforcement, 15% exceptions and reporting, 20% platform security, integration and cost.

Criterion	What good looks like	Weak signs
Format coverage	All your IaC formats parsed natively, including Helm rendering and Kustomize overlays, with a published list of supported resource types per provider	"Supports Terraform" with no detail on modules, providers or OpenTofu
Parsing accuracy	Resolves modules, variables and loops; offers plan JSON scanning with findings mapped back to source lines	Rules evaluated against unresolved expressions, producing both misses and noise
Built-in policy quality	Each check has an ID, rationale, severity, benchmark mapping and fix example; release notes list new and changed checks	Large check counts with duplicates and no change history
Custom policy	Policies as code in a repository, unit tested, versioned and promoted through environments; access to resolved values and resource relationships	Custom rules only through a web form; no tests; no way to reference related resources
Enforcement	Block by named policy or severity per repository, warn mode for rollout, consistent results in editor, CI and run platform	All-or-nothing gating; different results in different places
Exceptions	Suppressions need reason, owner and expiry; central exceptions approved and audited; expired exceptions fail the check again	Anyone can add a permanent skip comment
Secrets in IaC and state	Detects credentials in code and warns when sensitive values would be written to state and plan files; plan files handled without being stored by the vendor unless agreed	Plan JSON uploaded and retained by default
Runtime link and drift	Maps deployed resources and runtime findings to the owning file and team; reports drift from code	Runtime and IaC findings in separate products with no link
Output and integration	SARIF, JSON and JUnit output, code host security view integration, ticketing, API	PDF reports only; proprietary format
Platform security and exit	Least-privilege app integrations, SSO and role-based access, audit log of policy and exception changes, full export of policies and history	Personal access tokens with organisation-wide write; policies locked in the product

Where identity fits

IaC is where most cloud identities are created. A strong scanner checks roles, policies, service accounts, workload identity bindings and federated trust conditions as carefully as storage and network settings: wildcard actions on sensitive services, trust policies that accept any repository or branch, access keys created for users or machines, and service account keys generated in code. Ask vendors to show their identity rule set against your own modules, and see the Infrastructure as Code Security Guide.

The product itself is a non-human identity problem. Pull request comments and fix pull requests need a code host app with write scopes, plan scanning needs a pipeline that holds cloud credentials, and drift detection needs cloud read roles. Require app installations with fine-grained permissions and repository selection, short-lived tokens, federation from CI rather than stored cloud keys (see the GitHub Actions OIDC Federation Hardening Guide) and a named owner for every connector. Credentials left in code are still the most direct failure: the Toyota T-Connect key exposure involved an access key that sat in a public repository for years, exactly the class of finding a scanner with secrets detection should block at commit time. For secrets already written to state, see How to find secrets in Terraform state.

Framework mapping

Framework	Reference	How IaC scanning supports it
NIST CSF 2.0	PR.PS-01, PR.PS-06, ID.RA-01	Configuration management applied in code; security integrated into the development life cycle; misconfigurations identified and recorded before deployment
NIST SP 800-53 Rev. 5	CM-2, CM-3, CM-6, SA-11	Baseline configuration and configuration settings expressed and checked as code; change control at merge; developer testing evidence
CIS Controls v8.1	Control 4 (Secure Configuration of Enterprise Assets and Software), Control 16 (Application Software Security)	Secure configuration enforced before deployment and inside the development process
NIST SP 800-204D	Supply chain security in CI/CD pipelines	Security checks integrated into build, test and deploy stages for cloud-native applications

Questions to ask vendors

Which IaC formats and versions do you parse natively, and how do you resolve remote modules, variable files, count, for_each and Helm values? Show it on one of our modules.
Do you scan Terraform plan JSON, and if so, where is the plan processed and stored, for how long, and how are sensitive values in it handled?
How are custom policies written, tested and versioned? Can a policy reference related resources and resolved values? Can we run the same policy bundle in the editor, CI and the run platform?
How do suppressions work: is a reason required, can they expire, who approves them, and is every change audited?
How do you baseline existing findings so only new issues block merges, and how do you report the legacy backlog?
Which identity and secrets checks are included, and can you show findings for wildcard IAM actions, unconstrained federated trust and credentials in provider blocks?
Which code host, CI and cloud permissions does each integration require, which are write permissions, and can they use short-lived tokens or federation?
How are new and changed built-in checks communicated, and can we pin a policy version so results do not change without notice?
What does the licence count (developers, repositories, resources scanned, cloud accounts) and what happens to cost as code grows?

Red flags

Check counts used as the headline, with no evidence of parsing accuracy on modules and variables.
Different results for the same commit in the editor, the CI job and the dashboard.
Suppressions that never expire and need no reason, or exceptions edited without an audit trail.
Plan files or state uploaded to the vendor by default, with no data handling statement.
Integrations that need organisation owner rights or a long-lived personal token, or cloud roles broader than read-only for drift.
No SARIF or API export, or custom policies that cannot be exported at contract end.

Proof of concept

Scope it: three to five representative repositories covering your main IaC formats, including one shared module repository, one Helm or Kustomize repository and one legacy repository with known issues. Agree weights and pass criteria first.
Connect with least privilege: install each product through a dedicated app identity limited to the trial repositories, use federation for any cloud access, and record every permission requested.
Seed known issues: add a set of deliberate misconfigurations, including some only visible after variable or module resolution (a public bucket set through a variable, an IAM policy with a wildcard built in a module, a credential in a provider block). Measure true positives, false positives and misses for each product.
Compare source and plan scanning: run both on the same change and record which issues each finds and whether plan findings map back to source lines.
Write three custom policies: one tagging rule, one identity rule and one rule involving two related resources. Time how long each takes, add unit tests and run them in CI.
Test the gate in warn mode, then enforce: enable blocking on two named policies in one repository, raise an expiring exception, let it expire and confirm the check fails again.
Test exit and roll back: export findings, policies and exceptions, switch the gate back to warn, revoke every integration token and role, and confirm the product loses access.

Keep measuring after go-live: share of IaC repositories with a blocking check, new high-severity findings merged per month (target zero), median time from finding to fix, number and age of active exceptions, and custom policy coverage of your top organisation rules.

Standards and references

How NHI Mgmt Group can help

We provide independent requirements, RFP and evaluation support. Browse vendors in our products directory or contact us.

Infrastructure-as-Code Scanning Buyer’s Guide: How to Evaluate IaC Security Tools