Steampipe and CloudQuery Guide: Safe Cloud Inventory Queries and Misuse Detection

SQL tools that read every cloud API are among the most useful defender tools and, in the wrong hands or with the wrong credentials, among the fastest ways to map an estate. CloudQuery and Steampipe both turn cloud and SaaS APIs into tables you can query with SQL, which makes asset inventory, posture checks and investigations far quicker than writing scripts. Both also need broad read credentials across every account they cover, and the query traffic they generate looks much like an intruder's discovery phase. This guide explains how each tool works, how to deploy them with least privilege, how to protect the data they collect, and how to tell sanctioned inventory runs from misuse in CloudTrail and other logs. It is for cloud security engineers, platform teams and detection engineers. Tools are listed alphabetically in the body of the page; order is not a ranking.

This is an independent guide from NHI Mgmt Group, based on CloudQuery's public documentation and Steampipe's public documentation as of November 2026. It is not sponsored or reviewed by CloudQuery or Turbot. Product features change, so confirm settings against each project's current documentation.

Key takeaways

• Two models, one risk. CloudQuery syncs cloud metadata into a destination such as PostgreSQL or a data warehouse, while Steampipe queries the APIs live on demand; both hold read credentials that span every account in scope.
• The collector identity is a high-value non-human identity. A role that can list and describe everything in an organisation is a ready-made reconnaissance kit, so scope it, give it short-lived credentials and monitor it like a privileged account.
• Read-only is not harmless. Steampipe's AWS plugin documentation suggests the AWS ReadOnlyAccess policy, which also covers many data reads; narrow it to what your tables need.
• Protect where the data lands. A CloudQuery destination or a Steampipe service database is a map of your estate, including security group rules and IAM policies, and deserves the same access control as the cloud consoles.
• Baseline the sanctioned runs. Known role names, session names, schedules and source networks make it easy to spot the same discovery pattern coming from anywhere else.

What the two tools do

Both projects map provider APIs to SQL tables through plugins. They differ in when the API calls happen and where the results live.

Where these tools sit among other open source projects is covered in the Open Source Cloud Security Tools Landscape Guide. This page goes deeper on running these two safely.

CloudQuery

CloudQuery describes itself as a cloud asset inventory that syncs infrastructure metadata into your data warehouse, and states that it runs on your infrastructure so the data does not pass through CloudQuery's servers. A sync is driven by YAML specifications for each source and destination. Security-relevant settings from the AWS plugin documentation:

• Authentication: environment variables, shared credential files, a per-account local_profile, AWS SSO sessions, compute roles on EC2, ECS and Fargate, and role assumption with role_arn, role_session_name and external_id.
• Organisations: the org block takes member_role_name (default OrganizationAccountAccessRole, and the docs advise creating dedicated read-only roles instead), admin_account, organization_units and skip_member_accounts. The admin identity needs organizations:ListAccounts, ListAccountsForParent and ListChildren.
• Scope reduction: skip_tables, skip_regions and skip_specific_apis limit what is called, and use_paid_apis defaults to false.
• Hub credential: cloudquery login is required to download plugins from CloudQuery Hub; for automation an API key is set in CLOUDQUERY_API_KEY, with a choice of "Syncs operations only" or "Read and write" scope and a mandatory expiry date.
• Run evidence: cloudquery sync --summary-location appends a JSON line per sync with the sync ID, start time and duration, which is useful for reconciling runs with CloudTrail.
• Availability: as of November 2026, some integrations are closed source and usage of the AWS plugin counts toward a plan's row quota, per CloudQuery's documentation.

Steampipe

Steampipe exposes plugins as a PostgreSQL-compatible database. Security-relevant settings from the Steampipe and AWS plugin documentation:

• Two run modes: steampipe query starts the database on the loopback address only and stops it when the session ends; steampipe service start runs it as a service on port 9193 with user steampipe and a generated password.
• Listen setting: the database listen option defaults to network in service mode. Set it to local, or pass --database-listen local, unless remote clients genuinely need access; the dashboard has a matching --dashboard-listen option.
• Password: set it with --database-password or STEAMPIPE_DATABASE_PASSWORD and store it in a secrets manager; --show-password prints it, so keep that out of shared terminals and logs.
• Credentials: the AWS plugin resolves credentials in order from the connection file (~/.steampipe/config/aws.spc), environment variables, the profile named in AWS_PROFILE, the default profile, then an EC2 instance role. Named profiles allow SSO and role assumption.
• Aggregators: an aggregator connection queries many accounts as one. The docs warn that three connections across 16 regions means the same list call runs 48 times, which is also what your detection will see.
• Availability: as of November 2026, Steampipe's docs describe a hosted option, Turbot Pipes, alongside the self-run CLI.

Deployment patterns and their risks

Securing the collector identity

The role or service account that CloudQuery or Steampipe uses is a non-human identity with organisation-wide reach. Treat it with the same care as any privileged machine identity.

1. Use a dedicated identity per tool and purpose. Do not reuse OrganizationAccountAccessRole or an administrator role. Create a member role whose trust policy names only the collector role, and add an external ID condition. See How to review AWS cross-account trust policies for confused deputy.
2. Grant metadata reads, not data reads. AWS ReadOnlyAccess includes reads of data such as objects and table items. Start from the narrower AWS managed policies for audit or view-only access, then add the specific actions your tables need, using the per-table permission lists in each plugin's documentation. The AWS IAM Policies and Roles Guide covers policy design.
3. Prefer short-lived credentials. Run from a compute role, SSO or OIDC federation instead of static access keys. If a key is unavoidable, keep it in a secrets manager and rotate it. See the AWS Access Keys Security Guide.
4. Name sessions consistently. Set role_session_name (CloudQuery) or a named profile with a fixed session name (Steampipe) so CloudTrail shows a predictable value you can baseline.
5. Apply the same rules outside AWS. Use the Reader role or narrower custom roles on Azure subscriptions and Viewer or narrower roles in Google Cloud, assigned to a dedicated service principal or service account with federated credentials.
6. Protect the tool's own credentials. A CloudQuery Hub API key with "Read and write" scope, or a Steampipe service password, is a secret in its own right. Scope it, expire it and store it centrally.

Organisation-wide guardrails for these roles are covered in the AWS Organizations and SCPs Guide.

Protecting the collected data

An inventory database answers exactly the questions an intruder asks: which security groups allow 0.0.0.0/0, which roles trust external accounts, which buckets are public, which instances run old images. Protect it accordingly.

• Network: no public listener for a Steampipe service or a CloudQuery PostgreSQL destination; private subnets and security groups limited to named clients.
• Access: per-user database roles rather than one shared login, and read access limited to teams that need it.
• Encryption: TLS for client connections and encryption at rest for destination stores and snapshots.
• Sensitive columns: some tables can include values such as environment variables or tags that hold secrets; use skip options to drop tables you do not need and scan the destination for credentials. See the Secrets Management Guide.
• Retention: keep historical CloudQuery snapshots only as long as investigations and audits need them.

Detecting misuse and unsanctioned runs

Inventory tools produce the same API pattern as an intruder's discovery phase: ATT&CK T1580 Cloud Infrastructure Discovery, T1526 Cloud Service Discovery, T1087.004 Cloud Account discovery and T1619 Cloud Storage Object Discovery. The sanctioned run is a known identity from a known place at a known time; anything else is worth a look.

The TruffleNet campaign shows why this matters: attackers used GetCallerIdentity and GetSendQuota calls from more than 800 hosts to test stolen AWS keys at scale. A baseline of your own sanctioned discovery traffic makes such calls stand out. See the TruffleNet stolen AWS keys case, the AWS CloudTrail Guide and How to Detect aws access key use from unusual locations.

Rolling out a sanctioned inventory capability

1. Decide the model. Choose scheduled sync to a store (historical record, joins with other data) or live query (fresh answers, nothing stored), or both for different teams.
2. Pilot in one non-production account. Create the dedicated role, run the tool, and record exactly which API actions it calls from CloudTrail. Use that list to tighten the policy.
3. Write the baseline. Document role name, session name, source network, schedule and expected call volume. Turn it into detection allowlist entries scoped to all five attributes, not just the role name.
4. Expand by organisational unit. Add accounts in rings, watching for throttling and for alerts from GuardDuty or your SIEM that need tuning.
5. Lock down the data store. Apply the network, access and retention controls above before analysts get access.
6. Roll back cleanly. If the collector causes throttling or a permission is too broad, disable the schedule, remove the member role from the affected OU through your infrastructure as code, and keep the CloudTrail baseline for comparison.

Decision guidance

• Need history and trends: a sync-and-store model suits compliance evidence and change tracking, at the cost of protecting a new data store.
• Need fresh answers during an incident: live query avoids stale data and stores nothing long-term, at the cost of API load and throttling during large queries.
• Small organisations: one account or a handful of accounts, run interactively from a hardened admin workstation with SSO, local-only listen and no stored results, is usually enough.
• Large organisations: a central security account, dedicated member roles deployed by StackSets or equivalent, a protected destination, and the collector registered as a privileged non-human identity with an owner.
• Already have CSPM or CAASM: use these tools for questions the platform cannot answer and keep their identities as tightly scoped as the platform's. See the Cloud Security Posture Management (CSPM) Guide.

Detection and measurement

• Collector identity scope: number of actions granted versus actions actually used in the last 90 days (target: gap trending to zero).
• Credential age: share of collector credentials that are short-lived (target 100%).
• Coverage: percentage of cloud accounts and subscriptions included in the inventory.
• Exposure: count of inventory databases reachable from outside approved networks (target zero).
• Anomaly rate: discovery-pattern alerts from identities other than the collector, and time to triage each.

Framework mapping

Common failure patterns

• Running as an administrator: using OrganizationAccountAccessRole or an admin profile because it is already there, so the inventory tool can change what it reads.
• Static keys in config files: access keys pasted into aws.spc or a sync YAML file and committed to a repository; the EmeraldWhale case shows how quickly exposed configuration files yield cloud credentials.
• Service mode left on the network: a Steampipe service on a jump host listening on all interfaces with a shared password.
• Allowlisting by role name only: suppressing discovery alerts for the collector role from anywhere, so a stolen session looks routine.
• Unprotected warehouse: inventory tables in a broadly shared analytics schema.

Practitioner checklist

• Register each CloudQuery or Steampipe deployment with an owner, purpose and identity.
• Replace admin and ReadOnlyAccess grants with a dedicated metadata-read role per account.
• Restrict member role trust to the collector role and add an external ID.
• Remove static keys; use compute roles, SSO or OIDC federation.
• Set Steampipe service listen to local unless remote access is approved, and set a managed password.
• Scope CloudQuery Hub API keys to "Syncs operations only" with an expiry date.
• Put inventory stores on private networks with per-user access and encryption.
• Baseline collector activity by role, session name, source, schedule and volume, and alert on deviations.
• Alert on wide discovery patterns from any identity that is not the collector.

Standards and references

• CloudQuery: AWS source plugin documentation
• CloudQuery: Generate API key
• CloudQuery: Project repository and CLI reference
• Steampipe: Service mode documentation
• Steampipe: Options configuration reference
• Steampipe: AWS plugin documentation
• AWS: GuardDuty IAM finding types
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Open Source Cloud Security Tools Landscape Guide · Cloud IAM Review Checklist · Asset Inventory and CMDB Security Guide · Cloud Workload Identity Guide · Cross-Account and Cross-Tenant Access Guide