Once you collect non-human identities from more than one tool, the same service account shows up three times with three owners, and a key appears in the secrets manager that no identity system admits to. Reconciliation is the step that turns raw discovery output into an inventory you can act on: one record per identity, every credential attached to the right identity, conflicting attributes resolved by rule, and the gaps between sources treated as findings. This guide covers the canonical identifiers each platform uses, match rules that avoid false merges, how to join identity, cloud and secrets sources at the credential level, which source wins for which attribute, and how to run reconciliation as a daily process. It is for identity engineers, security data teams and anyone building an NHI inventory, a CMDB extension or an identity security platform rollout.
Key takeaways
- Match on immutable identifiers, never names. AWS, Google Cloud and Entra all allow a name to be reused after deletion while the underlying unique ID changes; joining on names merges a new identity with a dead one.
- Join credentials, not just identities. Access key IDs, client secret key IDs, service account key IDs and certificate thumbprints are the threads that tie a secret in a vault or a scanner finding to the identity that owns it.
- Decide a source of truth per attribute. Permissions come from the platform, owner from the ownership register, last used from authentication logs, purpose from the owner; no single tool is authoritative for everything.
- Unmatched records are the point. A credential with no identity, an identity with no owner and an identifier seen in logs but in no inventory are the highest-value outputs of reconciliation.
- Keep history. Store every source observation with its timestamp so you can explain why records merged, and undo a bad merge.
Why sources disagree
Each tool sees the estate from one angle. A cloud provider lists roles and keys; an identity provider lists applications and service principals; a secrets manager lists stored values and their paths; a secret scanner lists exposed strings; a CMDB lists applications and owners; a PAM vault lists onboarded accounts. They disagree for predictable reasons:
- Different object models: Microsoft Entra has one application object in the home tenant and a separate service principal in each tenant where the app is used, while a managed identity has a service principal and no application object. Tools that count either one will report different totals.
- Name reuse: AWS friendly names can be reused after deletion, and a new IAM user with an old name gets a different unique ID. Google Cloud gives a recreated service account with the same email a new unique ID and does not carry over the old bindings, which appear with a
deleted:prefix. - Timing: AWS can generate a credential report at most once every four hours; last accessed data appears in the console within about four hours; collectors run on different schedules.
- Partial coverage: the AWS credential report covers only the first two access keys per user; GitHub organisation owners cannot see classic personal access tokens in the active tokens view.
- Human entry: CMDB and spreadsheet owners, purposes and environments are typed by people and drift.
Canonical identifiers by platform
Choose one immutable primary key per platform and store the human-readable name as an attribute.
| Platform | Primary key | Also store | Watch for |
|---|---|---|---|
| AWS IAM user or role | Unique ID (prefix AIDA for users, AROA for roles) from get-user or get-role | ARN, account ID, path | Console does not show unique IDs; session ARNs (assumed-role/role/session) map back to the role, not a new identity |
| AWS access key | Access key ID (prefix AKIA; ASIA is temporary STS) | Owning user unique ID, status, created date | Temporary keys are not inventory items; record the role instead |
| Entra application | Application object ID plus appId | Home tenant, display name | One application, many service principals across tenants |
| Entra service principal | Service principal object ID in that tenant | appId, type (application, managed identity, legacy) | Managed identities and legacy service principals have no application object |
| Entra credential | Credential keyId | Type (secret or certificate), expiry, origin (application or service principal) | Credentials can sit on either object |
| Google Cloud service account | Unique ID | Email, project | Email reuse after deletion; default service accounts with Editor |
| Google Cloud key | Key ID | Service account unique ID, created and expiry dates | Keys outlive the people who created them |
| Kubernetes ServiceAccount | Object UID | Cluster, namespace, name | Same namespace and name in every cluster; cluster must be part of the key |
| Active Directory account | objectGUID | SID, sAMAccountName, SPNs | Renames keep the GUID; restores from backup can surprise you |
| SaaS integration | Platform's immutable app or user ID | Display name, connected app or client ID, tenant | Many SaaS APIs expose only names; record tenant plus name plus creation date as a composite |
Build a global key as platform:tenant:native-id (for example the AWS account ID with the role unique ID, or the Kubernetes cluster ID with the ServiceAccount UID) so records from different estates can never collide.
Match rules that avoid false merges
Apply rules in tiers, strongest first, and record which rule produced each link.
| Tier | Rule | Example | Action |
|---|---|---|---|
| 1 Deterministic | Same platform and native immutable ID | AWS role unique ID in collector and in Access Analyzer finding | Merge automatically |
| 2 Credential link | Credential identifier observed in two sources | Access key ID in the credential report and in a secrets manager tag or CloudTrail event | Attach credential to identity automatically |
| 3 Cross-reference | One source records the other's ID | CMDB record holding an Entra appId; Terraform state holding a role ARN | Link automatically; verify the referenced ID exists |
| 4 Composite | Tenant plus name plus creation date | SaaS integration user known only by name | Propose link; require review |
| 5 Heuristic | Similar names, same owner tag, same repository | svc-billing-prod and billing-prod-sa | Suggest only; never auto-merge |
Never merge across platforms on name alone. Identities that represent the same workload on different platforms (an AWS role and a Google Cloud service account used by one service) should be linked as related records under a shared application, not merged into one identity.
Joining identity, cloud and secrets sources at the credential level
The hardest reconciliation is between what the identity platform says exists and what is actually stored or exposed. Work outwards from credential identifiers:
- From the platform: list each identity's credentials with their identifiers and dates (AWS
ListAccessKeys; EntrapasswordCredentialsandkeyCredentials; Google Cloud key list). - From the secrets manager: for each stored secret, record the path, consumers and any tag or metadata that names the credential identifier. Where the value itself must be inspected to find its identifier (for example the public access key ID inside an AWS key pair), do it inside the secrets manager or scanner and export only the identifier.
- From scanners: take the provider, credential identifier or fingerprint, and validity status of each finding.
- From logs: take the credential identifiers seen authenticating, such as the access key ID in CloudTrail, the
keyIdin Entra application credential activity and the service account key ID in Google Cloud audit logs. - Join on the identifier and classify every credential into one of the states below.
| State | Platform | Vault | Scanner | Logs | Meaning and action |
|---|---|---|---|---|---|
| Managed | Yes | Yes | No | Yes | Healthy; check rotation date |
| Unvaulted | Yes | No | No | Yes | Credential used but stored somewhere unknown; find and vault it |
| Exposed | Yes | Any | Yes, valid | Any | Rotate now; see the Leaked Credential Response Playbook |
| Orphaned secret | No | Yes | Any | No | Vault holds a credential the platform no longer knows; delete after confirming |
| Dormant | Yes | Any | No | No | Candidate for disablement; see the NHI Dormancy Detection guide |
| Ghost | No | No | Any | Yes | Identifier authenticating that no inventory lists; investigate as a discovery gap or intrusion |
A worked example
A payments service runs on AWS. The cloud collector reports an IAM user with unique ID beginning AIDA and two access keys. The secrets manager holds one entry whose metadata names the first key ID. CloudTrail shows the second key ID signing requests from a build server every night, and a scanner reports that second key, still valid, in an old repository. The CMDB lists an application called "Payments" with an owner who left last year, and the PAM vault knows nothing. Reconciliation produces one identity record, two credential records (the first managed, the second exposed and unvaulted), a stale owner conflict and a work item to move the build server to a role. Without the credential-level join, three tools would each have reported a single, apparently healthy service account.
Resolving conflicting attributes
When sources disagree on an attribute, pick the winner by rule, not by whichever collector ran last.
| Attribute | Authoritative source | Fallback | Notes |
|---|---|---|---|
| Existence and status | The platform API | None | If the platform says deleted, the record is deleted, whatever the CMDB says |
| Permissions | Platform policy and role assignments | Analysis tools derived from them | Store effective and assigned separately |
| Last used | Authentication logs and native usage reports | Tool-derived activity | Take the latest date from any trustworthy source; note each source's lag |
| Owner | Ownership register or attested owner | Creator, IaC repository owners, cloud tags | Creator is a hint, not an owner; people leave |
| Purpose and criticality | Owner attestation | CMDB application link | Review on attestation cycle |
| Environment | Account, subscription or project classification | Tags and naming convention | Naming is the weakest signal |
| Credential storage location | Secrets manager | Scanner and IaC findings | Multiple locations is itself a finding |
Survivorship rules should be explicit configuration, versioned and reviewed, so that an auditor can see why the inventory shows a particular owner. See the NHI Ownership Guide for the ownership model this feeds.
Running reconciliation as a process
- Stage raw observations. Land each collector's output unchanged with source, run time and run status. Never edit raw data.
- Normalise. Map each source to a common schema: identity, credential, trust relationship, observation. Build the global key.
- Match. Apply tiered rules; write links with rule ID and confidence.
- Resolve. Apply survivorship rules to produce the current record; keep the previous version.
- Classify and route. Assign credential states and open work items: ownership tasks, vaulting tasks, rotation, dormancy review, discovery gaps.
- Publish. Expose the reconciled inventory to IGA, PAM, SIEM and the CMDB through one interface so downstream tools stop maintaining their own lists.
Testing and rollback: run the pipeline in shadow mode for two to four weeks, publishing to a report but not to downstream tools, and sample merged records by hand each week. Because every link records its rule, a bad rule can be reverted and the affected merges undone from history without re-collecting.
Small and large organisations
- Small estates: a database or even a well-structured spreadsheet with tier 1 to 3 rules and the credential state table is enough. Most value comes from the ghost and unvaulted states.
- Large estates: expect tens of thousands of identities and many tenants. Use a data platform or an identity security product, but insist that it exposes match rules and source lineage. Opaque merges cannot be audited. The NHI Security Platform Buyer's Guide lists evaluation questions.
- After mergers: run reconciliation per acquired estate first, then link at the application level; tenant boundaries and naming conventions will differ.
Measuring reconciliation quality
- Match rate: share of source records linked by tier 1 to 3 rules (target above 95 percent for cloud and IdP sources).
- Review queue age: median days that tier 4 and 5 proposals wait for a decision.
- Ghost count: credential identifiers seen in logs with no inventory record; should trend to zero.
- Unvaulted credentials: active platform credentials not found in any secrets manager.
- Owner conflict rate: records where two sources name different owners.
- Merge error rate: merges reverted after review, from weekly sampling.
Feed ghost and exposed states to detection: a ghost identifier authenticating from an unusual network is a hunting lead. See the Identity Threat Hunting Guide.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST CSF 2.0 | ID.AM-02, PR.AA-01, PR.AA-05 | Maintained inventories; managed identities and credentials; reviewed permissions |
| NIST SP 800-53 Rev 5 | CM-8, AC-2, IA-5, PM-5 | Accurate component inventory without duplicates; account and authenticator management |
| CIS Controls v8.1 | Safeguards 5.1 and 5.5 | Account and service account inventories with owner, purpose and review date |
| OWASP Non-Human Identities Top 10 2025 | NHI1 Improper Offboarding, NHI2 Secret Leakage, NHI9 NHI Reuse | Reconciliation exposes orphaned, leaked and shared identities |
| MITRE ATT&CK | T1078.004 Cloud Accounts, T1098.001 Additional Cloud Credentials | Ghost identifiers and unexpected new credentials |
Common failure patterns
- Merging on display name. Recreated identities inherit the history, owner and exceptions of deleted ones, and the real new identity looks reviewed when it is not.
- Counting Entra applications and service principals as the same thing. Multitenant apps then look like one identity when there is a service principal with its own consent in each tenant.
- Trusting a single "unused" signal. The Cloudflare breach began with one service token and three service account credentials that were not rotated after the Okta breach because they were wrongly believed to be unused. Reconcile usage from several sources before deciding.
- Overwriting history. Without stored observations, a bad merge cannot be explained or undone, and auditors cannot see why ownership changed.
- Ignoring credentials with no identity. Orphaned vault entries and scanner findings without a matching identity are dropped as noise. The leaked GitHub App private keys research found hundreds of keys still working for Apps nobody was monitoring.
- Letting each tool keep its own list. PAM, IGA and the CMDB continue to diverge unless they consume the reconciled inventory.
Practitioner checklist
- Pick an immutable primary key per platform and build a global
platform:tenant:native-idkey. - Store every raw observation with source and timestamp; never edit raw data.
- Implement tiered match rules and record the rule behind every link.
- Collect credential identifiers from platforms, vaults, scanners and logs, and join on them.
- Classify every credential as managed, unvaulted, exposed, orphaned, dormant or ghost.
- Write survivorship rules per attribute and version them.
- Run in shadow mode with weekly sampling before publishing to downstream tools.
- Route ghost and exposed records to the SOC the same day.
- Track match rate, ghost count, unvaulted count and owner conflicts monthly.
Standards and references
- NIST: SP 800-53 Rev 5 Security and Privacy Controls
- OWASP: Non-Human Identities Top 10 2025
- AWS: IAM identifiers
- Microsoft Learn: Application and service principal objects in Microsoft Entra ID
- Google Cloud: Service accounts overview
- Microsoft Learn: Usage and insights in Microsoft Entra
- AWS: Getting credential reports for your AWS account
- CIS: Control 5 Account Management
Related NHI Mgmt Group resources: NHI Discovery Across Cloud, SaaS, CI/CD and On-Premises Environments · Identity Threat Hunting Guide · Service Account Inventory Template · NHI Ownership Guide