Should organisations benchmark human behaviour and AI activity together?

Why combined benchmarking is the right question

Benchmarking people and AI separately can be useful for local reporting, but it breaks down when both act inside the same workflow. The more important unit is the business process: who or what can reach it, what permissions it uses, and what sensitive actions it can trigger. A combined view helps teams compare exposure across roles, tokens, tools, and decision paths rather than treating AI as a separate silo.

That matters because AI activity is often operationally similar to human activity: it logs in, calls APIs, reads records, writes outputs, and sometimes escalates into higher-value tasks. If the control model only measures human behaviour, then anomalous machine actions can look “normal” by omission, especially when automation is delegated to the same applications, data stores, or approval chains.

What a shared benchmark should measure

A useful benchmark does not ask whether AI is “more risky” in the abstract. It compares the same dimensions across both populations: access scope, frequency of sensitive operations, exception rates, privileged paths, and how often the actor touches regulated or business-critical data. That makes the metric actionable because it exposes where machine activity exceeds human norms, or where humans still hold the highest-risk privileges.

For that reason, the benchmark should be tied to observables, not just labels. Track authenticated sessions, token issuance, cross-system reach, failed authorisations, sensitive-data reads, and high-impact writes. If the AI can complete a workflow without a person in the loop, its behaviour should be measured as a first-class operational actor, not as background automation.

Good comparative baselines often align with least-privilege design and with CIS Benchmarks style hardening thinking: measure whether access is bounded, consistent, and auditable before asking whether the system is “working well.” The same logic also fits zero trust principles, where the trust decision depends on current context rather than on whether the actor is human or machine.

Where combined benchmarking becomes misleading

Combined metrics become noisy when organisations compare unlike behaviours or mix convenience automation with privileged decision-making. A high-volume AI assistant that drafts responses is not equivalent to an agent that can approve payments, change entitlements, or query production data. If the benchmark does not separate low-impact from high-impact activity, it can hide the true concentration of privilege and overstate the safety of routine usage.

The other common failure is treating benchmark parity as proof of safety. Similar behaviour does not mean similar risk, because the blast radius depends on what each actor can access, the quality of its controls, and how quickly it can be revoked. A machine identity that issues thousands of low-risk reads may be less concerning than a small set of high-privilege actions executed by either a person or an agent.

Risk and Threat Considerations

Benchmarking across humans and AI can create false comfort if the data model ignores hidden privilege, shared credentials, or delegated access. The main risk is that machine activity blends into ordinary workflow telemetry, so excessive access, abnormal tool use, or credential abuse is not separated from legitimate business traffic.

Failure mechanism: a combined metric is only useful when the underlying events are normalised for access scope and action criticality. If teams aggregate counts without separating privilege level, session source, or data sensitivity, they can miss overprivileged automation, compromised tokens, or agent-driven misuse.

Impact: defenders may under-detect abuse, overestimate control effectiveness, and delay containment until the same workflow has been used to reach sensitive systems or records.

Practitioner Guidance

What to prioritise: compare humans and AI at the workflow and privilege layer first, not at the raw activity-count layer. The most useful benchmark is the one that tells you which actor has more reach into sensitive systems, not which one generates more events.

What to verify: confirm that the benchmark distinguishes identity type, token provenance, authorisation scope, and data sensitivity. If those fields are missing, the metric is too blunt to support security decisions.

Decision rule: if a machine can execute the same sensitive action as a person, it should be measured with the same rigor, and if it can do it faster or at scale, its anomaly thresholds should be tighter, not looser.

Practitioner takeaway: the right benchmark is comparative, but it must be privilege-aware and workflow-aware; otherwise you compare volume instead of exposure.

Related resources from NHI Mgmt Group

• What breaks when organisations cannot distinguish human from AI agent activity?
• How should organisations think about AI email security and user behaviour monitoring together?
• Should organisations treat AI-assisted automation the same as human admin activity?
• How should security teams govern non-human identities at scale?