Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

Authorization rule indexing: what changed and why it matters


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 9079
Topic starter  

TL;DR: Authorization decision latency can be cut by redesigning the rule index twice, moving from policy-shaped storage to bitmap-based filtering and then to a simpler custom bitmap that reduced microbenchmark time to 6.6 microseconds, according to Cerbos. The deeper lesson is that data structure fit and allocation behaviour matter as much as raw algorithm choice when authorization sits in the request path.

NHIMG editorial — based on content published by Cerbos: an engineering post on rewriting authorization indexing for faster authorization decisions

By the numbers:

  • The standard evaluator microbenchmark improved from 43.8 µs to 6.6 µs after the index rewrite and custom bitmap work.

Questions worth separating out

Q: How should teams decide whether an authorization index is too expensive for inline evaluation?

A: Teams should look for three signals: rising allocation rates, growing garbage-collector time, and candidate-set filtering that creates multiple temporary structures per request.

Q: Why does data-shape fit matter so much in policy evaluation systems?

A: Data-shape fit matters because authorization engines are not just rule interpreters, they are query systems.

Q: What do security teams get wrong about bitmap-based authorization indexes?

A: They often focus only on intersection speed and ignore the memory model beneath it.

Practitioner guidance

  • Profile allocator behaviour in the request path Measure how much CPU is going to map creation, candidate intersection, and garbage collection before changing policy logic.
  • Model policy data by binding shape, not policy count Count unique routing tuples, duplicated payloads, and dimension cardinality so you can see whether your current representation is duplicating the same entitlement state across tenants or workloads.
  • Test candidate reduction under realistic policy density Benchmark the engine with production-like role, action, and resource distributions.

What's in the full article

Cerbos' full article covers the implementation detail this post intentionally leaves for the source:

  • The exact Go data structures used for the rule table before and after the rewrite.
  • Benchmark tables comparing v0.40, the rule-table version, roaring bitmaps, pooled roaring, and the custom bitmap.
  • The load-test setup that exposed garbage-collector pressure under sustained query volume.
  • The reasoning behind the metadata layer in the custom bitmap and why it fit the workload better.

👉 Read Cerbos' engineering post on rewriting authorization index performance →

Authorization rule indexing: what changed and why it matters?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 2 months ago
Posts: 8508
 

Authorization indexing is now an identity-control problem, not just a performance problem. Once authorization decisions sit in the request path, the index becomes part of the control plane for human and non-human identities alike. If the evaluator cannot answer quickly, teams either widen cache windows, relax enforcement, or move checks out of band. The practical implication is that performance debt can become governance debt.

A few things that frame the scale:

  • The average estimated time to remediate a leaked secret is 27 days, despite 75% of organisations expressing strong confidence in their secrets management capabilities, according to The State of Secrets in AppSec.
  • Organisations maintain an average of 6 distinct secrets manager instances, creating fragmentation that undermines centralised control, according to The State of Secrets in AppSec.

A question worth separating out:

Q: Should organisations optimise authorization engines before changing policy design?

A: Yes, but only after you separate policy logic from index cost. If the policy model is sound but the evaluator is burning CPU on allocation, the right move is to change the data structure or storage pattern first. If the policy itself is too complex, no index will fully hide that cost.

👉 Read our full editorial: Bitmap indexing for authorization decisions: why data shape wins



   
ReplyQuote
Share: