Subscribe to the Non-Human & AI Identity Journal

Notifications
Clear all

Pre-filtering vs post-filtering: what changes for access control?


(@nhi-mgmt-group)
Member Moderator
Joined: 1 year ago
Posts: 5855
Topic starter  

TL;DR: Centralised authorization only solves part of the problem if applications still fetch broad datasets and filter later, because post-filtering creates over-fetching, pagination drift, and brief in-memory exposure, according to Cerbos. Pre-filtering pushes authorization conditions into the query plan, making data access more efficient and safer by design.

NHIMG editorial — based on content published by Cerbos: pre-filtering versus post-filtering authorization for data access

Questions worth separating out

Q: How should teams implement authorization-aware filtering in data queries?

A: Teams should compile policy into datastore queries whenever possible so the database enforces access before rows enter the application.

Q: Why does post-filtering create security and scaling problems?

A: Post-filtering forces the application to handle a broader dataset than the user is allowed to see, which increases network load, memory pressure, and the chance of accidental exposure.

Q: How do organisations know whether pre-filtering is actually working?

A: Look for reduced row retrieval volumes, stable pagination behavior, and the absence of unauthorized records in application memory, logs, or caches.

Practitioner guidance

  • Move authorization into the query path Translate policy conditions into datastore-native filters so the database returns only rows already constrained by access rules.
  • Audit pagination after filtering Test whether page size, offsets, and result counts remain stable when authorization conditions are applied before retrieval.
  • Reserve hybrid verification for high-assurance paths Add a final authorization check after pre-filtering only where policy complexity, sensitivity, or regulatory exposure justifies the added runtime cost.

What's in the full article

Cerbos' full article covers the operational detail this post intentionally leaves for the source:

  • A worked example of partial evaluation turning policy conditions into an executable query plan
  • The AST structure used to generate database filters from authorization logic
  • A concrete walkthrough of translating the generated filter into SQL WHERE clauses
  • The hybrid pre-filter plus final PDP verification pattern for high-assurance systems

👉 Read Cerbos' guide to pre-filtering versus post-filtering authorization →

Pre-filtering vs post-filtering: what changes for access control?

Explore further

View Full Forum →  |  NHI Foundation Course →



   
Quote
(@mr-nhi)
Member Moderator
Joined: 1 month ago
Posts: 5343
 

Source-level authorization is the real control boundary, not the application loop. Post-filtering keeps the decision too late in the request path, which means the application becomes a temporary custodian of data the user should never have received. That is a governance flaw, not just an implementation inconvenience, because it moves enforcement away from the place that can most reliably constrain exposure. Practitioners should treat the retrieval layer as part of the authorization surface, not as a neutral transport step.

A few things that frame the scale:

  • 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools, according to the Ultimate Guide to NHIs.
  • 71% of NHIs are not rotated within recommended time frames, increasing the risk of compromise over time, according to the Ultimate Guide to NHIs.

A question worth separating out:

Q: Who is accountable when authorization logic is split between the application and the data layer?

A: Accountability sits with the team that owns the end-to-end data access path, not just the policy author. If the application translates policy incorrectly or retrieves too much data before filtering, the control failed in architecture, implementation, or governance, and ownership must be clear across those layers.

👉 Read our full editorial: Pre-filtering authorization data at the source beats post-filtering



   
ReplyQuote
Share: