Notifications
Clear all
Topic starter
02/08/2026 12:06 pm
TL;DR: Smarter systems still fail because knowledge is dispersed, tacit, and time-sensitive, so centralized data and autonomous workflows often replace judgment with false certainty, according to Tonic. The design challenge is not making agents omniscient, but building controls that work when no system ever has the full context.
NHIMG editorial — based on content published by Tonic: The Knowledge Problem, the hidden reason smart systems fail
Questions worth separating out
Q: How should security teams govern context access for autonomous workers?
A: Security teams should treat context as a separate control surface from action.
Q: Why do AI agents and automation tools break down in complex organisations?
A: They break down because complex organisations distribute knowledge across people, systems, and moments.
Q: What do security teams get wrong about centralising data for smarter decisions?
A: Teams often assume that more centralised data automatically creates better decisions.
Practitioner guidance
- Map where automation depends on tacit context Identify workflows where operators routinely override the system because the documented state is incomplete, wrong, or stale.
- Add explicit human escalation points Require intervention whenever an agent or workflow cannot prove that it has enough context to proceed safely.
- Constrain autonomy with bounded signals Give systems narrow, verifiable signals such as policy state, risk thresholds, and task scope instead of asking them to reason over an entire world model.
What's in the full article
Tonic's full article covers the conceptual argument and examples that this post intentionally leaves for the source:
- The Hayek framing behind the knowledge problem and why it matters for system design
- The sequence of examples showing how local context gets lost in centralised platforms
- The case for signals-based coordination in agentic workflows rather than global understanding
- The next-part teaser that connects this concept to omniscient-agent design
👉 Read Tonic's analysis of the knowledge problem in smart systems →
The knowledge problem in agentic systems: what teams are missing?
Explore further
02/08/2026 1:11 pm
False certainty is the real control failure in autonomous systems. The article’s strongest point is that centralised platforms often create confidence without context, which is a governance weakness rather than a technical achievement. In identity and access terms, this is what happens when policy engines, automation, or AI workflows are asked to decide without enough operational knowledge. Practitioners should treat overconfident automation as a control gap, not an optimisation.
A question worth separating out:
Q: Who is accountable when automated privacy workflows make the wrong decision?
A: Accountability remains with the organisation, not the workflow. Privacy, security, legal, and system owners must define decision boundaries, review thresholds, and escalation paths so automation supports policy enforcement instead of replacing human responsibility for sensitive cases.
👉 Read our full editorial: The knowledge problem is why smarter systems still fail
