Tacit knowledge is now a control plane issue for AI agents

At a glance

What this is: This is an analysis of tacit knowledge in agentic systems and the claim that human judgment must be designed into AI workflows as a control element.

Why it matters: It matters to IAM and AI governance teams because escalation, approval, and exception handling determine where agent autonomy ends and accountable human control begins.

👉 Read Tonic's analysis of tacit knowledge and AI agent design

Context

Tacit knowledge is the unspoken context that experienced practitioners use to make decisions when written rules are incomplete. In agentic systems, that missing context becomes a governance problem because the agent can only act on explicit instructions unless the workflow deliberately provides a human escalation path. For identity and access programmes, that means autonomy design and approval design now need to be treated as one control surface, not separate concerns.

The core issue is not that agents are incapable of reasoning, but that reasoning without situational knowledge can still produce the wrong outcome. In identity-heavy environments, that shows up when access, patching, or incident actions are technically valid but operationally unsafe. The article’s framing is relevant to NHI and agentic AI governance because the same control gap appears whenever an automated system needs a tie-breaker that policy alone cannot supply.

Key questions

Q: How should security teams design escalation for AI agents that hit conflicting instructions?

A: Security teams should define pre-approved stop conditions for conflicts that the agent cannot resolve safely on its own. The handoff should include the competing signals, the relevant policy references, and the operational context so a human can decide quickly. That keeps autonomy intact for routine work while preventing unsafe rule-only decisions.

Q: Why do AI agents need human judgment in high-stakes workflows?

A: AI agents can follow explicit rules, but many important decisions depend on tacit knowledge that was never written down. Human judgment is needed when the right action depends on local context, business impact, or exceptions that policy cannot fully express. Without that layer, an agent may be correct procedurally and wrong operationally.

Q: What breaks when agentic workflows rely only on written policy?

A: They become vulnerable to correct but disastrous decisions. The system may choose the action that satisfies the rule set while ignoring timing, business criticality, or hidden operational dependencies. That is why policy-only automation should be limited to low-risk actions and paired with review paths for unresolved conflicts.

Q: Who is accountable when an AI agent makes a risky decision?

A: Accountability should rest with the organisation that authorised the agent, the human owner of the workflow, and the control process that allowed the behaviour. If an agent can act independently, the programme must preserve attribution, action logs, and policy decisions so audit and remediation are possible after the event.

Technical breakdown

Tacit knowledge and explicit rules in agentic workflows

Explicit knowledge is what can be written into policy, code, or prompts. Tacit knowledge is the practical judgment built from experience, local exceptions, and operational history. Agentic systems struggle when the decision depends on context that was never formalised, because the model can only follow what it can see. That creates a mismatch between procedural correctness and operational correctness. In access and response workflows, this is where an agent can make the right rule-based move and still choose the wrong business outcome.

Practical implication: build escalation paths for decisions that cannot be fully expressed as policy.

Signal collision and the limit of autonomous decision-making

A signal collision occurs when an agent receives two valid but conflicting instructions, such as patch now and do not interrupt a critical service. The agent can detect the contradiction, but it cannot infer the organisation’s unwritten priority order unless that judgment is encoded somewhere. This is where autonomy needs a boundary. The control point is not constant permission seeking, but a structured stop condition that hands the case to a human when the logic has exhausted itself.

Practical implication: define explicit stop conditions for high-impact workflows before they reach operational conflict.

Why a help button is a governance control, not a failure state

In agentic design, a help request should be treated as a controlled escalation, not as a sign that the system has broken. The question is whether the agent can package the issue with enough context for a human to decide quickly and safely. That means the workflow must preserve the conflicting signals, the policy basis, and the reason the agent stopped. For identity programmes, this is similar to exception approval logic in PAM and JIT models, where the value lies in bounded escalation rather than unreviewed action.

Practical implication: log the conflicting signals and the escalation rationale so the human decision is reviewable later.

NHI Mgmt Group analysis

Tacit knowledge is now an AI governance requirement, not a soft skill. Agentic systems fail when organisations assume all important decisions can be translated into policy language. The article correctly identifies that expert judgment often lives outside manuals, logs, and prompts. That creates a governance gap in which the system can be compliant with instructions and still unsafe in context. Practitioners should treat tacit knowledge as part of the operating model, not as an informal exception path.

Signal collision is the right concept for describing autonomy boundaries. A well-designed agent should not ask for permission constantly, but it also should not force a binary autonomous action when two rules conflict. That tension is where identity governance, workflow design, and escalation logic meet. The named concept is useful because it explains why autonomy requires a stop condition rather than endless approval loops. Practitioners should map which decisions deserve escalation before deployment, not during incident review.

AI agent governance now overlaps with NHI and PAM controls. Once agents can act on infrastructure, access, or remediation tasks, the question becomes who or what is authorised to decide in the first place. That turns the agent into a governed non-human actor, even when no traditional credential abuse is involved. The relevant control challenge is not only access entitlement, but decision entitlement. Practitioners should align autonomous workflows with the same accountability discipline they already apply to privileged access.

Context-rich escalation is the difference between oversight and bottlenecking. The article is right to reject “Mother, may I” automation. Good escalation does not mean freezing the system until a human repeats the agent’s analysis. It means surfacing the colliding signals, the policy basis, and the operational risk in one reviewable package. Practitioners should optimise for decision quality at the handoff point, not for maximal deferral or maximal autonomy.

The next governance problem is not whether agents can act, but whether organisations can explain when they should stop. The more autonomous the workflow, the more important it becomes to define why the system halted and which human judgement filled the gap. That aligns with broader AI risk management thinking, including the NIST AI Risk Management Framework and agentic AI guidance. Practitioners should make explainable escalation a design requirement, not a documentation afterthought.

What this signals

Tacit knowledge becomes a control dependency as soon as agents touch privileged operations. Teams that treat autonomy as a pure automation problem will miss the real governance gap, which is deciding when a system must stop. The practical signal is whether your workflows can explain every handoff in language a reviewer can act on.

Signal collision is the operational pressure point for agentic governance. When an AI system faces mutually valid instructions, the programme needs a deliberate tie-breaking model rather than ad hoc intervention. That makes escalation design, exception logging, and decision ownership central to resilient AI operations.

The more your agents interact with identity and access processes, the more their behaviour resembles a governed non-human identity. That means the same discipline used for privileged access, approval trails, and auditability starts to apply to AI-driven actions as well. Practitioners should align these workflows with the NIST AI Risk Management Framework and agentic AI guidance such as the OWASP Agentic AI Top 10.

For practitioners

• Define escalation stop conditions Document the exact combinations of conflicting signals that require human review, especially where policy compliance and operational safety point in different directions.
• Capture decision context at handoff Preserve the policies, signals, and local exceptions that triggered escalation so reviewers can see why the agent stopped and what it could not resolve.
• Separate routine automation from exception routing Allow the agent to execute low-risk tasks directly, but route high-impact conflicts into a bounded review path with clear ownership and auditability.
• Treat tacit knowledge as a governed input Interview senior operators and convert repeated judgment patterns into explicit escalation criteria, then review those criteria after incidents or near misses.

Key takeaways

• Agentic systems fail when organisations assume every important decision can be reduced to explicit rules.
• The real governance issue is not constant permission seeking, but structured escalation when signals collide and context is missing.
• AI workflow design should treat human judgment as a control element, with traceable handoff logic and reviewable decision context.

Key terms

• Tacit Knowledge: Tacit knowledge is the practical judgment people use without being able to fully write it down. It comes from experience, context, and pattern recognition, and it often determines the right action when policy is incomplete or conflicting. In agentic systems, tacit knowledge becomes a governance input that must be surfaced through escalation.
• Signal Collision: Signal collision is the point at which an AI agent receives conflicting instructions or priorities that cannot be safely resolved by explicit logic alone. The term describes the moment a workflow needs human judgment, because the model has reached the limit of what policy, prompts, or rules can decide on their own.
• Decision Entitlement: Decision entitlement is the right to make or finalise a specific action in a workflow. In AI governance, it separates routine execution from high-impact approval and clarifies when an agent may act independently and when a human must decide. It is a useful control concept for privileged and agentic systems.
• Context-Rich Escalation: Context-rich escalation is a handoff to a human that includes the relevant signals, policies, and exceptions, not just a request for help. It reduces review burden, preserves auditability, and improves the quality of the final decision. The goal is a controlled transfer of judgment, not a generic pause in automation.

What's in the full article

Tonic's full article covers the conceptual detail this post intentionally leaves for the source:

• The article’s full explanation of tacit knowledge and the Polanyi Paradox in agentic systems.
• The collision model used to decide when an agent should stop and escalate.
• The Tier-0 patching scenario showing how rule-only automation can create business loss.
• The author’s framing of human judgment as a design element in autonomous workflows.

👉 Tonic's full article expands the collision model and the Tier-0 patching example in more detail.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance and agentic AI identity for practitioners building accountable workflows. It helps security and identity teams align privilege, escalation, and lifecycle controls across modern identity programmes.