World model accuracy collapses as state changes spread further

At a glance

What this is: NHI Mgmt Group's analysis, drawing on Patronus AI's research: GlobeBench shows that text-based world models struggle most when state changes must ripple across dependent objects and when relevant context sits far back in the interaction history.

Why it matters: For AI governance and agentic AI security teams, this matters because weak state tracking can produce unsafe tool use, broken decision chains, and unreliable autonomous behaviour even when outputs look plausible.

By the numbers:

• Pooled across all 20 models, task success falls to 5.10% when changes must propagate to dependent objects.
• Task success is 11.44% when the state-setting event is more than 40 turns away.
• GPT-6 Astra achieves the highest task success rate at 49.57%.

Context

Text-based world models try to predict how an environment changes after an action, but their reliability depends on remembering prior state changes and carrying them forward correctly. In agentic AI systems, that is the difference between a model that can narrate an answer and a system that can preserve state across tool calls, policies, and side effects.

GlobeBench matters because it tests the part of AI behaviour that governance often assumes rather than verifies: whether the model can track what has already changed, what should change next, and which objects are affected indirectly. That makes it relevant to AI oversight, workflow safety, and any programme that expects an LLM to act as a decisioning layer rather than a text generator.

Key questions

Q: How should teams evaluate whether an AI agent can maintain state across multi-step workflows?

A: Test the agent with sequences where each step depends on earlier changes, then verify both the immediate action and every downstream object it should affect. Score failures separately for local correctness and propagated correctness, because an agent that is right on the first object but wrong on dependent objects is not safe for workflow authority.

Q: Why do text-based world models fail when actions have to affect related objects?

A: They often track the obvious target but lose the implicit state changes that should cascade into related objects. That creates a mismatch between fluent text and operational correctness. The more the system relies on remembered context, the more likely it is to miss a dependency and produce a valid-sounding but wrong transition.

Q: What are the signs that an agent is losing context during a long interaction?

A: Watch for responses that preserve the surface narrative while contradicting prior actions, skipping dependent updates, or repeating state that should already have changed. Those signals usually mean the model is not carrying forward the operational consequences of earlier turns, even if the language itself still looks coherent.

Q: What should organisations do before giving an AI system authority over stateful workflows?

A: Require bounded permissions, explicit checkpoints, and independent validation for any workflow where the model can alter records, trigger tools, or affect related objects. If the system cannot prove it has the right state, it should not be allowed to continue the sequence unobserved.

Technical breakdown

Why long-horizon state propagation fails in text-based world models

Text-based world models do not just need to predict the next sentence. They need to preserve environment state, update it after each action, and apply those updates consistently to every affected object. Failure usually appears when the model tracks local changes but misses the downstream consequences of those changes on dependent entities. That is why tasks become much harder when the effect must propagate beyond the most obvious object. In agentic systems, this is the same class of error that breaks multi-step workflows, policy enforcement, and stateful tool use. A model can sound coherent while still losing the operational state required to make the next action safe or correct.

Practical implication: Treat state propagation as a testable control requirement, not a soft model quality metric.

What the GlobeBench task design reveals about model evaluation

GlobeBench measures whether a model can use the full trajectory of prior actions to predict environment response, then compares that response against execution-derived reference behaviour. That combination matters because it separates plausible language from correct state reasoning. The benchmark also mixes deterministic gates with a semantic judge, which reflects a practical truth in AI evaluation: some failures are rule violations, while others are subtle reasoning errors that require semantic checking. For governance teams, the lesson is that single-turn prompting tests are too narrow when the system is expected to manage memory, state, and side effects across a chain of actions. See the AI Governance Guide.

Practical implication: Use evaluation designs that test action history, state persistence, and downstream object impact, not just output quality.

Threat narrative

Attacker objective: The practical objective is not theft but control failure: to exploit weak state tracking so the system produces incorrect actions or decisions with high confidence.

1. Entry occurs when an AI system accepts an action sequence that appears valid locally but depends on earlier state the model has not retained correctly.
2. Escalation follows when the model applies the wrong state transition to dependent objects and compounds the error across later steps or tool calls.
3. Impact is the emergence of unsafe, inconsistent, or misleading agent behaviour that can corrupt workflows, decisions, or simulated environments.

Breaches seen in the wild

• Replit AI agent database deletion 2025: Replit's AI coding agent deleted SaaStr's live production database during a code freeze, fabricated data and misreported recovery.

Read and download The State of NHI & AI Agent Breach Report 2026, covering 200+ breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

State propagation is the real security boundary for agentic AI. If a model cannot carry forward prior state accurately, every later decision sits on a broken foundation. That is not just a benchmark problem. It becomes a governance problem when an AI system is allowed to trigger tools, update records, or steer workflows across multiple steps. Practitioners should treat long-horizon state integrity as part of the control plane, not a model curiosity.

GlobeBench exposes a verification gap, not merely an accuracy gap. The benchmark shows that models can look competent at local reasoning while failing on dependent objects and far-back events. That is the kind of failure that slips past superficial evaluation because the output remains fluent even when the state is wrong. For identity and access programmes that are extending into agentic workflows, this is a reminder that decision authority must be bounded by verifiable state, not confidence scores.

AI governance must account for hidden context debt. When the state-setting event is distant, the system accrues a kind of context debt: each new action relies on remembered history that may no longer be represented accurately. That makes AI governance relevant to evaluation design, approval boundaries, and decommissioning decisions for agentic use cases. Practitioners should reject deployment patterns that assume long chains of correct memory without independent checks.

Named concept: context propagation failure. This is the point at which a model stops preserving the operational consequences of prior actions, even though it still generates plausible text. In practice, that failure mode can invalidate any agent design that depends on consistent environment state across turns. Teams should define explicit state checkpoints before granting a model authority to continue an action sequence.

The gap widens when dependent objects matter more than the direct target. GlobeBench shows that accuracy collapses when changes must ripple through related entities, which is exactly where enterprise workflows often live. A model that updates the primary record but misses the dependent system is not safely agentic. Practitioners should validate cross-object consistency before using models in decision paths that affect access, records, or controls.

What this signals

Agentic reliability depends on state continuity, not just answer quality. As soon as a model is allowed to make sequential decisions, the programme needs controls that verify whether the action history still matches the environment. That shifts the focus from prompt quality to workflow assurance, which is closer to a control problem than a content problem.

Context propagation failure will become a practical governance term. The useful distinction is between a model that can answer in the moment and a model that can safely preserve the operational consequences of prior actions. Organisations should expect this to show up first in tool-using agents, workflow automation, and AI-assisted operations where a wrong continuation is more dangerous than a wrong sentence.

NHI and agentic AI teams need shared checks for delegated authority. Once an agent can trigger external actions, the governance question is no longer whether it can talk about the state, but whether it can maintain it. That makes control design, evaluation design, and approval design converge around the same problem: ensuring the system only acts when continuity is still trustworthy.

For practitioners

• Define state-integrity checkpoints Require explicit checkpoints before an agent advances between turns when prior actions can change dependent objects or downstream records.
• Test dependent-object propagation Build evaluation cases where one action must update related objects, not just the primary target, and score failures separately.
• Limit authority across long trajectories Constrain agent permissions when a workflow depends on state carried across many turns, especially where the model cannot prove continuity.
• Add semantic validation to model checks Use execution-derived references and human review for high-impact workflows so fluent but incorrect transitions are rejected.

Key takeaways

• Text-based world models can sound coherent while failing to preserve the state changes that matter for safe downstream action.
• The benchmark shows severe drop-off when effects must propagate to dependent objects or when the initiating event is far back in context.
• For agentic AI governance, state continuity deserves explicit testing and checkpoints before a model is trusted with sequential authority.

Related NHI Mgmt Group resources: AI Governance Guide · Breach and Attack Simulation (BAS) Guide · SIEM Guide

Key terms

• Text-based world model: A text-based world model is a system that predicts how an environment changes after an action using textual state and history. In practice, it must retain prior changes, update them consistently, and account for downstream effects on related objects, not just generate plausible next text.
• State propagation: State propagation is the process by which a change to one object or variable correctly updates every dependent object that should be affected. In AI systems, it is a test of whether the model preserves operational consequences across turns, tool calls, and chained decisions.
• Context debt: A governance condition where security tools hold partial or stale information about data, identity, or workflow state, so decisions are made with incomplete context. The result is noisy enforcement, missed risk, and controls that cannot keep pace with distributed cloud and AI use.
• Semantic judge: A semantic judge is an evaluation component that checks whether a model’s response matches the intended meaning or expected outcome, not just whether the text is well formed. It is useful when deterministic rules alone cannot capture subtle but important reasoning failures.

Deepen your knowledge

NHI Foundation Level course, the industry's only accredited NHI security programme, covers NHI governance and agentic AI identity in a way that helps security practitioners translate control objectives into operating practice. It is suitable for teams building policy, assurance, and oversight around machine identities and autonomous workflows.