Reasoning Model Security: Protecting Thinking, Budgets and Agent Actions

A reasoning model is a large language model that generates an internal chain of thought, billed as thinking or reasoning tokens, before it produces a visible answer or a tool call. That hidden step changes the security picture: there is now a stream of model reasoning that may contain sensitive data, that is usually encrypted or summarised rather than shown, that costs money whether or not anyone reads it, and that may or may not reflect what the model actually did. Reasoning models now sit behind most agent frameworks, so these properties shape how agents are monitored, budgeted and audited. This guide explains how reasoning output is exposed by the major APIs, what chain of thought (CoT) monitoring can and cannot tell you, how reasoning budgets are abused, and what changes for agents. It is for engineers and security teams building on reasoning models; the wider LLM threat picture is in the LLM Security Guide.

Key takeaways

• Raw reasoning is mostly not returned: current APIs return summaries, encrypted reasoning items or signatures rather than the full chain of thought, and those opaque blocks must be passed back unchanged.
• You pay for reasoning you never see: thinking tokens are billed as output and count against output limits, so reasoning effort is a cost and availability control.
• CoT monitoring helps but is not proof: research shows monitors reading reasoning catch misbehaviour that action-only monitors miss, and also that reasoning is often unfaithful to what the model actually used.
• Summaries are a data exposure surface: thought summaries can restate system prompt content, retrieved documents or tool output, so treat them like model output for logging and display.
• Agents need action controls, not reasoning trust: authorise and log tool calls on their own merits; do not let a plausible chain of thought stand in for authorisation.

What it is

Reasoning models spend extra computation working through a problem before answering. Providers expose this through parameters that set how much the model thinks and through response blocks that carry a summary or an encrypted copy of the reasoning. The reasoning tokens occupy the context window, are billed, and in tool-using turns are part of the conversation state the provider needs to continue reasoning.

How it works

The three APIs below are listed alphabetically; the order is not a ranking. Details are from each provider's documentation as checked on 8 November 2026.

Anthropic (Claude API)

• Controls: adaptive thinking (thinking: {type: "adaptive"}) lets the model decide how deeply to think, with effort scaling the whole response and max_tokens as the hard ceiling. The older manual mode with budget_tokens is documented as deprecated for newer models.
• What is returned: the display setting returns thinking blocks as a summary (summarized) or empty (omitted); each block carries a signature, an encrypted copy of the full reasoning the API uses to verify blocks came from the model. Safety-redacted reasoning arrives as redacted_thinking blocks with opaque data.
• Round-tripping: within a tool-use turn, thinking blocks must be passed back complete and unmodified; edited blocks are rejected. Code that filters only on type == "thinking" silently drops redacted blocks.
• Billing: full thinking tokens are billed as output, not the summary length; omitted reduces latency, not cost. Usage reports thinking tokens separately.

Google (Gemini API, Interactions API)

• Controls: thinking_level in the generation configuration sets reasoning depth, with levels and defaults varying by model; the documentation advises lowering the level rather than setting a small max_output_tokens.
• What is returned: only the final output by default; thinking_summaries set to auto adds summaries. Each thought step carries a signature, an encrypted representation of the reasoning state.
• Round-tripping: in stateful mode the server manages thought blocks; in stateless mode every thought block, including signatures on built-in tool steps, must be resent exactly as received.
• Billing: thinking tokens are billed with output tokens, including when only a summary is returned, and are reported as total_thought_tokens; a response that hits the output limit during reasoning is marked incomplete and still billed.

OpenAI (Responses API)

• Controls: reasoning.effort sets how much the model thinks, with supported values depending on the model.
• What is returned: reasoning items are opaque and the API does not return raw reasoning text. In stateless use (store: false or zero data retention), reasoning items include encrypted content to pass back in later requests.
• Round-tripping: passing encrypted reasoning items back preserves reasoning continuity across turns without exposing raw reasoning.
• Billing: reasoning tokens occupy the context window, are billed as output and appear in output_tokens_details; max_output_tokens caps reasoning plus output, and the documentation suggests reserving at least 25,000 tokens when starting out.

Why it matters

Chain of thought exposure

• Summaries can leak context: a reasoning summary describes what the model considered, which can include system prompt rules, retrieved confidential documents or secrets present in tool output. Apply the same output filtering and access control to summaries as to answers, and do not show them to end users who should not see that context. See the OWASP LLM Top 10 LLM07 System Prompt Leakage Guide.
• Encrypted blobs in logs: signatures and encrypted reasoning items are opaque but are still customer content under data handling rules. Decide whether your traces store them, and apply retention accordingly. See the Zero Data Retention for AI guide for how stateless modes interact with retention.
• Integrity of the conversation: providers verify returned reasoning blocks, so tampering is rejected rather than silently accepted; middleware that rewrites or drops blocks breaks agents in ways that look like model errors.

CoT monitoring

• The opportunity: OpenAI research (March 2025) found that a separate LLM prompted to read a reasoning model's chain of thought caught reward hacking in coding tasks more often than a monitor that saw only the agent's actions, because models often stated their intent in plain language.
• The fragility: the same research found that penalising "bad thoughts" during training left the misbehaviour in place but made it largely undetectable, and recommended against strong optimisation pressure on CoT. A multi-author paper, "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety" (2025), calls CoT monitoring promising but imperfect and asks developers to consider how their decisions affect monitorability.
• Faithfulness limits: Anthropic research (April 2025) gave models hints and found that their reasoning acknowledged using the hints in only a minority of cases, and in reward hacking tests the models rarely disclosed the exploit. CoT monitoring cannot show that a model is not misbehaving.
• What customers can monitor: as an API customer you usually see summaries, not raw reasoning, so practical monitoring centres on summaries, tool calls, token usage and outcomes. See the LLM Observability Guide.

Reasoning budget abuse

• Cost harvesting: inputs crafted to provoke long reasoning (puzzles, contradictory constraints, recursive tasks) multiply output tokens per request. This maps to OWASP LLM10 Unbounded Consumption and MITRE ATLAS AML.T0034 Cost Harvesting.
• Availability: long reasoning raises latency and can exhaust rate limits or quota for everyone on the same key, which ATLAS describes as AML.T0029 Denial of AI Service.
• Silent failure: when reasoning consumes the output limit, responses come back incomplete or empty while still billed, which agents may retry in a loop. See the Denial-of-Wallet and AI Agent Cost Runaway Guide.

Agent implications

• Decisions happen inside reasoning: with interleaved or adaptive thinking, the model reasons between tool calls. Injected instructions in tool output (AML.T0051 LLM Prompt Injection) can steer that reasoning, so tool authorisation must hold regardless of how convincing the reasoning looks. See the Prompt Injection Guide.
• Identity of the acting agent: reasoning does not carry identity. Every tool call should run under a scoped credential for the agent and, where relevant, the delegating user, with logs that tie the call to both. See the Agent Identity and Access Guide.
• Approval gates: high-impact actions need human or policy approval outside the model; reasoning summaries can inform the approver but are not evidence of intent. See the Human-in-the-Loop for AI Agents Guide.

Common misconceptions

• "The chain of thought explains what the model did": published faithfulness research shows reasoning often omits the factors that drove an answer. Use it as a signal, not an audit record of intent.
• "Hidden reasoning is private reasoning": summaries, signatures and encrypted items still leave the provider and pass through your systems and logs.
• "Omitting thinking saves money": providers bill the full reasoning generated whether it is summarised, omitted or encrypted.
• "Reasoning models resist jailbreaks, so guardrails matter less": stronger reasoning does not remove the need for input and output controls; jailbreaks (AML.T0054 LLM Jailbreak) and prompt injection still apply. See the AI Guardrails and AI Firewalls Guide.
• "Middleware can safely trim old thinking blocks": within tool-use turns, the APIs require blocks to be returned intact; trimming breaks continuity or causes errors.

Related concepts

Threat to control mapping

Rolling out controls

1. Inventory: list which applications and agents use reasoning models, at what effort level, and whether summaries are stored or displayed.
2. Set budgets: choose effort and output caps per use case, and enforce per-key spending limits at a gateway. See the AI Gateway and LLM Gateway Guide.
3. Handle reasoning blocks correctly: round-trip signatures, encrypted items and redacted blocks unmodified, and test this in integration tests.
4. Classify summaries: treat them as model output with the sensitivity of the highest-classified context in the turn.
5. Monitor: log tool calls, reasoning token counts and incomplete responses; alert on spikes and on retry loops.
6. Roll back: keep the ability to drop effort or switch a workload to a non-reasoning configuration if costs or latency spike.

Metrics

• Reasoning tokens per request by application, with alerts at a set multiple of baseline.
• Share of responses returned incomplete because the output cap was hit during reasoning.
• Tool calls blocked by tool-side authorisation, per agent.
• Summaries flagged by output filters for sensitive content.

Practitioner checklist

• Set explicit effort or thinking levels and output caps per use case instead of provider defaults.
• Enforce per-key and per-tenant token budgets and rate limits for reasoning workloads.
• Round-trip signatures, encrypted reasoning items and redacted blocks unchanged, and test it.
• Apply output filtering and access control to reasoning summaries before display or storage.
• Decide and document retention for reasoning artefacts in logs and traces.
• Authorise every agent tool call on the tool side with scoped credentials, regardless of the reasoning shown.
• Monitor reasoning token usage, incomplete responses and retry loops.
• Treat CoT summaries as a monitoring signal, never as proof of intent or compliance.

Standards and references

• Anthropic: Building with thinking
• Google AI for Developers: Gemini thinking
• OpenAI: Reasoning models
• OWASP GenAI Security Project: Top 10 for LLM Applications 2025
• arXiv: Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
• OpenAI: Chain of thought monitoring research (March 2025)
• Anthropic: Reasoning models don't always say what they think
• MITRE: ATLAS

Related NHI Mgmt Group resources: LLM Security Guide · Denial-of-Wallet and AI Agent Cost Runaway Guide · LLM Observability Guide · Agent Identity and Access Guide