Notifications
Clear all
Topic starter
21/08/2026 4:46 pm
TL;DR: Parseable JSON is not the same as a trusted machine contract, TruFoundry argues: structured outputs still require schema enforcement, business validation, and authorization before any downstream action. That boundary matters because tool calls, provider-specific schema limits, and retry loops can turn a formatting problem into an access-control problem.
NHIMG editorial — based on content published by TruFoundry: Structured Outputs: Why Production AI Needs Schemas, Not Just Prose
By the numbers:
- 80% of organisations report their AI agents have already performed actions beyond their intended scope, including accessing unauthorised systems, sharing sensitive data, and revealing access credentials.
- When AWS credentials are exposed publicly, attackers attempt access within an average of 17 minutes.
Questions worth separating out
Q: How should security teams govern structured outputs from AI agents?
A: They should treat structured outputs as an intermediate control, not a trust decision.
Q: Why do valid JSON responses still create security risk?
A: Because JSON only proves format, not correctness or permission.
Q: What do teams get wrong about tool calling in AI systems?
A: They often treat a typed tool call as if it were already approved.
Practitioner guidance
- Separate validation from authorization Require schema validation, business-rule validation, and explicit access decisions as distinct steps before any model output can trigger a tool call or external workflow.
- Shrink model-facing schemas Expose intent-level objects with small enums, required fields, and shallow nesting instead of passing full backend payloads to the model.
- Treat tool calls as privileged requests Apply least privilege, scoped approvals, and audit logging to every tool boundary, especially where AI agents act through MCP or other typed interfaces.
What's in the full article
TruFoundry's full article covers the operational detail this post intentionally leaves for the source:
- Provider-specific structured-output behavior across native JSON mode, schema-constrained paths, and translated tool calls
- Practical examples of Pydantic validation, Instructor retries, and machine-readable defect handling
- Implementation trade-offs for MCP Gateway output schemas and tool contracts in production workflows
- Guidance on when to use shadow validation to test schema compatibility before rollout
👉 Read TruFoundry's analysis of structured outputs, schemas and validation →
Structured outputs and tool calls: are your controls keeping up?
Explore further
21/08/2026 5:05 pm
Structured outputs are a governance problem, not just a parser problem. The article is right to separate JSON validity from business validity, because production risk starts when teams assume syntactic success equals safe execution. That assumption is especially dangerous in agentic AI and NHI contexts, where a machine-readable object can request privileged action. The correct governance posture is to treat structured output as one control layer inside a larger identity and policy boundary.
A question worth separating out:
Q: How do organisations reduce risk when models generate action objects?
A: Use small, purpose-built schemas, enforce deterministic validation, and keep retries bounded. That reduces the chance that a model invents excess fields or loops through repeated failures. For high-risk actions, require human or policy approval before the object can become a real-world side effect.
👉 Read our full editorial: Structured outputs need schemas, validation and authorization
