How should teams isolate sandboxed agent execution from grading logic?

Why sandboxing and grading must be split

The core design principle is separation of duties. The agent should only control a constrained task environment, while the grader should remain a separate, trusted component that sees only the artefacts it is meant to score. That reduces the chance that a prompt, tool call, or side effect can alter the criteria being applied.

A clean split also makes the trust boundary explicit. The sandbox can be treated as an execution zone for untrusted model behaviour, while the grader is part of the evaluation plane. When those roles blur, hidden tests, labels, scoring rules, or reference answers become part of the agent’s attack surface.

What the sandboxed runner must hide

The sandbox should expose only the minimum interface needed to complete the task, typically inputs, execution time, and a bounded output channel. It should not expose hidden tests, rubric internals, scoring code, or filesystem paths that reveal evaluation artefacts. If the agent can inspect those details, it can optimise for the grader rather than for the task.

This is also where output mediation matters. The safest pattern is to let the agent produce a result, then pass only that result, or a narrowly typed summary of it, to the grader. The grader should not accept direct control signals from the agent, and it should not run in the same process, notebook, or memory space as the model.

How to keep grading logic out of the agent's reach

The grading side should be treated as trusted infrastructure, not as another tool the model may discover. Keep prompts, policies, scoring thresholds, and reference data outside the sandbox, and avoid embedding them in environment variables or shared mounts that the agent can read back. For teams building agent evaluation pipelines, AI Agent Sandboxing and Code Execution Guide is a useful reference for isolating code execution and egress.

That boundary should extend to orchestration. If the agent can call back into grader logic, adjust its own prompt after seeing partial scores, or influence what gets logged and replayed, then the evaluation is no longer independent. The right model is one-way flow: sandboxed execution produces artefacts, and the grader consumes them without exposing its internals in return.

Risk and Threat Considerations

When grading and execution are coupled, the main risk is score manipulation through information leakage or prompt injection into the evaluation path. The agent may not need to break the sandbox if it can infer hidden tests, exploit shared state, or cause the grader to accept a crafted output format that bypasses intended checks.

Failure mechanism: A shared runtime, shared memory, or shared prompt stack lets the agent observe or influence labels, scoring rules, or test cases, which defeats independent evaluation and can produce inflated or meaningless scores.

Impact: Teams lose trust in benchmark results, reward the wrong behaviour, and may ship agents that optimise for the evaluator rather than the underlying task. In adversarial settings, that same flaw can leak hidden evaluation assets or enable test-set contamination.

Practitioner Guidance

What to verify: Confirm that the agent process cannot read grading inputs, hidden tests, or scorer code through mounted volumes, logs, exceptions, or debug endpoints. Also verify that the grader cannot consume raw prompts, mutable agent memory, or tool outputs that were not explicitly intended for scoring.

Implementation sequence: Start with process and filesystem isolation, then constrain the egress path to a single output channel, and finally run the grader in a separate trust zone with its own secrets and configuration. If you need richer evaluation metadata, add it deliberately and one field at a time rather than exposing the whole evaluation environment.

Practitioner takeaway: The evaluation boundary only works when the agent can affect results through outputs, not through visibility into the rules of scoring.

Related resources from NHI Mgmt Group

• Execution (MITRE ATT&CK Tactic)
• What breaks when an AI agent has sandboxed execution but still inherits host credentials?
• What breaks when agent harnesses mix control logic and execution in the same process?
• What is the difference between sandboxed and unsandboxed AI agent execution?