How do evaluation environments need to control internet access for agents?

Why evaluation sandboxes should default-deny outbound traffic

Agent evaluations are only meaningful if the environment behaves like a controlled testbed, not a general-purpose workstation. Default-deny egress keeps the agent inside that boundary by blocking outbound traffic first and then allowing only the destinations the test genuinely needs, such as a specific API, package source, or verification domain.

That matters because many agent failures are environmental, not model-only, a network path that is open by default can turn a contained evaluation into real-world access, unexpected tool use, or external data transfer. If the test does not require internet reachability, the safest design is no internet at all.

How allowlisting should be scoped and reviewed

Allowlisting works best when it is narrow, explicit, and tied to a named test purpose. The exception should identify the exact host or service, the protocol or port if relevant, and the duration or condition under which it is valid, so reviewers can tell whether the access is part of the evaluation or a convenience added later.

Logging is part of the control, not an optional add-on. Every permitted destination should be observable, and exceptions should be reviewed against the test plan so teams can spot unnecessary reachability, hidden dependencies, or agents that are silently expanding their scope during execution.

AI Lab and Agent Evaluation Incidents shows why evaluation labs need strict egress control, because real incidents have emerged from sandbox environments that were too permissive.

What to do when an agent needs the internet during a test

If internet access is genuinely required, treat it as a controlled exception rather than a standing permission. The practical question is whether the agent needs broad web access or only one narrow dependency, because those two cases call for different controls and very different review standards.

Prefer a staged approach: start with no outbound access, add only the smallest viable destination set, and remove the exception as soon as the test case is complete. Where the agent must reach external services, route that access through monitored infrastructure so you can trace what was contacted and why.

AI Security Evaluations Guide is useful here because evaluation design should include both the test harness and the control plane that keeps the harness trustworthy.

Risk and Threat Considerations

Open internet access can turn an evaluation environment into an unintended bridge to live systems, external APIs, or attacker-controlled infrastructure. That creates exposure if the agent is prompted to fetch data, follow links, exfiltrate content, or interact with services that were never meant to be part of the test.

Failure mechanism: The environment allows outbound connections by default, so the agent can reach destinations outside the intended test boundary and use them as a data source, command path, or exfiltration channel.

Impact: Results become untrustworthy, sensitive data may leave the lab, and a compromised or misbehaving agent can interact with real external services instead of a sandboxed substitute.

Practitioner Guidance

What to verify: Before trusting an evaluation result, confirm that outbound traffic was blocked by default and that every permitted destination was explicitly approved for that test case. If the agent can reach a general browser, package mirror, or webhook endpoint without a written reason, the sandbox is too open.

Decision rule: If the test can be run without internet, keep it fully disconnected. If it cannot, scope the exception to the smallest possible destination set and require review of any new domain, API, or package source before the run is treated as valid.

Practitioner takeaway: The goal is not to simulate the whole internet, it is to preserve a clean boundary so the evaluation measures the agent, not the side effects of uncontrolled network reach.

Related resources from NHI Mgmt Group

• How should security teams control AI evaluation environments so autonomous agents cannot escape into production systems?
• Why do AI gateways and connected agents complicate access control in cloud environments?
• Why do AI agents and partner applications complicate access control in application environments?
• Why do AI agents and service accounts create new access-control risks in API-first environments?