Which accountability model works best when AI affects OT safety?

Why This Matters for Security Teams

When AI influences OT safety, the accountability model is not a paperwork detail. It determines who can approve actions, who can stop automation, and who carries responsibility when a process behaves unexpectedly. In OT environments, safety, availability, and physical impact are tightly coupled, so a weak ownership model can turn a software decision into a plant-level incident. NIST guidance on control ownership and auditability, including NIST SP 800-53 Rev 5 Security and Privacy Controls, is useful here because it reinforces that accountability must be assigned, not assumed.

The practical mistake is to treat the AI as the decision-maker because it is the fastest actor in the loop. That breaks the chain of responsibility. Human-led accountability means the organisation can prove who approved deployment, who reviewed model behaviour, who owns the exception process, and who can intervene when AI recommendations conflict with process safety rules. In OT, the answer must be operationally usable during an incident, not just defensible in a policy.

In practice, many security teams encounter accountability gaps only after a safety-relevant recommendation has already been executed, rather than through intentional governance design.

How It Works in Practice

The strongest model is a layered one: human ownership, machine-assisted execution, and pre-defined safety overrides. Procurement should define what the AI is allowed to influence, operations should define the control boundary, and safety or engineering leadership should retain final authority over actions that could affect equipment, people, or environmental conditions. That separation is consistent with the accountability emphasis in the NIST AI Risk Management Framework, which expects governed AI use rather than opaque delegation.

• Assign a named business owner for the AI use case, not just a technical owner.
• Define which outputs are advisory, which are executable, and which require approval.
• Keep immutable logs for prompts, recommendations, overrides, and escalations.
• Test fail-safe states so the system degrades safely if the model, sensor feed, or network path fails.
• Review access to the AI itself, including tools, connectors, and maintenance privileges.

For OT safety, that model must also account for how the AI reaches the environment. If a model can trigger a work order, alter a setpoint, or recommend a shutdown, then its authority should be constrained by policy, conditional access, and human confirmation. Where AI is connected to industrial telemetry, treat its inputs as safety-relevant data and validate them before use. MITRE’s ICS attack technique catalog is useful for thinking about how adversaries or faults can misuse these pathways.

This approach works best when engineering, operations, cybersecurity, and safety teams agree on the same decision thresholds. These controls tend to break down when legacy OT assets cannot support logging, approval gates, or rapid rollback because the AI ends up bridging systems that were never designed for accountable automation.

Common Variations and Edge Cases

Tighter accountability often increases operational overhead, requiring organisations to balance speed against safety assurance. That tradeoff matters most in high-availability plants, remote operations, and environments where AI is used to triage alerts before a human reviews them. Current guidance suggests that autonomy should increase only when evidence of control stability, testing, and rollback maturity is strong; there is no universal standard for this yet.

One common edge case is advisory AI that becomes functionally authoritative because staff trust it more than fragmented manual processes. Another is vendor-managed AI where procurement assumes the provider owns the risk, even though plant safety consequences remain with the operator. The governance model must make that distinction explicit in contracts, change control, and incident response plans. Where cyber-physical systems are involved, OT accountability should align with resilience expectations in CISA industrial control guidance and the resilience focus of NIS2-related guidance when applicable.

Another nuance is that some organisations use AI to recommend emergency responses, but final authority must remain with a trained human who understands process safety and can override automation when sensor quality, weather, maintenance state, or upstream process conditions make the recommendation unsafe. Best practice is evolving for agentic AI in OT, but the accountability principle is already clear: the AI may assist, yet the organisation must retain a human chain of command that can be audited after the fact.

Related resources from NHI Mgmt Group

• What is the difference between model safety and identity-aware access for AI agents?
• Should AI model safety scores be used as the main approval criterion for deployment?
• Who should own accountability for AI safety controls when models can call tools?
• How can organisations balance AI-driven testing with accountability and operational safety?