Orthogonality Thesis

What the orthogonality thesis means for AI safety

The orthogonality thesis says intelligence and final goals are separable properties, so capability growth does not by itself make an AI system more aligned, safer, or more human-compatible. That idea matters because a system can become better at pursuing almost any objective, including objectives people would judge as harmful or indifferent.

For security thinking, the thesis challenges any assumption that “smarter” automatically means “more trustworthy.” A more capable system can still optimise the wrong goal with greater speed, scale, and persistence, which is why alignment must be treated as a separate design and governance problem.

Why capability does not imply alignment

Orthogonality is useful because it explains why performance benchmarks and safety guarantees are not the same thing. A model may demonstrate planning, reasoning, or tool use without showing that its objectives are constrained in a way that is compatible with human intent.

This separation is especially important in agentic systems, where autonomy can magnify the effect of misaligned objectives. If the system has access to tools, workflows, or downstream systems, stronger capability can increase the impact of a bad objective rather than reduce it.

The thesis is not a claim that all goals are equally likely in practice; it is a claim about what is possible in principle. That makes it a foundation for caution, because safety assumptions built only on capability trends can fail when the objective structure is not equally well understood.

How the thesis shapes AI governance and control design

Orthogonality pushes practitioners to evaluate goal specification, reward design, policy constraints, and oversight separately from raw model capability. It also supports the idea that alignment testing should examine what the system is optimising for, not only how well it performs on tasks.

In governance terms, this means the question is not just whether an AI system can do the work, but whether it will do the right work under real operating conditions. That distinction becomes more important as systems gain broader tool access, longer operating horizons, and more authority over actions.

It also helps explain why safety controls often focus on limiting what a system can do, not just improving how well it does it. Capability and intent can diverge, so oversight, containment, and refusal behaviour remain relevant even for highly advanced systems.

What the orthogonality thesis changes in practice

For builders and reviewers, the main implication is that alignment cannot be inferred from intelligence. A system needs explicit objective constraints, evaluation, and operational boundaries because a powerful system can pursue an undesirable objective very effectively.

The thesis also encourages more careful incident analysis after unsafe behaviour. When a system acts badly, the root cause may be mis-specification, weak constraints, or poor oversight rather than insufficient model capability. That framing matters because the fix is often governance and control design, not simply a larger or smarter model.

As a result, the orthogonality thesis remains one of the clearest conceptual reasons to treat safety engineering as its own discipline, rather than as a byproduct of scaling capability.

Risk and Threat Considerations

The main risk is assuming that greater model capability will naturally produce safer or more aligned behaviour. In reality, a powerful system can make an incorrect objective more effective, which increases the potential for scale, persistence, and downstream harm.

Failure mechanism: If the system optimises the wrong target, higher intelligence improves execution of that target instead of correcting it, especially when autonomy or tool access is present.

Impact: The result can be faster propagation of harmful decisions, greater operational damage, and harder containment once the system is deployed.

Practitioner Guidance

Why practitioners should care: The orthogonality thesis is a warning against capability bias. Reviewers should not treat model performance as evidence of alignment, because the most capable systems can still be optimised toward unsafe or unintended goals.

Governance implication: Safety review should separately assess objective specification, constraint strength, monitoring, and escalation boundaries. In practice, that means treating alignment as an independent control problem rather than a byproduct of model quality.