How do teams decide whether to optimise AI workloads or invest in greener infrastructure first?

Teams should start with the biggest avoidable waste. If models are overly large, retrained too often, or using inefficient workflows, optimise the workload first. If the environment is already efficient but powered by carbon intensive electricity, then renewable sourcing, better cooling, and more efficient hardware may deliver greater returns. Most programmes need both in sequence.

Why This Matters for Security Teams

Teams usually treat this as a sustainability-only decision, but the operational reality is broader: energy efficiency, model efficiency, and identity-driven control all affect cost, resilience, and risk. A workload that wastes compute because it is oversized or retrained too often creates avoidable emissions and unnecessary attack surface. A greener data centre does not fix an inefficient model. Current guidance suggests starting where the largest waste is concentrated, then layering infrastructure improvements where the environment still drives the footprint.

This matters because infrastructure and AI operations are now tightly coupled. If AI systems are making autonomous changes or consuming shared platform capacity, poor optimisation can amplify both energy use and governance gaps. NHIMG’s The 2026 Infrastructure Identity Survey found that 67% of organisations still rely heavily on static credentials despite the risks they pose to agentic AI deployments, which is a reminder that efficiency discussions cannot ignore control boundaries. Security teams that only measure carbon without measuring workload waste tend to miss the simplest savings first. In practice, many teams discover their biggest efficiency gains only after production cost overruns or model sprawl have already become visible.

How It Works in Practice

The decision usually starts with measurement. Teams should profile the workload to identify avoidable waste: oversized models, redundant fine-tuning, repeated inference on stale inputs, excessive logging, and poor scheduling. If the workload is inefficient, optimising it often yields immediate wins in both compute and emissions. That can mean choosing a smaller model, batching requests, caching results, reducing retraining frequency, or pruning unused features.

Once the workload is reasonably efficient, the next question is whether the remaining footprint is dominated by the environment. If so, greener infrastructure can have a larger marginal effect. That includes renewable electricity sourcing, higher-efficiency accelerators, better cooling, workload placement, and retiring underused hardware. The key is to avoid treating these as competing projects. They are sequential levers that interact.

For identity-aware AI operations, workload optimisation also reduces governance risk. A model or agent that is constantly redeployed, reconfigured, or routed through multiple services needs stronger workload identity controls, not just more electricity. The SPIFFE workload identity specification is relevant here because it anchors runtime trust to what the workload is, not where it happens to run. NHIMG’s Guide to SPIFFE and SPIRE is useful for teams deciding how to pair workload identity with least privilege in dynamic environments.

• Optimise first when the biggest waste is inside the model or pipeline itself.
• Invest in greener infrastructure first when workloads are already efficient but still energy intensive due to the facility or power mix.
• Measure both compute intensity and carbon intensity before setting priority.
• Reassess after each change, because workload tuning often shifts where the remaining emissions come from.

In practice, these controls tend to break down when AI is deployed across fragmented cloud, edge, and on-prem environments because no single team can see both workload waste and infrastructure emissions end to end.

Common Variations and Edge Cases

Tighter optimisation often increases engineering overhead, requiring organisations to balance carbon reduction against delivery speed and reliability. That tradeoff is especially visible in regulated environments, where model changes must be tested, approved, and documented before rollout. Current guidance suggests avoiding premature infrastructure upgrades if the application layer is still wasteful, because new hardware can simply accelerate inefficient usage.

There is also no universal standard for the exact order of operations. For some teams, carbon-aware scheduling or shifting to lower-carbon regions is the fastest win. For others, the model is the main problem and infrastructure changes would produce only marginal gains. A practical rule is to prioritise the control point that gives the biggest reduction per unit of effort, then move to the next layer.

NHIMG’s Ultimate Guide to NHIs — Standards is helpful when these decisions intersect with machine identity governance, while the broader Ultimate Guide to NHIs — What are Non-Human Identities page gives teams a common vocabulary for understanding which systems actually need strict identity and lifecycle control. For workload efficiency and carbon planning, the best practice is evolving rather than settled: teams should track both until the dominant source of waste is obvious, then switch emphasis accordingly.

Related resources from NHI Mgmt Group

• How do security teams decide whether to invest first in data discovery or data masking?
• How do organisations decide whether to invest first in an AI gateway or in per-application controls?
• How should teams decide whether to let AI generate remediation policies?
• How should security teams decide whether an AI agent gets human or non-human identity?