Model optimization

Model optimization is the process of changing an AI model so it runs more efficiently without an unacceptable loss of quality. In practice, this usually means reducing size, latency, or memory use while preserving the level of accuracy required for production decisions and controls.

Expanded Definition

Model optimization refers to changing a model’s structure, parameters, or execution path so it delivers acceptable outputs with less compute, memory, or latency. In NHI and agentic AI environments, this matters because a model that is smaller or faster can be deployed more widely, but it may also reduce interpretability, degrade edge-case performance, or alter control behavior.

Definitions vary across vendors on whether optimisation includes only inference-time methods such as quantisation and pruning, or also training-time changes such as distillation and architecture search. NHI Management Group treats model optimization as a governance issue as well as an engineering choice, because the optimized model still performs actions, consumes secrets, and may influence control decisions inside systems governed by the NIST Cybersecurity Framework 2.0. The key question is not just whether the model runs faster, but whether it remains trustworthy for the production task after the change.

The most common misapplication is assuming a smaller model is automatically safer or equivalent, which occurs when teams optimise for cost or latency without revalidating decision quality and control coverage.

Examples and Use Cases

Implementing model optimization rigorously often introduces a verification burden, requiring organisations to weigh lower inference cost against the risk of altered outputs, degraded recall, or reduced auditability.

• Quantising a model so an AI agent can run on a smaller instance while preserving enough accuracy for ticket triage or policy lookup.
• Pruning redundant parameters to reduce latency in a tool-using assistant that must respond before a session token expires.
• Distilling a larger model into a compact model for repeated NHI-related classification tasks, such as identifying risky secret references in logs.
• Recompiling or optimising an embedded model used in an access workflow, then validating that its output remains stable under production traffic.
• Comparing baseline and optimised models against the same evaluation set before rollout, consistent with guidance in the Ultimate Guide to NHIs and deployment controls in the NIST Cybersecurity Framework 2.0.

These use cases are most relevant when the model is part of an operational control plane rather than a standalone analytics experiment.

Why It Matters in NHI Security

Model optimization becomes security-relevant when a model is embedded in workflows that touch service accounts, API keys, or autonomous actions. If the optimisation step changes output behavior without a corresponding control review, an AI agent may approve the wrong action, miss a risky secret reference, or follow a weakened policy path. That is why model optimization should be treated as part of change management, not just MLOps tuning.

This concern is amplified in NHI environments because weak governance around identities and secrets already creates significant exposure. NHI Management Group reports that 96% of organisations store secrets outside of secrets managers in vulnerable locations including code, config files, and CI/CD tools, making any model that interacts with those systems especially sensitive. Model changes that affect detection, classification, or orchestration must therefore be tested against the same operational risks described in the Ultimate Guide to NHIs.

Organisations typically encounter the impact only after a production incident, at which point model optimization becomes operationally unavoidable to address.

Related resources from NHI Mgmt Group

• When does model optimization create more risk than it reduces?
• What is the Model Context Protocol (MCP) and why does it matter for security?
• What does AI model abuse reveal about the current NHI threat surface?
• Why do attackers often check model availability before trying to generate content?