How do organisations decide whether to deploy a new model like Gemini 3 or keep their current agent model?

Why This Matters for Security Teams

The choice between a newer model and the current agent model is not just a product decision. It affects security posture, operational reliability, and how much trust the organisation can place in autonomous actions. A model upgrade can improve reasoning, tool use, or multilingual coverage, but it can also change failure modes, expand prompt injection exposure, or alter how safeguards behave under load. That is why the decision should be governed like any other material AI change, with risk, testing, and rollback considered together.

Security teams often underestimate how quickly a “better” model can introduce new attack surface. Agentic systems are especially sensitive because model output is not just content, it can drive actions, calls, and downstream decisions. Guidance from the NIST AI Risk Management Framework is useful here because it frames deployment as a managed risk decision, not a benchmark contest. The practical question is whether the new model improves the scenarios that matter most while preserving control over access, output quality, and escalation paths. In practice, many security teams encounter model risk only after a production workflow has already been disrupted by a silent regression.

How It Works in Practice

Organisations usually decide by running the new model through a controlled evaluation pipeline that reflects their own tasks, data, and guardrails. That means comparing the candidate model and the current agent model on the same prompts, the same tools, and the same success criteria. A generic leaderboard score is rarely enough. The evaluation should include task accuracy, refusal quality, hallucination rate, tool-call correctness, latency, cost, and safety behaviour under adversarial prompts.

For agentic systems, the test plan should also check whether the model becomes easier to manipulate through prompt injection, indirect prompt injection, or tool abuse. The OWASP Top 10 for Agentic Applications 2026 and the MITRE ATLAS adversarial AI threat matrix both help teams map realistic failure and attack patterns into test cases. In practice, a useful rollout decision often comes from three questions:

• Does the new model improve the highest-value workflows, not just the easy demos?
• Does it preserve the controls around data access, tool use, and human approval?
• Does it fail safely when prompts are malformed, hostile, or ambiguous?

Teams should also compare incident handling. If the new model increases false escalations or produces harder-to-audit actions, that operational overhead can outweigh its gains. For higher-risk deployments, current guidance suggests pairing model evaluation with threat modeling, especially where the agent can send emails, modify records, trigger workflows, or interact with external systems. The CSA MAESTRO agentic AI threat modeling framework is useful for structuring those reviews. These controls tend to break down when the model is swapped into a live agent stack without re-testing tool permissions, memory, and downstream automation logic.

Common Variations and Edge Cases

Tighter model governance often increases evaluation time and operational overhead, requiring organisations to balance performance gains against deployment friction. That tradeoff becomes sharper when the new model is clearly better in one narrow domain but weaker in routine production flows. Best practice is evolving here, and there is no universal standard for declaring one model “better” across all agent workloads.

Some organisations should keep the current model even if the new one looks stronger on paper. That usually happens when the system supports regulated decisions, critical customer interactions, or high-volume automations where stability matters more than incremental quality. Others may adopt the new model only for a bounded use case, such as drafting, summarisation, or internal research, while leaving action-capable agents on the existing version until the evaluation suite matures. The decision should also account for governance maturity: if monitoring, approval gates, or audit logging are weak, a model upgrade can magnify existing control gaps rather than solve them.

This is where the distinction between capability and operational fit matters. A model with better reasoning may still be the wrong choice if it changes refusal style, expands context sensitivity, or makes tool calls less predictable. Where a deployment touches security-sensitive workflows, alignment with NIST AI Risk Management Framework and the OWASP Agentic AI Top 10 helps teams keep the question grounded in risk, not novelty.

Related resources from NHI Mgmt Group

• How should organisations decide whether to keep a central vault or move to a new model?
• How do organisations decide whether a model is safe enough to deploy?
• How should organisations decide whether to keep using traditional MFA?
• How do security teams decide whether an AI agent should keep access to regulated data?