AWS model fine-tuning permissions are becoming a privilege risk

At a glance

What this is: This analysis tracks newly surfaced AWS privileged permissions and argues that model fine-tuning now expands cloud identity risk from data access into model integrity.

Why it matters: IAM, PAM, and NHI teams need to treat model-shaping permissions as privileged access because they can change system behaviour, not just expose resources.

By the numbers:

• 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job.
• Only 44% of organisations have implemented any policies to manage their AI agents, despite 92% agreeing that governing AI agents is critical to enterprise security.

👉 Read Sonrai Security's analysis of new AWS privileged permissions and model risk

Context

AWS permission growth is no longer limited to classic infrastructure operations. As cloud platforms extend into model training and AI workflow control, identity teams have to treat certain permissions as governance over behaviour, not just access to resources.

That shift matters for NHI, IAM, and PAM programmes because fine-tuning and telemetry permissions can influence how systems act after authentication succeeds. In practice, the question moves from who can reach a service to who can alter what that service will do next.

Key questions

Q: How should security teams govern permissions that can change AI model behaviour?

A: Treat those permissions as privileged access, not ordinary application functions. Put approval, logging, lineage, and rollback controls around any action that can retrain, fine-tune, or alter a model’s safety behaviour. If a permission can change what the system does later, it belongs in the privileged estate and should be reviewed accordingly.

Q: Why do model fine-tuning permissions create a bigger risk than ordinary cloud permissions?

A: Because the impact can persist after the session ends. A normal permission grants access to a resource, but a fine-tuning permission can alter future behaviour, safety filters, or output quality. That makes the control problem one of behaviour governance, not just resource protection or least privilege.

Q: What breaks when AI permissions are reviewed like standard DevOps access?

A: Review cycles miss the fact that some AI permissions are not transient. If a privilege can rewrite model logic, then the risk survives the request itself and may show up much later in a different workflow. Standard access reviews are too slow and too narrow for that kind of effect.

Q: Which controls should organisations use for AI training and telemetry access?

A: Use separate approval paths, tight role scoping, provenance checks, and explicit rollback rules. Training rights should not travel with broad telemetry access, because both can be abused in different ways. The goal is to prevent model manipulation from hiding inside routine operational permissions.

Technical breakdown

Model integrity permissions in cloud IAM

Cloud IAM traditionally controls whether an identity can read, write, launch, or administer a resource. Model-integrity permissions are different because they govern actions that can shape a model’s outputs, internal behaviour, or safety posture. In AI-enabled cloud services, that means a privileged action can function like a configuration change, a training event, and a persistence mechanism at once. This is why security teams need to classify some AI permissions as privileged even when they do not look destructive in the usual infrastructure sense. The access path is legitimate, but the downstream effect can be operationally toxic.

Practical implication: classify model-training and model-customisation permissions as privileged access and subject them to PAM-style controls.

Why fine-tuning creates a new persistence path

Fine-tuning can become a persistence mechanism because it changes the system’s behaviour without needing repeated attacker interaction. If a malicious or unauthorised dataset is used, the resulting model may preserve harmful patterns, ignore guardrails, or leak sensitive content in later sessions. That is materially different from one-time abuse of a token or API key. The identity layer authorises the training action, but the security consequence survives after the session ends. For NHI governance, that is a lifecycle problem as much as an access problem because the privilege outlives the action that used it.

Practical implication: require approval, lineage tracking, and rollback controls for any permission that can modify model behaviour.

Deep-tier telemetry and the limits of visibility

Telemetry permissions matter because they can expose detailed operational signals that help attackers test, tune, or hide model manipulation. In cloud environments, deep visibility is often assumed to improve defence, but privileged observability can also become a source of intelligence for abuse. That creates an uncomfortable asymmetry: the same permissions that support detection can also support iterative evasion when granted too broadly. This is especially relevant where AI agents or automated workflows consume those signals quickly and repeatedly. Visibility without entitlement boundaries becomes another attack surface.

Practical implication: separate read-only observability from sensitive training and control-plane privileges, then review both under least privilege.

Threat narrative

Attacker objective: The attacker’s objective is to persist control inside the model itself so that harmful behaviour continues after the original access event ends.

1. Entry occurs through legitimate cloud permissions that allow an identity to create or influence a fine-tuning workflow.
2. Escalation happens when that authorised training path is used to introduce malicious or unsafe model behaviour that survives beyond the initial request.
3. Impact follows when the altered model leaks data, ignores safety filters, or returns intentionally harmful responses in later sessions.

Breaches seen in the wild

• Codefinger AWS S3 ransomware attack — Codefinger used compromised AWS credentials to encrypt S3 buckets via SSE-C.
• Cisco DevHub NHI breach — IntelBroker exploited exposed Cisco credentials, API tokens and keys in DevHub.

Read our 52 NHI Breaches Analysis report for a comprehensive view of breaches impacting Non-Human Identities including AI Agents.

NHI Mgmt Group analysis

Model-integrity permissions are privileged access, even when they do not look like classic admin rights. The cloud security habit of separating “dangerous” from “routine” permissions breaks down once a permission can alter model behaviour, training data, or safety filters. That means identity governance has to classify AI training actions as a privileged control surface, not a feature toggle. Practitioners should treat model-shaping rights as part of the privileged access estate.

Fine-tuning permissions create a persistence channel, not just a workflow step. The usual IAM assumption is that access is exercised in discrete sessions and the security impact is bounded to the request. When a permission can rewrite model behaviour, the resulting effect survives the session and can influence future interactions, which is a different governance problem entirely. The implication is that review cycles built for transient access are too slow for behaviour-changing privileges.

Cloud model governance now sits at the intersection of NHI, PAM, and AI risk management. OWASP-NHI and ZT-NIST-207 remain relevant because the identity granting the training action is still a non-human actor consuming and changing resources. NIST-AIRMF becomes relevant when the question moves from access control to how the organisation governs model behaviour outcomes. Practitioners need a shared control language for identities that can change what the system does, not only what it can reach.

Model integrity is the right named concept for this category of risk. It captures the fact that a privilege can compromise the trustworthiness of a model without ever breaking authentication or exfiltrating a secret. That is why this class of permission should be evaluated on behavioural impact, not only on technical privilege level. Security teams should build governance around what the permission can rewrite, not just what it can access.

From our research:

• 70% of organisations grant AI systems more access than they would give a human employee performing the exact same job, according to The 2026 Infrastructure Identity Survey.
• Only 13% of organisations feel extremely prepared for the reality of agentic AI, even as 53% expect AI to run major portions of infrastructure autonomously within three years.
• For a broader identity-governance lens on this shift, see Ultimate Guide to NHIs, which maps privilege, lifecycle, and control boundaries across machine identities and AI agents.

What this signals

Model-integrity permissions are becoming a governance boundary for cloud programmes. Once a permission can reshape how an AI system behaves, the control question is no longer just whether the identity is authorised, but whether the organisation can explain and reverse the effect. Teams that still fold these permissions into ordinary DevOps roles will struggle to distinguish safe automation from hidden behaviour change.

The operational signal is that identity programmes need a second inventory layer for AI-specific privileges. With 67% of organisations still relying heavily on static credentials despite the risks they pose to agentic AI deployments, the programme gap is no longer theoretical. Security teams should map where training, telemetry, and inference privileges diverge before those paths converge into the same role.

Privilege blast radius is now measured in model outcomes as much as in infrastructure reach. That is why least privilege for AI must be tied to both access and effect. If a permission can change future decisions, the governance programme has to track the downstream behaviour, not just the immediate entitlement.

For practitioners

• Reclassify model-training permissions as privileged access Inventory every permission that can create, modify, or retrain AI models and place it under PAM review, approval, and logging. Do not leave these actions inside general developer entitlements or broad automation roles.
• Separate training rights from observability rights Split permissions that can write to model pipelines from permissions that can read deep telemetry or configuration data. Review both paths independently so that visibility access does not become an indirect route to model manipulation.
• Require provenance for every fine-tuning input Track who supplied the dataset, which identity approved the job, and whether the source data was vetted for poisoning or prompt-injection content. Treat unverified lineage as a governance failure, not a tuning quality issue.
• Add rollback criteria for behaviour-changing AI permissions Define what evidence would trigger reversal of a training job, model version, or privilege grant. If a permission can rewrite the model’s rules, the organisation needs a fast path to revert that state before the next downstream use.

Key takeaways

• Cloud model-fine-tuning permissions expand privilege risk from access control into model integrity.
• When a permission can change future AI behaviour, conventional access review cadences are no longer sufficient.
• IAM and PAM teams should govern model-shaping rights as privileged access with provenance and rollback controls.

Key terms

• Model Integrity: The trustworthiness of a model’s behaviour over time, including whether its outputs, safety filters, and decision patterns remain aligned with authorised intent. In cloud identity terms, it is what gets protected when a permission can retrain or modify the model rather than merely access it.
• Behaviour-Changing Permission: An entitlement that can alter what a system does in future sessions, not just what it can read or launch right now. For AI services, this includes fine-tuning and similar control-plane actions that can create lasting governance risk after the original access event ends.
• Privileged Observability: Telemetry or logging access that exposes deep operational detail about systems, pipelines, or AI behaviour. It helps defenders see what is happening, but if over-granted it can also help attackers tune abuse, test defences, or hide manipulation across repeated interactions.

Deepen your knowledge

NHI governance, agentic AI identity, and machine identity security are core topics in our NHI Foundation Level course, the industry's only accredited NHI security programme. If you are building or maturing an identity security programme, it is worth exploring.

This post draws on content published by Sonrai Security: Feb Recap. New AWS Privileged Permissions and Services. Read the original.