Accountability is shared across the service provider, the customer using the platform, and the defenders responsible for identity and fraud monitoring. The provider must enforce abuse controls and takedown processes. Security teams must assume that legitimate platforms can be weaponised and should build response workflows that detect, evidence, and contain reuse quickly.
Why This Matters for Security Teams
When an AI service is used to host phishing infrastructure, the accountability question is not academic. Abuse can begin with a legitimate tenant, a stolen NHI, or a misconfigured automation workflow, then expand into fraud, credential theft, and brand damage before normal review cycles catch it. NHI Management Group has documented adjacent AI abuse patterns in LLMjacking, where compromised non-human identities are used to hijack AI services for attacker-controlled activity. Security teams should treat the platform, the tenant, and the identity layer as separate accountability domains, not one shared blame bucket.
Current guidance suggests that providers remain responsible for abuse handling, while customers remain responsible for what they deploy and which identities they expose. That split matters because phishing infrastructure often appears as ordinary application traffic until it is used at scale. Control failures usually show up first as weak telemetry, missing takedown paths, or over-permissive service accounts. In practice, many security teams encounter the abuse only after a phishing kit has already been operational long enough to harvest credentials and rotate domains.
How It Works in Practice
Operational accountability starts with knowing which identity created the workload, which tenant owns it, and which logs can prove intent. The provider should enforce acceptable-use controls, abuse detection, and rapid suspension workflows. The customer should scope permissions tightly, review delegated access, and secure secrets and API keys so that one compromised credential cannot launch a phishing cluster. This is where workload identity, short-lived credentials, and strong audit logging become more important than traditional user-centric controls.
Security teams should align response playbooks to the evidence chain, not just the incident label. For example, if a service instance is used to send credential-harvest links, teams need to preserve:
- Identity provenance for the workload, including the issuing account and token source.
- Request and activity logs showing when the phishing assets were created and by whom.
- Secrets and token handling history, especially for reused or long-lived credentials.
- Takedown contacts and escalation paths for the provider, registrar, and hosting layer.
That approach matches the spirit of NIST SP 800-53 Rev 5 Security and Privacy Controls, which expects accountability, auditability, and incident response to be assigned clearly across shared environments. It also reflects the pattern described in CoPhish OAuth Token Theft via Copilot Studio, where agentic abuse can pivot from a trusted platform into token theft and phishing operations. These controls tend to break down when platform telemetry is fragmented across vendors and the customer cannot correlate identity events to content generation events.
Common Variations and Edge Cases
Tighter abuse controls often increase friction for legitimate automation, requiring organisations to balance rapid incident containment against developer velocity and tenant independence. The main edge case is hosted AI infrastructure that behaves like a general-purpose application platform, because responsibility can blur between platform terms of service, tenant configuration, and downstream content generated by end users. There is no universal standard for this yet, so current guidance suggests documenting ownership explicitly in service contracts and internal control mappings.
Another frequent exception is reseller or managed-service deployment, where a third party provisions the AI service on behalf of the customer. In that model, accountability is still shared, but the operational burden shifts toward the party that controls configuration, secrets, and abuse escalation. Security teams should also watch for collateral impact: a single abused service can trigger IP reputation loss, OAuth token abuse, or domain-level blocking beyond the original tenant. NHI Management Group’s DeepSeek breach research shows how exposed secrets and exposed systems can quickly widen the blast radius when governance is weak. The hard limit is multi-tenant environments with minimal logging, because attribution becomes too weak to prove which actor actually launched the phishing infrastructure.
Standards & Framework Alignment
This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.
OWASP Non-Human Identity Top 10, OWASP Agentic AI Top 10 and CSA MAESTRO address the attack and risk surface, while NIST CSF 2.0 and NIST AI RMF set the governance and control requirements practitioners need to meet.
| Framework | Control / Reference | Relevance |
|---|---|---|
| OWASP Non-Human Identity Top 10 | NHI-06 | Covers detection and response for abused non-human identities. |
| OWASP Agentic AI Top 10 | A-04 | Agentic abuse can weaponise trusted services into phishing infrastructure. |
| CSA MAESTRO | ID-02 | Shared accountability depends on clear identity ownership and governance. |
| NIST CSF 2.0 | RS.CO-2 | Incident communications must support provider-customer coordination during abuse. |
| NIST AI RMF | GOVERN | AI risk governance requires clear accountability for harmful downstream use. |
Assign identity ownership for each AI service and document who can create, revoke, and investigate it.
