AI Incidents vs Security Incidents: Classification, Ownership and Reporting

When an AI system misbehaves, the first hours are often lost deciding whether it is a security incident, an AI incident, a privacy breach or all three, and each answer starts a different clock with a different owner. A prompt injection that leaks customer data is a security incident and a personal data breach; a hiring model that starts rejecting a protected group is an AI incident with no attacker; a hijacked agent that pays a fraudulent invoice can be both. This guide gives a classification method for these mixed cases: the criteria that separate the two incident types, a triage decision tree that assigns an owner and identifies which reporting clocks may have started, and rules for sharing evidence between the security and AI teams without destroying it. It is for incident managers, SOC leads, AI governance leads and legal counsel who must agree routing before the next incident, not during it.

Key takeaways

• Classify by harm and cause, not by technology. A security incident is defined by compromise of confidentiality, integrity or availability; an AI incident by harm that an AI system's development, use or malfunction leads to. Many incidents are both.
• One intake, several labels. Run every AI-related report through one triage step that can attach security, AI harm and privacy labels at once, instead of separate queues that each decide the case is someone else's.
• Labels start clocks, owners run them. The security label drives the incident process; the AI harm label can start EU AI Act serious incident reporting; the privacy label can start a 72-hour breach notification. Legal decides notification, the incident manager runs the response.
• Containment can conflict with AI reporting. Article 73(6) of the EU AI Act stops a provider altering a high-risk AI system in a way that could affect later evaluation of the incident's causes before informing the authorities. Snapshot first, then change.
• Evidence must serve both investigations. Prompts, outputs, tool calls, model and prompt versions and retrieval context are AI evidence; logs, credentials and network data are security evidence. Collect both once, under one chain of custody.

The short answer

A security incident is an event that actually or imminently jeopardises the confidentiality, integrity or availability of a system or its information, usually through compromise or misuse. NIST SP 800-61 Rev. 3, the CSF 2.0 community profile for incident response, frames handling around triage, categorisation, escalation, analysis and communication (CSF 2.0 RS.MA-02, RS.MA-03, RS.MA-04).

An AI incident, in the OECD definition, is an event, circumstance or series of events where the development, use or malfunction of one or more AI systems directly or indirectly leads to harm: to people's health, to critical infrastructure, to human rights or legal obligations protecting fundamental, labour and intellectual property rights, or to property, communities or the environment. The OECD calls the near miss, an event that could plausibly lead to such harm, an AI hazard. The EU AI Act's narrower "serious incident" (Article 3(49)) covers death or serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of Union law obligations protecting fundamental rights, and serious harm to property or the environment.

The key difference: a security incident needs a breach of protection; an AI incident needs harm, and the cause can be ordinary model behaviour with no adversary. The general escalation process for AI incidents is in AI Incident Management and Escalation; this page is about the boundary cases.

Classification criteria compared

Threat model and failure modes for each process

Security incident process

• What it is built for: adversaries and accidents that breach protection, including attacks on AI: prompt injection, model or data poisoning, theft of model weights, abuse of AI API keys and hijacking of agent credentials.
• Architecture in words: detections and reports reach the SOC; an analyst triages and categorises; an incident manager declares and runs containment, eradication and recovery; legal and privacy decide notification; lessons learned update controls.
• Failure modes for AI cases: closing a ticket as "no compromise" when an agent behaved harmfully within its permissions; lacking the prompts, retrieval context and tool-call logs needed to reconstruct what the model saw; containing by redeploying the model and overwriting the version that caused harm.

AI incident process

• What it is built for: harm from AI behaviour: unsafe or false outputs, discriminatory outcomes, failed human oversight, unexpected autonomous actions, and incidents in third-party models the organisation depends on.
• Architecture in words: reports from users, overseers and monitoring reach the AI system owner; the AI governance lead classifies harm and tier; the system is suspended or rolled back; root cause analysis covers data, model, prompt and context; the provider is informed where the organisation is a deployer; regulatory reporting is assessed.
• Failure modes for security cases: treating an injected instruction as a model quality defect and retraining, while the attacker's content is still in the knowledge base; never telling the SOC, so the same technique hits other systems; resetting the agent without revoking its credentials.

Overlapping cases and who should own them

The rule behind the table: if confidentiality, integrity or availability is affected, security leads, because containment and evidence preservation are time-critical and the SOC has the tooling. If no protection was breached, the AI system owner leads under governance oversight. In every dual case the other team is a named deputy with its own workstream, not a consultee. A fuller agent case is in the AI Agent Incident Response Playbook.

Triage decision tree

Run these questions in order at intake. Each "yes" adds a label; labels are cumulative.

1. Is an AI system involved? Include models embedded in products, AI features in SaaS, agents and third-party AI services. If yes, add the system's register entry and tier to the ticket. If you cannot find an entry, flag it as shadow AI for governance.
2. Is there evidence of compromise or misuse of access? Unauthorised access, stolen credentials, injected instructions from untrusted content, tampered data or models, or exfiltration. If yes, add the security label; the security incident manager leads.
3. Has the AI system caused, or could it plausibly cause, harm to people, rights, property, infrastructure or the environment? If caused, add the AI harm label; if only plausible, record an AI hazard. The AI system owner leads unless step 2 assigned security.
4. Could the harm meet the EU AI Act serious incident definition, and is the system high-risk? Check Article 3(49) points (a) to (d). If yes, legal opens a reporting assessment; as a provider, Article 73 applies; as a deployer, Article 26(5) requires you to inform the provider first, immediately.
5. Is personal data affected? If yes, add the privacy label; the data protection officer assesses risk to individuals. Under UK GDPR the ICO expects notification of reportable breaches without undue delay and where feasible within 72 hours.
6. Is the organisation in scope of NIS2, DORA or a sector rule, and is the service significantly affected? If yes, add the regulated service label and run that regime's significance test.
7. Is a third party's model, data or service the cause? If yes, notify the vendor through the contracted route and open a vendor workstream.
8. Set severity using the highest label. The case takes the worst severity any label implies; do not average.

Record the time each label was added and the reason. Most AI and cyber reporting deadlines run from awareness or determination, so the label time becomes the evidence of when the clock started. The deadlines themselves are tabulated in the AI Incident Reporting Clocks Map.

Which reporting clocks start

EU AI Act high-risk obligations, including Article 73 reporting, apply on the dates the Act sets for high-risk systems, which the Digital Omnibus moved; confirm current application dates before relying on them. Article 73(5) allows an incomplete initial report followed by a complete one, which suits mixed incidents where root cause is still open.

Evidence: what each team needs and how to share it

• One evidence store: both teams work from a single incident evidence location with access logged, meeting CSF 2.0 RS.AN-06 and RS.AN-07 on integrity and provenance. The Evidence Handling and Chain of Custody Guide covers custody rules.
• Snapshot before change: for a high-risk system, any containment that alters the model or configuration waits for a snapshot, and for providers the authority is informed first under Article 73(6). Use traffic controls (suspend the use case, route to a fallback) instead of changing the system.
• Log retention: deployers of high-risk systems must keep the logs they control for at least six months (Article 26(6)); align AI logging retention with security log retention so neither investigation finds gaps.
• Privilege: if counsel directs the investigation, agree how AI evaluation work is labelled. See the Legal Privilege in Incident Response Guide.

Worked scenario: a poisoned support knowledge base

A customer support assistant starts telling some customers to verify their accounts on an external site. A customer service lead reports it to the AI system owner as a quality issue.

1. Intake: the owner logs it through the single AI incident intake. Step 1: the assistant is in the register as a medium-tier, customer-facing system.
2. Step 2: the triage analyst finds the instructions come from a help article edited through a compromised contributor account. Security label added at 10:40; the security incident manager takes the lead.
3. Step 3: customers following the link risk credential theft. AI harm label added; AI owner is deputy lead.
4. Step 5: the external site captures customer credentials. Privacy label added; the DPO starts the risk assessment and logs the awareness time.
5. Containment: the assistant is switched to a fallback mode answering only from a frozen, verified article set. The knowledge base index and the edited article are snapshotted before removal; the contributor account is disabled and its sessions revoked.
6. Close-out: security fixes contributor access; the AI owner adds output filtering for external links and a content approval step; legal closes the notification decisions with timestamps from the ticket.

Implementing joint triage

1. Agree definitions: adopt the security incident definition from your incident response plan and the OECD AI incident and AI hazard definitions, with Article 3(49) as the serious incident test where the EU AI Act applies.
2. Build one intake: add an "AI system involved" field to the existing incident form and give customer service, human overseers and model monitoring a route into it.
3. Write the decision tree into the runbook: label rules, lead and deputy per label combination, and the timestamp field for each label. Map owners to the Incident Response RACI Template.
4. Pre-stage AI evidence collection: make sure prompts, tool calls, model and prompt versions and retrieval context are logged and exportable before you need them.
5. Exercise mixed cases: run a tabletop for each row of the overlap table, timing how long routing takes. Roll back to the previous routing by keeping the old queues live during the pilot.
6. Review quarterly: sample closed tickets for mislabelled cases and update the tree.

Detection and measurement

• Time to label: median time from first report to all applicable labels set; the target is hours, because privacy and NIS2 clocks run in hours.
• Reclassification rate: share of AI-related cases whose lead changed after triage; high numbers mean the tree is unclear.
• Bounced tickets: AI cases closed by one team as "not ours" and reopened by another.
• Evidence completeness: share of AI incidents where prompts, versions and tool calls were preserved before containment.
• Hazard reporting: near misses recorded as AI hazards; zero usually means under-reporting, not safety.

Framework mapping

Common failure patterns

• The "no compromise" close: the SOC closes an agent incident because no attacker was found, and nobody opens the AI harm workstream.
• The quality-defect misroute: an injection is treated as a model accuracy issue. The Gemini calendar prompt injection disclosure shows how instructions hidden in ordinary content can produce output that looks like normal assistant behaviour while exposing private data.
• Overwritten evidence: the team redeploys a fixed prompt or model and loses the version that caused the harm.
• Clock started without a timestamp: awareness was obvious in chat messages days before anyone recorded it, which a regulator can treat as the start.
• Deployer waiting on provider: a deployer waits for the vendor's root cause before acting, while Article 26(5) expects the provider to be informed immediately.
• Agent reset without revocation: the agent is restarted but its tokens remain valid. See Revoking Delegated Access During an Active Agent Incident.

Practitioner checklist

• Adopt written definitions for security incident, AI incident, AI hazard and serious incident.
• Route all AI-related reports through one intake with an "AI system involved" field linked to the AI register.
• Put the triage decision tree in the incident runbook with lead and deputy for each label combination.
• Timestamp each label at the moment it is added and keep the reason.
• Pre-configure logging for prompts, outputs, tool calls, model and prompt versions and retrieval context.
• Snapshot model, configuration and knowledge base state before any containment that changes them.
• Use one evidence store with integrity hashes and access logs for both investigations.
• Confirm in each AI vendor contract how and how fast incidents are reported in both directions.
• Exercise at least one mixed incident a year with security, AI, privacy and legal in the room.

Standards and references

• European Commission AI Act Service Desk: Article 73, Reporting of serious incidents
• European Commission AI Act Service Desk: Article 3, Definitions
• European Commission AI Act Service Desk: Article 26, Obligations of deployers of high-risk AI systems
• OECD.AI: AI Incidents and Hazards Monitor, overview and methodology
• NIST: SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management
• NIST: AI Risk Management Framework Core
• ICO: Personal data breach reporting
• NIST: Cybersecurity Framework 2.0

Related NHI Mgmt Group resources: AI Incident Management and Escalation · AI Incident Reporting Guide · AI System Incident Report Template · AI Governance vs AI Security