Problem Management for Security: Fix Repeat Incidents

Security teams close the same phishing compromise, the same exposed storage bucket and the same over-privileged service account again and again, because each ticket is resolved as an incident and nobody owns the underlying cause. IT service management solved this decades ago with problem management: a separate record and process for causes, with known errors, workarounds and permanent fixes tracked to closure. Security operations rarely uses it, and post-incident reviews produce actions that drift once the incident is closed. This guide shows how to run problem management for security: when to open a problem, how to write a security problem record, how known errors and workarounds work for control gaps, how to prioritise and verify fixes, and how to tell whether repeat incidents are falling. It is for SOC leads, incident managers, IT service management owners and control owners.

Key takeaways

Incidents restore; problems remove causes. Keep the two records separate so closing an incident never closes the work on its cause.
Open problems from patterns, not only from big incidents. Clusters of low-severity alerts and repeated near misses are often the best problem candidates.
A known error needs a named control owner. The problem record states which control failed or was missing and who owns its fix.
Workarounds are temporary risk acceptances. Give each one an expiry date and a monitoring rule, or it becomes the permanent control.
Close a problem only when recurrence stops. Verification is a period of evidence after the fix, not the date the change was deployed.

Why security needs problem management

Most security incident processes end with eradication, recovery and a lessons-learned meeting. The Major Incident Management guide covers the handover from a major incident to a problem record, and the Post-Incident Review and Root Cause Analysis Template covers running a review. What is usually missing is the step after: a single register of causes, each with an owner, a fix and proof that it worked, that persists across many incidents and many reviews.

The NCSC Cyber Assessment Framework describes the target state well. Under Principle D2, lessons learned, the "not achieved" indicators include incidents not usually being traced to a root cause or to contributing factors in the wider system, and lessons not being implemented or being given low priority. The "achieved" indicators include a documented review process that captures lessons from every incident, including near misses, and improvements that are prioritised with the highest-priority items completed promptly and reported to senior management. Problem management is the operational machinery that produces those outcomes.

Core concepts adapted for security

Concept	ITSM meaning	Security adaptation	Example
Incident	An unplanned interruption or reduction in quality of a service	A security event that needs response	Mailbox compromised through a phished session token
Problem	A cause, or potential cause, of one or more incidents	A control gap, design weakness or process failure behind one or more security incidents or near misses	Session tokens for webmail remain valid after password reset
Known error	A problem that has been analysed but not resolved	A confirmed control gap with an agreed cause and a fix owner	Token revocation step missing from the account compromise runbook and identity platform automation
Workaround	A way to reduce or remove the impact of incidents for which a full resolution is not yet available	A compensating control with an expiry date	Analysts manually revoke sessions on every compromise ticket
Permanent fix	A change that removes the cause	A control change, usually through change management	Automated session revocation on password reset and risky sign-in

IT service management practice commonly splits problem work into three activities: identifying problems, analysing them while documenting workarounds and known errors, and managing known errors through to a permanent fix. The ITIL 4 Information Security Management Guide places security within the ITIL practice model; this page applies the problem practice to security causes.

When to open a security problem

Our recommendation at NHI Mgmt Group is to define triggers in advance, so opening a problem is routine rather than a judgement made under pressure. Google's SRE guidance makes the same point for postmortems: define criteria before an incident occurs and let any stakeholder request one.

Every high-severity security incident: Open a problem by default; close it as "no systemic cause" only after analysis says so.
Recurrence threshold: Three or more incidents in 90 days with the same technique, asset class or control failure, for example repeated MFA fatigue approvals or the same misconfigured storage type.
Detection failures: Any incident found by a third party, a user or by chance rather than by monitoring.
Near misses: Blocked attacks that reached the last control in the chain, such as malware stopped only at execution.
Audit and test findings: Penetration test or red team findings that map to a real incident category. NIST CSF 2.0 ID.IM-02 covers improvements from tests and exercises.
Repeat exceptions: The same policy exception requested by many teams, which signals a control that does not fit how work is done.

Writing the security problem record

A problem record is the long-lived home of a cause. It links to every incident it explains and every change that addresses it.

Field	Content
Problem statement	One sentence describing the cause, not the symptom ("service account keys are created without expiry" rather than "key leaked again")
Linked incidents and near misses	All incident references, with dates and severity
Failed or missing control	The control, standard or runbook step that should have prevented or detected it, with framework reference if used
Contributing factors	Organisational, technical and human factors, and any change in threat (the CAF D2.a list)
Control owner	The person accountable for the fix, outside the SOC where the control lives elsewhere
Workaround and expiry	The compensating action, who performs it, and the date it must be replaced
Risk rating	Likelihood of recurrence and impact, using the enterprise risk scale
Fix and change reference	The permanent change, linked to the change record
Verification criteria	What evidence, over what period, proves recurrence has stopped

Analysis methods

Timeline first: Build the timeline across all linked incidents before asking why. Patterns often appear in where the attacker's path crossed the same control.
Five whys and fishbone: Useful for process gaps, provided each answer is backed by evidence and the analysis does not stop at a person.
Control path analysis: For each linked incident, list each preventive and detective control on the attack path and whether it was absent, bypassed or failed. Repeated failures point at the known error.
Alternative outcomes: The CAF asks analysts to consider "what if" and "if only" scenarios. Ask what would have happened if the attacker had targeted a more privileged account.
Avoid early theories: CAF lists investigators who form theories early and seek only supporting evidence as a "not achieved" indicator.

Workarounds and permanent fixes

Security workarounds often become permanent because the incident pressure has passed. Treat each one as a temporary risk acceptance.

Workaround type	Example	Maximum life (ours)	Watch for
Manual runbook step	Analyst revokes tokens by hand	90 days	Missed steps in busy periods
Temporary detection rule	Alert on a specific indicator	60 days	Alert fatigue, narrow coverage
Blocking rule	Block a file type or domain	Until fix verified	Business bypass requests
Access restriction	Remove a role from a team	Until fix verified	Shadow workarounds by users

Permanent fixes go through change management (see Change Management Security), with the problem reference on the change. Fixes that change identity controls, such as removing standing privileges or adding key expiry, often need a staged rollout; the problem stays open until the last stage is verified.

Prioritising problems

A problem backlog competes with project work. Prioritise by risk, not by the noise the last incident made.

Recurrence and blast radius: Frequency of linked incidents multiplied by the impact of the worst plausible case, not the worst actual case.
Control leverage: Fixes that close many problems at once, such as phishing-resistant MFA for administrators, rank above single-incident fixes.
Workaround cost: Analyst hours spent each month on the workaround are a direct, reportable cost of leaving the problem open.
External signal: CAF D2.b expects organisations to learn from incidents in their sector; sector incidents that match an open problem raise its priority.

Report the top problems, their owners and age to the security steering group monthly. NIST SP 800-53 PM-4 and CA-5 describe plan of action and milestones processes that fit this reporting if you already use them.

Where identity fits

Many recurring security problems are identity problems: long-lived keys, shared service accounts, missing session revocation, help desk resets without strong verification, and OAuth grants that nobody reviews. They recur because the fix sits with identity or platform teams rather than the SOC. Name the identity control owner on the problem record and link the relevant standard, for example the Service Account Security Guide for repeat service account misuse or the Account Recovery and Help Desk Security Guide for repeat reset abuse.

How to implement

Agree definitions: Adopt the incident, problem, known error and workaround definitions above in the incident response plan and the ITSM tool.
Create the record type: Use the existing ITSM problem module with a security category and the fields above, restricted to the security and control owner groups where details are sensitive.
Set triggers: Publish the opening criteria and assign a problem manager role, separate from the incident manager.
Backfill: Review the last 12 months of security incidents, cluster them by technique and control, and open problems for the top five clusters.
Run a monthly problem review: Walk the open problems, workaround expiries and verification evidence with control owners.
Link to risk: Record long-running known errors in the risk register so acceptance is explicit and owned.
Adjust: After two quarters, retire triggers that create noise and add any that missed real patterns.

Detection and measurement

Clustering: Tag incidents with ATT&CK technique, asset class and failed control so recurrence can be counted automatically.
Recurrence watch: After a fix, alert if a new incident links to a closed problem; reopen it rather than raising a new one.

Metric	What it shows
Repeat incident rate (incidents linked to an existing problem)	Whether causes are being removed; should fall over time
Median age of open security problems by priority	Whether fixes are delivered
Workarounds past expiry	Hidden risk acceptance
Problems reopened after closure	Quality of fixes and verification
High-severity incidents with a linked problem record	Process coverage; target 100%
Analyst hours spent on workarounds per month	Cost of open problems

Common failure patterns

Problem closed with the incident: The tool or the team closes the cause when service is restored.
Actions owned by the SOC for controls it does not run: The fix belongs to identity, cloud or application teams who were never assigned it.
Blame as root cause: "User clicked a link" is recorded as the cause; the control that let one click become a compromise is never examined.
Endless workarounds: Manual steps added after each incident accumulate until the runbook is unworkable.
No verification period: A fix is marked done on deployment and the next recurrence is logged as a new problem.
Near misses ignored: Only damaging incidents trigger analysis, so causes are found late.

Framework mapping

Framework	Reference	Relevance
NIST CSF 2.0	RS.AN-03	Analysis to establish what happened and the root cause
NIST CSF 2.0	ID.IM-01; ID.IM-02; ID.IM-03; ID.IM-04	Improvements from evaluations, tests and operations; plans maintained and improved
NIST SP 800-53 Rev. 5	IR-4 Incident Handling; CA-5 Plan of Action and Milestones; PM-4 Plan of Action and Milestones Process; SI-2 Flaw Remediation	Lessons into handling, tracked remediation
NCSC CAF	D2.a Post incident analysis; D2.b Using incidents to drive improvements	Root cause and improvement outcomes
NIST SP 800-61 Rev. 3	Incident response as a CSF 2.0 community profile	Places improvement inside risk management
ISO/IEC 20000-1:2018	Service management system requirements	Certifiable service management context for problem management

Practitioner checklist

Keep incident and problem records separate and linked.
Publish triggers for opening security problems, including near misses and recurrence thresholds.
Record the failed or missing control and a named control owner on every problem.
Give every workaround an expiry date and review expiries monthly.
Route permanent fixes through change management with the problem reference.
Define verification evidence and period before closing.
Report top problems, age and repeat incident rate to the steering group.
Backfill the last year of incidents into clusters and open the top problems.

Problem Management for Security: Turning Repeat Incidents Into Owned Control Fixes