What are the signs that a SOAR program is not working as intended?

A SOAR program is struggling when analysts still spend hours on repetitive tasks, workflows remain too manual, containment happens late, or playbooks vary by responder. Other warning signs include limited integration coverage, inconsistent alert enrichment, and patching or phishing response still depending on ad hoc actions instead of automated, repeatable execution.

How to Tell a SOAR Program Has Not Moved Beyond the Pilot Stage

A SOAR program is usually failing in practice when it still behaves like a set of isolated automations rather than an operational capability. The clearest signal is not that no playbooks exist, but that analysts continue to work around them because the workflows are brittle, incomplete, or too slow to trust. When enrichment, containment, and ticketing are still handled manually for common alert types, the program has not reduced response friction in a meaningful way.

The practical test is whether the platform changes day-to-day operations under real alert volume. If teams only use SOAR for a few low-risk cases, if integrations stop at the perimeter of the stack, or if responders re-enter the same data across multiple tools, the program is delivering partial value at best. NIST SP 800-53 Rev. 5 is useful here because it frames automation as part of repeatable control execution, not as a dashboard feature, and that distinction matters when teams assess whether orchestration is actually improving response consistency. In practice, many security teams discover a SOAR problem only after analysts have already built their own shadow processes around the official playbooks.

Where SOAR Breaks Down in Real Operations

SOAR works when detection, triage, enrichment, decision support, and response actions are connected tightly enough that the workflow is repeatable across incidents. It does not need to eliminate human judgment, but it should remove the repetitive parts that slow down response and create variation between analysts. A healthy program usually has a narrow set of high-confidence automations first, then expands as the organisation proves that those actions are accurate, monitored, and maintainable.

Failure usually shows up in one of three places. First, the integrations are shallow, so the platform can open tickets or send notifications but cannot actually execute the response steps that matter. Second, the playbooks are too rigid, so they break when the alert contains missing fields, unusual context, or a different asset type. Third, ownership is unclear, so engineers, analysts, and incident responders each assume someone else is maintaining the workflow. That leaves automations stale, and stale automations quickly become ignored automations.

Look for repeated manual handoffs in tasks that should be fully or mostly deterministic.
Check whether enrichment sources are reliable enough to support action, not just display context.
Verify that analysts can explain when a playbook should run, when it should pause, and who approves exceptions.
Confirm that response actions are logged in a way that supports audit, review, and post-incident learning.

When SOAR is working well, the strongest evidence is not speed alone but consistency under normal operational load. If the program depends on a few expert users, collapses when inputs are messy, or cannot be trusted for common incident classes, it is functioning more like a demo environment than an operational control.

Common Failure Patterns That Make SOAR Look Better Than It Is

Tighter automation often reduces analyst effort, but it also increases dependency on clean data, stable integrations, and disciplined workflow ownership, so organisations must balance speed against brittleness. One common failure pattern is to automate the visible part of the process while leaving the actual containment decision manual, which creates the appearance of maturity without changing response outcomes.

Another problem is overfitting playbooks to a narrow alert shape. That can make the first use case look successful while masking the fact that the same workflow fails when the alert source, endpoint type, or business context changes. Industry guidance here is consistent even if implementation styles differ: a SOAR program should be judged on operational repeatability, not on the number of automations in the catalog. That is where the boundary between useful orchestration and decorative automation becomes obvious.

If your analysts are still rebuilding the same context by hand, if response is delayed by approval loops that were never designed into the workflow, or if the platform only works when the alert is already clean and complete, the program has not yet become resilient enough to depend on.

Standards & Framework Alignment

This section maps relevant standards and security frameworks to the operational risks and controls described in this guidance.

MITRE ATT&CK address the attack and risk surface, while NIST CSF 2.0 and CIS Controls v8 set the governance and control requirements practitioners need to meet.

Framework	Control / Reference	Relevance
NIST CSF 2.0	RS.AN-1 — Analysis	SOAR failure shows up in weak incident analysis and manual handoffs.
	RS.MI-1 — Mitigation	Late containment indicates mitigation actions are not being executed effectively.
Recommendation — Use RS.AN-1 to validate that automated triage improves incident analysis quality. Apply RS.MI-1 to automate and verify rapid containment actions for common incidents.
CIS Controls v8	17.4 — Incident Response Playbooks	SOAR playbooks should make response repeatable and testable.
	8.2 — Audit Log Management	SOAR needs reliable logging to support review and operational learning.
Recommendation — Use 17.4 to standardise and test response playbooks that analysts can execute consistently. Use 8.2 to retain workflow execution evidence for review and post-incident analysis.
MITRE ATT&CK	T1566 — Phishing	Phishing response is a common SOAR use case when automation is expected.
Recommendation — Map phishing workflows to T1566 and verify automated containment triggers fire reliably.

Practitioner Guidance

What to prioritise: Measure the response steps that still require human copying, reformatting, or rekeying. Those are the strongest indicators that the programme has not removed operational friction, even if the SOAR console shows activity.

What to verify: Test the playbooks against incomplete alerts, duplicate alerts, and common exceptions. If the workflow only succeeds with ideal inputs, it is not ready for consistent production use.

Common mistake: Treating automation count as success. A small number of dependable, well-owned workflows is more valuable than a large library that analysts bypass when incidents become messy.

Practitioner takeaway: A SOAR programme is working only when responders trust it enough to use it under normal pressure, not just when the incident is simple.