When the electronic health record, laboratory system or a key interface goes down, clinicians still have to identify patients, give medicines, order and read tests and hand over care, and downtime procedures decide whether that happens safely or by improvisation. Ransomware has turned downtime from a few planned hours into weeks: after the June 2024 attack on pathology provider Synnovis, NHS England reported that King's College Hospital and Guy's and St Thomas' had postponed 10,152 acute outpatient appointments and 1,710 elective procedures by late September. This guide covers the clinical side of EHR and clinical system downtime: types and triggers, downtime viewers and paper workflows, patient identification, medication and results safety, the identity and access arrangements downtime needs, communication, recovery and back-entry, drills and the US rules that require them. It is for clinical informatics, nursing and medical leaders, IT and security teams and emergency preparedness leads in hospitals and health systems.
Key takeaways
- Plan for weeks, not hours. Downtime plans built for a scheduled upgrade fail during ransomware recovery; design tiers that cover short, extended and prolonged loss of systems.
- Patient identification is the first safety risk. The ONC SAFER Contingency Planning guide makes accurate patient identification before, during and after downtime a recommended practice in its own right.
- Read-only access and paper are both needed. A downtime viewer with recent patient data gives clinicians history; paper forms and runners carry new orders, results and medication administration.
- Downtime needs identity planning. HIPAA requires an emergency access procedure, and downtime viewers, shared workstations and temporary accounts must be controlled and reconciled afterwards.
- Recovery is a clinical process. Back-entry of orders, results and medication records, and orderly interface restart, must be planned and staffed, or the record stays wrong long after systems return.
Types of downtime and how to trigger the plan
The SAFER Contingency Planning self-assessment, published by ONC on HealthIT.gov, addresses planned and unplanned EHR unavailability, including partial loss, and also treats unacceptably slow response time as "functional downtime" to be identified and addressed proactively.
| Type | Example | Typical duration | Plan emphasis |
|---|---|---|---|
| Planned | Upgrade or maintenance window | Minutes to hours | Scheduled at low activity; pre-printed reports; downtime viewer refreshed just before |
| Unplanned, short | Network, power or data centre fault | Hours | Fast activation, downtime viewer, paper orders and medication administration records |
| Partial or interface | Laboratory, pharmacy, radiology or HL7 interface failure while the EHR is up | Hours to days | Department-specific workflows; flag results and orders not flowing; orderly interface restart |
| Functional (slow system) | Response times too slow for safe use | Variable | Thresholds that trigger downtime even though the system is technically up |
| Extended cyber incident | Ransomware at the hospital or a supplier | Days to weeks or months | Sustained paper operations, staffing, mutual aid, diversion decisions, staged recovery |
| Third-party or regional | Clearing house, pathology provider or shared service outage | Days to weeks | Alternative suppliers, manual claims and results routes |
The Synnovis attack is the example of a supplier outage with clinical impact: NHS England reported that in the first week pathology was running at about 10% of normal capacity, 814 elective procedures and 736 hospital outpatient appointments were postponed, and NHS Blood and Transplant appealed for O-type blood donors. The Change Healthcare breach showed the same for an intermediary: providers across the US could not submit claims or check coverage for weeks.
Activation
- Named decision makers: who declares downtime (for example the IT service desk with the duty clinical lead), who escalates to a major or critical incident, and who decides on diversion.
- Clear triggers: loss of the EHR, loss of a critical department system, loss of an interface for more than a set time, or response times above a defined threshold.
- Downtime levels: for example level 1 (expected under four hours), level 2 (up to 24 hours) and level 3 (more than 24 hours or cyber incident), each with its own staffing, paper and communication steps. The thresholds are local decisions; write them down.
- Announce consistently: overhead or pager announcement, intranet banner if available, and a call cascade, using channels that do not depend on the EHR, as SAFER recommends.
Clinical workflows during downtime
Each clinical process needs a written downtime workflow, owned by the department, with the forms and roles to run it. The SAFER guide recommends keeping paper forms on hand to replace key EHR functions.
| Process | Downtime approach | Key safety control |
|---|---|---|
| Registration and patient identification | Pre-assigned downtime medical record numbers and wristbands; manual registration log | Two-identifier checks at every step; a single reconciliation list of temporary numbers |
| Orders (medications, labs, imaging) | Paper order sets; runners or tube system to departments | Legible, signed, timed orders; read-back for verbal and telephone orders |
| Medication administration | Printed medication administration records from the latest downtime report; paper eMAR | Pharmacist verification route; allergy information from the downtime viewer |
| Results | Laboratory and radiology phone critical results; paper reports delivered to wards | Critical result call log with read-back; clear marking of results not yet in the record |
| Blood transfusion | Manual cross-match and issue records; emergency group O stock | Strict sample labelling; separate downtime transfusion procedure agreed with the laboratory |
| Clinical documentation and handover | Paper notes and handover sheets filed with the patient | Scanned or back-entered after recovery; marked as downtime documentation |
| Discharge and transfers | Paper discharge summary and medication list | Copy to patient and receiving provider; medicines reconciliation on return to service |
| Claims and billing | Charge capture on paper; alternative clearing routes | Cash-flow plan for extended third-party outages |
Downtime viewers and pre-printed reports
- Read-only copy of recent data: most EHR platforms offer a business continuity or downtime viewer that holds recent census, medication, allergy, results and care plan data on designated workstations, refreshed on a schedule. Know how fresh it is and where it runs.
- Isolation from the outage: a viewer on the same network, domain and identity infrastructure as the EHR will be encrypted or unreachable in a ransomware event. Place some devices so they survive a domain-wide incident, with local power, a local printer and local accounts.
- Printed fallbacks: scheduled printouts of census and medication administration records for high-acuity units cover the period before the viewer is reachable.
- Backups and power: SAFER recommends regular, tested backups of patient data and critical configuration settings, including off-site and air-gapped copies, and a generator with fuel and an uninterruptible power supply to keep the EHR running through extended power loss.
Identity and access during downtime
Downtime changes who can access what, and how. HIPAA's technical safeguards require an emergency access procedure for obtaining necessary electronic protected health information (45 CFR 164.312(a)(2)(ii), Required), alongside unique user identification and audit controls. Plan the identity side explicitly.
- Downtime viewer access: decide whether viewers use individual accounts, which need a working directory, or local or shared accounts, which need tight physical control, short password lifetimes and post-event rotation. Record which approach each viewer uses.
- Break-glass accounts: emergency accounts for critical systems with credentials held offline in sealed envelopes or a local vault, use logged on paper, and rotation after every use. See the Break-Glass Account Guide.
- Smartcards and tap-and-go: if sign-in depends on a central identity service, know which clinical systems still accept cached credentials and for how long. See the Healthcare Identity Security Guide.
- Temporary accounts during recovery: accounts created for agency staff, vendors or incident responders while normal provisioning is down must be listed and removed when it is restored.
- Service accounts and interfaces: interface engines, integration accounts and device credentials are often reset or recreated during recovery; inventory them so that recovery does not leave default or shared credentials behind. See the HL7 v2 and Clinical Interface Engine Security Guide.
In a cyber incident, also assume that identity infrastructure itself may be compromised. Restore and verify directory services, privileged accounts and identity providers before reconnecting clinical systems to them.
Communication
- Independent channels: SAFER recommends communication methods that do not depend on the EHR's computing infrastructure: overhead paging, radios, landlines, mobile phones on a separate network and printed contact lists.
- Who needs to know: clinical units, pharmacy, laboratory, radiology, emergency department, bed management, ambulance services, neighbouring hospitals, GPs and referrers, and patients with appointments.
- Regular status updates: fixed-time briefings with the expected duration, what works, what does not, and the next update time.
- Sharing patient information with other providers: hospitals participating in Medicare and Medicaid must have a communication plan that includes a method for sharing information and medical documentation for patients under their care with other providers (42 CFR 482.15(c)), which matters when patients are diverted or transferred.
Recovery, back-entry and interface restart
Recovery is where downtime errors become permanent. Plan and staff it as carefully as the downtime itself.
- Decide what is back-entered: active medication orders, allergies, results, vital signs for inpatients and key documentation are typically entered; other paper is scanned. Agree this in advance.
- Restart interfaces in order: SAFER recommends defined procedures for stopping and restarting data exchange across interfaces in an orderly way. Queued messages may arrive late, duplicated or out of order; check results and orders after restart.
- Reconcile patient identities: merge downtime medical record numbers into permanent records, checking for duplicates and overlays.
- Reconcile medications: pharmacist-led reconciliation for every inpatient after back-entry, comparing paper administration records with the restored record.
- Close out access: rotate break-glass and shared downtime credentials, remove temporary accounts and review audit logs for downtime access.
- Review: SAFER recommends an in-depth root cause review of unexpected downtimes lasting over 24 hours. Feed findings into the plan.
For a long cyber incident, recovery is staged by clinical priority. NHS England described Synnovis restoring systems by clinical priority before moving on to administrative systems. Agree the restoration order (for example laboratory, pharmacy, emergency department, theatres, then outpatient and administrative systems) with clinical leaders before an incident.
Training and drills
- Regular drills: SAFER recommends training and testing staff on downtime and recovery procedures, including drills. Hospitals subject to 42 CFR 482.15 must test the emergency plan at least twice a year, including a full-scale or functional exercise, and must analyse the response and revise the plan after drills and real events.
- Department practice: short, frequent unit-level drills (pull a paper downtime kit, admit a mock patient, give a mock medication) build familiarity that annual tabletop exercises do not.
- Include long scenarios: rehearse day three and week two of a ransomware event: staffing fatigue, paper supply, results backlogs, and when to stop elective work.
- Include new staff: clinicians trained only on electronic workflows may never have written a paper medication chart.
Rules and guidance that apply
| Source | Requirement or recommendation | Applies to |
|---|---|---|
| HIPAA Security Rule, 45 CFR 164.308(a)(7) | Contingency plan: data backup plan, disaster recovery plan and emergency mode operation plan (Required); testing and revision procedures and applications and data criticality analysis (Addressable) | US covered entities and business associates |
| HIPAA Security Rule, 45 CFR 164.312(a) and (b) | Emergency access procedure (Required); unique user identification (Required); automatic logoff (Addressable); audit controls | US covered entities and business associates |
| CMS Conditions of Participation, 42 CFR 482.15 | All-hazards emergency preparedness programme; medical documentation system that preserves and secures records and maintains their availability; communication plan; training at least every 2 years; testing at least twice a year | US hospitals participating in Medicare and Medicaid |
| ONC SAFER Guides: Contingency Planning | 13 recommended practices across safe health IT, using health IT safely and monitoring safety | Voluntary self-assessment for organisations using EHRs |
See the HIPAA Security Rule Guide and the Healthcare Cyber Regulation Map for the wider regulatory picture, including proposed changes to the Security Rule.
Detection and measurement
| Metric | What it shows | Target |
|---|---|---|
| Time from system failure to downtime declared | Activation speed | Minutes, against a defined threshold |
| Units with a current downtime kit and trained staff on shift | Readiness | 100% of clinical units |
| Downtime viewer data age at the start of an outage | Usefulness of read-only data | Within the configured refresh interval |
| Viewers that survived the last exercise or incident | Isolation of the fallback | All designated devices |
| Back-entry completion time and error rate | Recovery quality | Completed within the agreed window; errors trending down |
| Downtime-related patient safety incidents | Clinical risk | Each reviewed; trend falling |
| Break-glass and temporary accounts closed after recovery | Access hygiene | 100% |
Common failure patterns
- Downtime viewers that go down with everything else: the business continuity devices sit on the encrypted domain and are unusable in a ransomware event.
- Plans written for four hours: paper supplies, staffing and results handling collapse after the first few days of an extended outage.
- Patient identification errors: temporary numbers are issued from several lists and patients end up with duplicate or merged records.
- Uncontrolled interface restart: queued messages flood in after recovery and late or duplicated results are acted on without review.
- Shared downtime passwords that never change: the same downtime account password has been on a sticky note in the ward office for years.
- Single supplier dependency: a pathology or clearing house provider has no alternative route, as the Synnovis and Change Healthcare incidents showed.
Framework mapping
| Framework | Reference | How it applies |
|---|---|---|
| NIST SP 800-53 Rev. 5 | CP-2 Contingency Plan; CP-4 Contingency Plan Testing; CP-9 System Backup; CP-10 System Recovery and Reconstitution; IR-4 Incident Handling | Downtime plans, drills, backups and staged recovery |
| NIST CSF 2.0 | PR.IR-03; RC.RP-01; RC.RP-03; RC.CO-03; ID.IM-04 | Resilience mechanisms, executing recovery, verifying backups, recovery communication, improving plans |
| MITRE ATT&CK v20 | T1486 Data Encrypted for Impact; T1490 Inhibit System Recovery | Why downtime viewers and backups must be isolated from the main estate |
| HIPAA and CMS | 45 CFR 164.308(a)(7), 164.312(a)(2)(ii); 42 CFR 482.15 | Contingency, emergency access, emergency preparedness and testing |
Practitioner checklist
- Define downtime types, levels and triggers, including functional downtime thresholds, and name who declares them.
- Write a downtime workflow for every critical clinical process, owned by the department, with forms in a downtime kit on every unit.
- Run downtime viewers with known refresh intervals on devices isolated from a domain-wide incident, with local power and printing.
- Pre-assign downtime medical record numbers and wristbands and keep one reconciliation list.
- Plan downtime identity: viewer accounts, break-glass credentials held offline, temporary accounts and post-event rotation.
- Set up communication channels that do not depend on the EHR, and a status update rhythm.
- Plan back-entry, interface restart, identity and medication reconciliation, and staff them.
- Drill at unit level frequently and run at least one extended cyber scenario a year; meet the 42 CFR 482.15 testing schedule where it applies.
- Review every unplanned downtime over 24 hours and update the plan.
Standards and references
- HealthIT.gov: SAFER Guides
- HealthIT.gov: SAFER Guide, Contingency Planning
- eCFR: 45 CFR 164.308 Administrative safeguards
- eCFR: 45 CFR 164.312 Technical safeguards
- eCFR: 42 CFR 482.15 Condition of participation: Emergency preparedness
- NHS England: Update on cyber incident clinical impact in south east London, 14 June 2024
- NHS England: Update on cyber incident clinical impact in south east London, 26 September 2024
Related NHI Mgmt Group resources: Degraded-Mode Operations and Manual Workarounds Guide · Healthcare Identity Security Guide · Hospital Clinical Network Segmentation Guide · Healthcare Cyber Regulation Map · Break-Glass Account Guide