Most IT runbooks say what to type but not who may type it, with which account, after whose approval, or what record proves it happened. That gap is where privileged operations go wrong: an engineer uses a standing domain admin account because the runbook did not say otherwise, a script runs under a shared service account nobody owns, or an auditor asks for evidence of a production change and gets a chat thread. This guide sets out a house standard for privileged operations runbooks: the identity, approval and evidence fields every runbook should carry, how to tier runbooks by risk, how to write them so automation and AI agents can execute them safely, and how to keep them current. It is written for IT operations leads, platform and infrastructure teams, PAM owners and the auditors who test them. Task-specific runbooks already exist on this site, such as the Secrets Rotation Runbook Template and the SOC Runbook Template; this page is the standard they and your own procedures should follow.
Key takeaways
- Every privileged step names its identity. A runbook should say which role or account performs each step and how that privilege is obtained, not leave it to whoever has access.
- Approval belongs in the procedure. The NCSC's privileged access guidance ties credential requests to a support ticket and to a stated intent; the runbook is where that link is defined.
- Evidence is designed, not collected afterwards. Each runbook lists the records it produces, so audit and incident response can reconstruct who did what, when and from where.
- Tier runbooks by blast radius. Tier 0 operations on identity and control planes need stronger approval and recording than routine tasks.
- Write for machines as well as people. Runbooks that automation or AI agents will run need explicit identities, inputs, stop conditions and approval gates.
What a privileged operations runbook is
A privileged operations runbook is a documented procedure for a task that needs elevated rights on a production system: restarting a domain controller service, rotating a credential, changing firewall rules, restoring a database, granting emergency access. NIST SP 800-53 Rev. 5 SA-5 (System Documentation) covers administrator documentation. It does not tell you what fields make a procedure safe to run with privilege. The NCSC's secure system administration principles supply the questions: control who, where, when, why and how administration happens, grant least privilege, and log and audit administration activities. A runbook that answers those five questions for its task is a privileged operations runbook; one that only lists commands is a how-to note.
| Document | Answers | Typical owner |
|---|---|---|
| Policy | What must happen and who is accountable | Security or IT governance |
| Standard (this page) | What every runbook must contain | IT operations lead with security |
| Runbook | How to do one task safely, with which identity, approval and evidence | Service or platform owner |
| Playbook | How to respond to a situation, with decision points across several runbooks | Incident response or SOC |
The required header: identity, approval and evidence
Put these fields at the top of every privileged runbook. If a field does not apply, say so explicitly rather than leaving it blank.
| Field | What to record | Example |
|---|---|---|
| Runbook ID, version, owner | Unique ID, version, accountable owner and reviewer | OPS-AD-014 v3, owner: Directory Services lead |
| Risk tier | Tier from the table below | Tier 0 |
| Systems in scope | Services and assets from the service register | Production AD forest, all domain controllers |
| Performer role | Role allowed to run it, not a person's name | Tier 0 directory administrator |
| Execution identity | The account or role used, and whether it is personal, shared, a service account or a workload identity | Personal admin account, elevated just in time |
| How privilege is obtained | Just-in-time request, PAM checkout, role assumption or constrained endpoint | PAM checkout, 2-hour lease |
| Where it is run from | Permitted admin workstation, jump host or pipeline | Tier 0 privileged access workstation only |
| Approval | Change type, approver role, number of approvers, ticket requirement | Normal change, CAB approval plus a second Tier 0 administrator |
| Pre-conditions | Backups, maintenance window, health checks, communications | System state backup within 24 hours |
| Evidence produced | Logs, session recordings, transcripts, ticket updates, output files | PAM session recording, event IDs, ticket closure notes |
| Rollback | How to undo and the point of no return | Restore from system state backup; no rollback after schema change |
| Emergency path | What changes when the approval or PAM route is unavailable | Break-glass procedure BG-02, retrospective review within 1 working day |
The NCSC's guidance describes approval models from multi-party approval to rule-based auto-approval linked to a ticketing system, and says the level of approval is a risk decision. The header records which model applies to this task. See the Dual control and four-eyes approval for privileged and agent actions guide and the Access Request and Approval Workflow Design Guide.
Risk tiers for runbooks
| Tier | Examples | Approval | Execution and evidence |
|---|---|---|---|
| Tier 0: control planes | Directory, identity provider, PAM vault, PKI, backup platform, cloud organisation root, hypervisor management | Change ticket plus a second authorised approver | Just-in-time privilege, privileged access workstation, full session recording, post-task review |
| Tier 1: production services | Application servers, databases, network devices, Kubernetes clusters | Change ticket approved by service owner or CAB; standard changes pre-approved | Just-in-time or constrained role, command logging, ticket linked to logs |
| Tier 2: routine and reversible | Service restarts, log collection, read-only diagnostics | Pre-approved standard change or incident ticket | Constrained endpoint or automation, logged |
Align the tiers with your administration model. See the Tiered Administration and Enterprise Access Model Guide and the Privileged Access Workstations Guide.
Writing the steps
- One identity per step: If the identity changes mid-runbook, for example from a read-only check to an elevated change, mark the step where elevation happens and where it ends.
- Least privilege in the step itself: Prefer a constrained interface over full admin. Microsoft's Just Enough Administration (JEA) lets users run only specified cmdlets and commands through a session endpoint, under a virtual account or group managed service account, with transcripts of what each user ran. See the Windows Remote Management and WinRM/PowerShell Remoting Security Guide.
- Verification after each risky step: State the expected output and what to do if it differs. "Stop and escalate" is a valid instruction.
- No secrets in the document: Reference where a credential is held and how to check it out. Never paste passwords, keys or tokens into the runbook or its ticket. See the Secrets in chat and tickets.
- Point of no return: Mark the step after which rollback is no longer possible, and require a fresh confirmation before it.
- Closure: End with returning privilege (lease ended, session closed), updating the ticket and attaching evidence.
The evidence model
NCSC guidance says administration logs should answer who carried out an action, when and from where, what the action was and what it did, and that PAM records must reach the SOC to be acted on. Design the runbook so those answers exist without extra effort.
| Question | Evidence source | Linked by |
|---|---|---|
| Who requested and who approved? | Change or incident ticket; PAM or just-in-time request record | Ticket number entered in the access request |
| Which identity acted? | Directory, cloud audit or PAM checkout logs | Personal or leased account, never a shared login |
| From where? | PAM, jump host or workstation logs | Source host recorded in session metadata |
| What was done? | Session recording, command transcript, pipeline log | Session ID attached to ticket |
| What changed? | Configuration diff, system audit events | Diff or event export attached to ticket |
| Was it reviewed? | Post-task review record for Tier 0 | Reviewer sign-off on ticket |
See the Privileged Session Management Guide for recording and the Change Management Security guide for reconciling changes with tickets.
Runbooks run by automation and AI agents
Codified runbooks need the same fields, expressed in the tooling. AWS Systems Manager Automation, for example, provides an aws:approve step that pauses a runbook until named principals approve or reject it, with Approvers (up to 10), MinRequiredApprovals, an optional SNS notification and a default timeout of 7 days (maximum 30). Equivalent gates exist in other automation platforms; use the one your platform documents.
- Execution identity: Name the role the automation assumes, scoped to that runbook, owned by a person and reviewed like any other non-human identity. See the AWS Systems Manager Security Guide for one platform's detail.
- Separate the approver from the executor: The identity that starts the runbook should not be listed as its approver.
- Inputs and stop conditions: Validate parameters, cap scope (for example, a maximum number of hosts per run) and define the conditions that abort the run.
- AI agents: An agent following a runbook acts under a delegated identity. Keep Tier 0 steps behind human approval, log the agent's actions against the ticket and limit its tools to the runbook's steps. See the AI Agents in DevOps and SRE guide.
Keeping runbooks current
- Inventory: List existing runbooks, wikis and scripts for privileged tasks, and map each to a service and tier.
- Prioritise Tier 0: Rewrite Tier 0 runbooks to the standard first, then Tier 1 for services with regulatory or customer impact.
- Peer review: Require a second engineer and, for Tier 0, a security reviewer to approve each new version.
- Test: Run each Tier 0 and Tier 1 runbook in a non-production environment or tabletop at least annually, and after platform upgrades.
- Version control: Keep runbooks in a repository or controlled wiki with change history; the ticket should cite the version used.
- Retire: Archive runbooks for decommissioned systems so nobody runs an obsolete procedure with live privileges.
Small and large organisations: A small team can apply the header to its ten most dangerous tasks and use a shared approvers mailbox with a recorded reason, which the NCSC describes as a reasonable light-touch model. A large or regulated organisation should enforce the header with a template, link runbook IDs into the PAM and ITSM tools, and sample runbook evidence in internal audit. See the SOX IT General Controls (ITGC) Guide where SOX applies.
Detection and measurement
- Coverage: Percentage of Tier 0 and Tier 1 privileged tasks with a runbook that meets the standard.
- Linkage: Percentage of privileged sessions linked to a ticket and runbook ID.
- Standing privilege use: Privileged actions performed outside a just-in-time lease or constrained endpoint, which should trend to zero.
- Freshness: Runbooks past their review date.
- Test rate: Tier 0 runbooks exercised in the last 12 months.
- Exceptions: Break-glass uses with a completed retrospective review.
Detect privileged activity with no matching ticket or runbook, and alert, as the NCSC suggests, on rules such as Tier 0 administrator accounts being used to sign in.
Framework mapping
| Framework | Reference | Runbook field it supports |
|---|---|---|
| NIST SP 800-53 Rev. 5 | SA-5 System Documentation | Documented administrator procedures |
| NIST SP 800-53 Rev. 5 | AC-5 Separation of Duties, AC-6 Least Privilege | Approver separate from performer; constrained execution identity |
| NIST SP 800-53 Rev. 5 | CM-3 Configuration Change Control, CM-5 Access Restrictions for Change | Ticket and approval; who may change production |
| NIST SP 800-53 Rev. 5 | AU-12 Audit Record Generation, AU-6 Audit Record Review, Analysis, and Reporting | Evidence produced and reviewed |
| NIST CSF 2.0 | PR.AA-05, PR.PS-01 | Least privilege; configuration management practices |
Common failure patterns
- "Log in as admin": The runbook assumes a standing privileged account, so standing privilege survives every PAM project.
- Shared execution accounts: A team login or shared service account performs the task, so logs cannot show which person acted.
- Approval implied, not required: The runbook says "after approval" without naming the approver role or the ticket, so the step is skipped under pressure.
- Secrets in the procedure: Passwords or keys pasted into the wiki page or the ticket that links to it.
- Automation without an owner: A scheduled script runs a privileged runbook under a service account whose owner left years ago.
- Untested rollback: The rollback section was written once and never exercised, and fails during the outage it was meant for.
Practitioner checklist
- Adopt the header fields above as a mandatory template for privileged runbooks.
- Assign every runbook a tier, an owner and a review date.
- Name the execution identity and how privilege is obtained for every privileged step.
- Require a ticket reference for each run and record the runbook version used.
- Separate approver and performer, and add a second approver for Tier 0.
- List the evidence each runbook produces and make sure it reaches the SOC.
- Express approval gates and scoped roles in automation that runs runbooks.
- Test Tier 0 and Tier 1 runbooks yearly and after major changes.
- Report coverage, linkage and standing-privilege metrics quarterly.
Standards and references
- NCSC: Secure System Administration
- NCSC: Use Privileged Access Management
- NCSC: Log and Audit Administration Activities
- NIST: SP 800-53 Rev. 5
- NIST: Cybersecurity Framework 2.0
- Microsoft Learn: Just Enough Administration overview
- AWS: Systems Manager Automation aws:approve
Related NHI Mgmt Group resources: Break Glass Access Procedure Template · JIT Access Guide · IT Operations Security Maturity Model · Service Security Register Template