Join our Newsletter — 33% off our NHI Course
Home› Guides› Malware Analysis Guide for Defenders: Triage, Static, Dynamic…
Guide Incident Response & Forensics

Malware Analysis Guide for Defenders: Triage, Static, Dynamic and IOC Reporting

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 30 October 2026 12 min read
On this page

When a suspicious file or memory region turns up during an incident, the response stalls until someone can say what it does, how it persists, what it talks to and what it stole. Malware analysis for defenders is the disciplined process that answers those questions fast enough to drive containment, using triage, static analysis, controlled dynamic analysis and, only where needed, reverse engineering. This guide sets out a tiered analysis method, a safe lab and handling model, how samples try to defeat analysis, and how to turn findings into indicators, detections and hardening. It is written for SOC analysts, incident responders and detection engineers who analyse samples as part of response rather than as full-time reverse engineers. For deep code analysis, see the Ghidra for Malware Analysis Guide; this page covers the overall method.

Key takeaways

  • Analyse to answer response questions. Start from what the incident needs (scope, persistence, C2, data at risk, credentials exposed) and stop when those are answered with enough confidence.
  • Work in tiers. Triage and reputation in minutes, static properties in under an hour, sandbox and controlled dynamic analysis next, and reverse engineering only for the questions the earlier tiers cannot answer.
  • Isolation and handling come first. A dedicated, isolated lab, hashed and password-protected sample storage and rules on what may be uploaded to public services protect both the organisation and the investigation.
  • Assume the sample resists analysis. Packing, obfuscation and sandbox checks are normal; a clean sandbox report is not a verdict.
  • Output is detection and action, not a report alone. Every analysis should produce behavioural detections, YARA or Sigma rules, prioritised indicators and a credential rotation list.

How the attack works

For the analyst, the "attack" is the sample's attempt to run, persist and hide while you study it. Understanding the common lifecycle tells you what to look for at each tier.

StageWhat malware typically doesWhat the analyst looks forRelated ATT&CK technique
Delivery and executionArrives as a document, archive, script, installer or loader that the user or a service runsFile type versus extension, parent process, embedded scripts or macrosT1204.002 Malicious File
StagingDownloads further stages or toolsEmbedded URLs, download functions, second-stage hashesT1105 Ingress Tool Transfer
ConcealmentPacks, encrypts or encodes code and strings; loads code reflectively or injects into other processesHigh entropy sections, few imports, unbacked executable memoryT1027 Obfuscated Files or Information, T1620 Reflective Code Loading, T1055 Process Injection
Analysis evasionChecks for virtual machines, debuggers, sandbox artefacts, user activity or delays executionEnvironment checks in strings or behaviour; activity that only starts after long sleepsT1497 Virtualization/Sandbox Evasion
PersistenceCreates services, scheduled tasks or Run keysRegistry and service changes during detonationT1543.003 Windows Service, T1053.005 Scheduled Task, T1547.001 Registry Run Keys
Credential accessReads browser stores, LSASS memory or token cachesAccess to credential files and processesT1003.001 LSASS Memory
Command and controlBeacons over HTTPS or other protocols, often to cloud or compromised infrastructureDomains, IP addresses, URI patterns, certificate and timing characteristicsT1071.001 Web Protocols

Families and delivery chains are covered in the Malware Guide, the Malware Loaders and Droppers Guide and the Fileless Malware Guide.

Where it shows up

Samples reach analysts from several sources, and each changes what can be learned.

  • EDR quarantine and alerts: the file plus process tree and command line; the richest starting point.
  • Email security and user reports: attachments and links, often before execution. See the Phishing Report Button and User Reported Message Triage Guide.
  • Disk and triage collections: files found by timeline or artefact review, with execution evidence from prefetch, Amcache and event logs.
  • Memory images: injected code and unpacked payloads that never touch disk; see the Volatility 3 Guide.
  • Network captures and proxies: downloaded payloads and C2 traffic.
  • Software supply chain: malicious packages and build artefacts, where the "sample" is a dependency.

The tiered method

TierActivitiesTypical timeOutput
1. TriageHash (SHA-256), file type identification, reputation lookup by hash in internal and approved threat intelligence sources, check whether it is signed and by whomMinutesKnown or unknown; initial severity
2. Static propertiesStrings, imports, sections and entropy, embedded resources, compile and signature metadata, YARA matches; for documents and scripts, extract and deobfuscate embedded codeUnder an hourCapabilities hypothesis, candidate indicators
3. Automated dynamicDetonation in a sandbox with realistic configuration: process, file, registry and network activity, dropped files, memory dumpsMinutes to an hourBehaviour report, network indicators, dropped payloads
4. Controlled manual dynamicRun in an instrumented, isolated VM with monitoring tools and a simulated network; observe persistence and C2 handlingHoursConfirmed behaviours, configuration and protocol details
5. Reverse engineeringDisassembly and decompilation to recover configuration, encryption, kill switches or capabilities not seen in executionHours to daysConfig extractors, high-fidelity rules, capability list

Most incident samples are resolved at tiers 1 to 3. Escalate to tiers 4 and 5 when the sample is novel, targeted, evasive, or when the response depends on details such as a decryption routine or the full list of C2 servers. Sandbox selection and configuration are covered in the Malware Sandbox Guide.

Impact

The quality and speed of analysis directly change the cost of the incident.

  • Scope: indicators and behaviours from analysis decide which other hosts are hunted. Missing a second C2 domain or persistence method leaves footholds after "eradication".
  • Credentials: knowing that a sample is an infostealer, or dumped LSASS, turns a host clean-up into an identity incident covering every account, token and API key used on the machine. See the Infostealer Threat Guide.
  • Data: exfiltration capability and destinations inform notification decisions.
  • Exposure from poor handling: uploading a targeted sample or an internal document to a public service can reveal the investigation to the attacker and disclose confidential data embedded in the file.

Where identity fits

Treat every confirmed credential-access capability as an identity event: list human and non-human identities whose secrets were present on the host (service accounts, cached domain credentials, cloud CLI tokens, browser sessions, API keys in configuration files), then rotate and review their recent use. See the Credential Dumping Guide and the Secrets Management Guide.

Detection

Analysis exists to improve detection. Convert findings into durable detections in order of resilience: behaviours first, tooling and code patterns second, infrastructure last.

FindingDetection artefactDurability
Process behaviour (parent and child chain, injection, persistence method)EDR or Sigma rule on the behaviour; see the Sigma Rules GuideHigh: survives recompilation and infrastructure changes
Unique code, strings or structureYARA rule for files and memory; see the YARA Rules GuideMedium to high, depending on what the rule anchors on
Network protocol traits (URI pattern, header order, beacon timing)Network or proxy detectionMedium
Domains, IP addresses, file hashesBlocklists and IOC sweeps; see the IOC Scanners GuideLow: change quickly, but useful for immediate scoping

Reporting indicators

  • Context with every indicator: type, role (payload, C2, staging), first and last seen, confidence and the analysis tier that produced it.
  • Machine-readable sharing: express indicators and behaviours in STIX 2.1 for platforms and partners. See the STIX and TAXII Guide and the Indicators of Compromise vs Indicators of Attack Guide.
  • ATT&CK mapping: map observed behaviours to technique IDs so coverage gaps are visible to detection engineering.
  • Audience split: a one-paragraph verdict and actions for incident command, and a technical annex for the SOC and threat intelligence. See the Threat Intelligence Report Template.

Measuring the capability

  • Time to verdict: minutes from sample receipt to a triage verdict and severity (target under 30 minutes for tier 1 and 2).
  • Escalation rate: share of samples needing tier 4 or 5; a high rate suggests weak sandbox configuration.
  • Detection yield: behavioural detections or YARA rules created per analysed family.
  • Retro-hunt hit rate: share of new rules that find additional affected hosts.
  • Handling exceptions: count of samples or documents shared outside approved channels (target zero).

Prevention and hardening

Building a safe lab

  1. Isolate: analysis VMs on a separate network segment or host with no route to production, no shared folders or clipboard by default, and snapshots to revert after every run.
  2. Control the network: default to a simulated internet; allow real egress only through a controlled, attributable connection when the investigation needs live C2 behaviour, with approval.
  3. Use dedicated identities: no corporate accounts, tokens or VPN profiles in the lab; lab credentials should be unique and monitored.
  4. Store samples safely: keep samples in password-protected archives named by SHA-256, in a restricted repository, with access logged.
  5. Define sharing rules: state which samples may be submitted to public sandboxes and reputation services (for example commodity malware) and which may not (targeted samples, files containing internal data); hash lookups are usually safe where uploads are not.
  6. Test and rehearse: run benign test files through the full workflow quarterly and confirm tooling, snapshots and isolation.

Feeding findings back

  • Endpoint: add behaviour blocks, attack surface reduction rules or application control for the observed execution chain. See the CIS Control 10 Malware Defenses Guide.
  • Email and web: block the delivery file types and URL patterns seen in the chain.
  • Identity: rotate exposed credentials and reduce standing privileges on hosts where credential theft was possible.
  • Response: update the Malware Outbreak Playbook with containment steps specific to the family.

Framework mapping

FrameworkReferenceHow malware analysis supports it
NIST SP 800-53 Rev. 5SI-3 Malicious Code Protection; IR-4 Incident Handling; SI-4 System MonitoringAnalysis improves malicious code protection and drives incident handling decisions
NIST CSF 2.0DE.AE-02, RS.AN-03Adverse events are analysed; root cause is established
MITRE D3FENDD3-DA Dynamic Analysis, D3-FA File Analysis, D3-EFA Emulated File Analysis, D3-FHRA File Hash Reputation AnalysisDefensive techniques that correspond to the analysis tiers
NIST SP 800-83 Rev. 1Malware incident prevention and handlingOverall prevention and response process for malware incidents

Binary formats and packers

Static triage starts with the file format, because the headers tell an analyst what platform the sample targets, where code starts and whether the file looks normal. This section covers recognition only; unpacking a live sample belongs in an isolated lab.

  • PE (Windows): Microsoft's PE format specification describes an MS-DOS stub at the front of the image, with the file offset of the "PE\0\0" signature at 0x3c. A COFF header follows (machine type, number of sections, characteristics), then an optional header that is required for images, whose magic value distinguishes PE32 (0x10b) from PE32+ (0x20b) and which holds the entry point and data directories such as the import table. A section table of 40-byte entries follows.
  • ELF (Linux and Unix): the file starts with 0x7f followed by "ELF". The elf(5) manual page describes two views: the section header table, which locates sections such as .text, .data and the symbol tables used in linking, and the program header table, whose entries describe segments the system needs to prepare the program for execution.
  • Mach-O (macOS): Apple's loader.h defines the 64-bit magic 0xfeedfacf. Load commands directly follow the header, and each LC_SEGMENT_64 command describes a segment such as __TEXT, __DATA or __LINKEDIT, followed by its sections, for example __text.

Sections versus segments: sections are the linker's view of a file; segments are the loader's view of memory. In ELF and Mach-O, segments contain one or more sections. In PE, each section header records both a size on disk (SizeOfRawData) and a size in memory (VirtualSize), and the specification says the remainder is zero-filled when the virtual size is larger.

Packers: UPX is a free executable compressor for Windows, macOS and Linux programs that can also list, test and unpack the files it produced. MITRE ATT&CK tracks packing as T1027.002 Software Packing, naming UPX and MPRESS and noting custom packers that leave fewer known artefacts. Analysts suspect packing when they see:

  • very few imports, often little beyond loader functions;
  • a section whose virtual size is far larger than its raw size, or raw size near zero;
  • high entropy in code or data sections and few readable strings;
  • non-standard or packer-named sections, and an entry point outside the normal code section;
  • at runtime, memory allocation followed by execution from non-image memory, which ATT&CK lists as a behavioural detection.

For tooling, see the Ghidra for Malware Analysis Guide.

Practitioner checklist

  • Write down the response questions before analysing any sample.
  • Apply the five-tier method and record which tier produced each finding.
  • Maintain an isolated lab with snapshots, simulated networking and no corporate credentials.
  • Store samples by SHA-256 in password-protected archives with logged access.
  • Publish clear rules on what may be submitted to public services.
  • Produce at least one behavioural detection or YARA rule per new family and retro-hunt with it.
  • Share indicators with context and confidence, in STIX where possible, mapped to ATT&CK.
  • Turn any credential-access capability into an identity rotation task.
  • Track time to verdict, escalation rate and detection yield.

Standards and references

Related NHI Mgmt Group resources: Ghidra for Malware Analysis Guide · Malware Sandbox Guide · YARA Rules Guide · Malware Analyst Career Guide · Volatility 3 Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 30 October 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.