A file's metadata tells every parser how much to read, where to jump and what the data should hash to, so a file whose metadata lies about its own content is a file built to mislead a parser. MITRE D3FEND defines File Metadata Consistency Validation (D3-FMCV) as "The process of validating the consistency between a file's metadata and its actual content, ensuring that elements like declared lengths, pointers, and checksums accurately describe the file's content." Where magic byte and value checks ask whether fields hold allowed values, this technique asks whether those fields tell the truth about the bytes they describe. This card covers what D3FEND says, consistency checks from real specifications, where they run, how to deploy and test them, and their limits. It is for engineers who run upload pipelines, archive handling, mail gateways and import jobs. The family overview is D3-CV Content Validation.
Key takeaways
- It cross-checks the file against itself. Declared lengths, offsets and checksums are compared with the content they describe, not with an external list.
- Recalculate, do not trust. D3FEND's own example: if a header declares a 50-byte block, verify it, and recalculate CRC values and compare them.
- Many formats carry two copies of the truth. ZIP stores CRC-32 and sizes in both the local file header and the central directory; disagreement is a red flag.
- Checksums cost time. The van den Bos thesis notes that checksum checks take more time than other validation, so place them after cheap rejects.
- Consistency is not safety. An attacker who writes a correct file with hostile content passes; follow with content analysis and scanning.
D3-FMCV at a glance
| Attribute | Value |
|---|---|
| D3FEND ID | D3-FMCV (d3f:FileMetadataConsistencyValidation) |
| Definition | "The process of validating the consistency between a file's metadata and its actual content, ensuring that elements like declared lengths, pointers, and checksums accurately describe the file's content." |
| Tactic | Isolate (through Content Filtering) |
| Parent technique | File Format Verification (D3-FFV), under Content Validation (D3-CV) |
| Digital artefacts | Analyzes File Content Block Data ("The actual content or main data within a file or data block") and File Metadata ("Information that describes and provides context about a file's content, structure, and attributes"); inherits filters File and enforces Content Policy |
| D3FEND reference | van den Bos, "Gathering Evidence: Model-Driven Software Engineering in Automated Digital Forensics" |
Hierarchy and artefacts are taken from the D3FEND 1.6.0 ontology, checked November 2026. D3-FMCV is the only sibling under D3-FFV that analyses two artefacts at once, metadata and content data, which is what makes it a consistency check rather than a value check. Value checks are in D3-FMVV File Metadata Value Verification.
How it works
D3FEND describes the technique as checking elements such as declared lengths, pointers and checksums so that they accurately describe the content: a header that specifies a 50-byte content block should be verified, and CRC values should be recalculated and compared. The three families of check work differently.
| Check | What is compared | What a mismatch suggests |
|---|---|---|
| Length | A declared size field against the bytes actually present before the next structure or end of file | Truncation, appended or hidden data, or a crafted size meant to make a parser over-read |
| Pointer and offset | An offset or index against the file's real size and structure | A jump outside the file or into the wrong structure, a classic route to parser memory errors |
| Checksum | A stored CRC or hash against one recalculated from the data | Corruption, or content altered after the checksum was written |
| Duplicated metadata | Two copies of the same fact held in different places in the file | A file built so that two parsers see different content |
The "what a mismatch suggests" column is NHI Mgmt Group's reading; D3FEND does not attach meanings to failures.
Consistency rules from format specifications
| Format | Consistency rule a validator can enforce | Source |
|---|---|---|
| PNG | Each chunk has Length, Chunk Type, Chunk Data and CRC; the CRC is calculated over the chunk type and chunk data (not the length), and the length must not exceed 231-1 bytes. Recalculate every CRC and check each length lands on the next chunk boundary | W3C PNG Third Edition, section 5.3 |
| ZIP | CRC-32, compressed size and uncompressed size appear in both the local file header and the central directory header; when general purpose bit 3 is set, the local values are zero and the true values follow the data in a data descriptor. Check that the copies agree and that the CRC matches the decompressed data | PKWARE APPNOTE 6.3.10, sections 4.3.7, 4.3.9, 4.3.12 |
| Compound File | Sector Shift must match Major Version (0x0009 for version 3, 0x000C for version 4); sector pointers must stay within the file's sector count | Microsoft [MS-CFB]; Garfinkel's MSOLE validator rejects negative or out-of-range pointers |
| GIF | Length-prefixed data blocks let a validator detect errors at fine granularity | van den Bos thesis, chapter 2 |
ATT&CK techniques D3FEND relates to D3-FMCV
Relationship type: inferred only. D3FEND has no direct ATT&CK mapping for D3-FMCV. Its technique page lists techniques it infers through a shared artefact, chiefly File: the ATT&CK technique involves a File and D3-FMCV, through its Content Filtering ancestor, filters File. D3FEND labels these "inferred, experimental" (D3FEND 1.6.0, checked November 2026), and the same list appears for every sibling under Content Validation. Each ID below was checked against ATT&CK v20.
| ATT&CK tactic (v20) | Techniques D3FEND relates through the File artefact |
|---|---|
| Initial Access | T1566.001 Spearphishing Attachment, T1566.002 Spearphishing Link, T1566.003 Spearphishing via Service |
| Execution | T1204.002 Malicious File, T1127.001 MSBuild |
| Persistence and Privilege Escalation | T1037.001 to T1037.004 logon, network logon and RC scripts and Login Hook; T1546.002, .004, .005, .006, .008, .009, .010, .013, .014, .015 event triggered execution; T1547.001, .006, .007, .008, .009 autostart execution; T1543.001, .002, .004 Launch Agent, Systemd Service, Launch Daemon; T1137.001 Office Template Macros, T1137.003 Outlook Forms; T1505.003 Web Shell; T1548.002 Bypass User Account Control, T1548.003 Sudo and Sudo Caching |
| Stealth | T1055.001, .002, .003, .009, .014 process injection; T1218.005 Mshta, T1218.011 Rundll32; T1574.001, .004, .006, .007, .008, .009, .012 hijack execution flow; T1036.001, .003, .005, .006 masquerading; T1027.001 Binary Padding, T1027.002 Software Packing, T1027.004 Compile After Delivery; T1564.002, .003, .006, .007 hide artefacts; T1070.004 File Deletion |
| Defense Impairment | T1556.002 Password Filter DLL, T1556.003 Pluggable Authentication Modules; T1647 Plist File Modification; T1685.006 Clear Linux or Mac System Logs; T1690 Prevent Command History Logging |
| Credential Access | T1555.003 Credentials from Web Browsers, T1003.007 Proc Filesystem, T1003.008 /etc/passwd and /etc/shadow, T1552.001 Credentials In Files, T1552.003 Shell History |
| Collection | T1074.001 Local Data Staging, T1114.001 Local Email Collection, T1560.001 to T1560.003 Archive Collected Data |
| Command and Control, Exfiltration | T1071.001 Web Protocols, T1573.002 Asymmetric Cryptography, T1048.002 Exfiltration Over Asymmetric Encrypted Non-C2 Protocol |
| Impact | T1565.001 Stored Data Manipulation, T1565.003 Runtime Data Manipulation |
D3FEND still lists some IDs that ATT&CK v20 has revoked; the table shows the successors: Plist Modification is now T1647 Plist File Modification, DLL Side-Loading is folded into T1574.001 DLL, Clear Linux or Mac System Logs is now T1685.006, and Impair Command History Logging is now T1690 Prevent Command History Logging. D3FEND also lists Launchd, which ATT&CK v20 marks deprecated with no successor, so it is left out, as are entries the page shows by name without an ID.
Where the check genuinely bites (NHI Mgmt Group view): crafted documents and archives delivered by attachment or download (T1566.001, T1204.002) that rely on lying length or offset fields to exploit a reader; Binary Padding (T1027.001), where data is added to change a file's size or hash, which shows up when declared and actual lengths disagree in formats that record them; and Embedded Payloads (T1027.009), where content hides in regions the metadata does not account for. See the T1027 Obfuscated Files or Information Defensive Guide.
Where it is implemented
The examples are listed alphabetically and are illustrations, not recommendations.
| Implementation | Consistency checks it performs | Point to note |
|---|---|---|
| Archive extraction and testing tools | Test modes that decompress each member and compare the CRC with the stored value, as Garfinkel's ZIP validator does with unzip -t | Apply size limits after decompression, as the OWASP File Upload Cheat Sheet advises |
| Content disarm and reconstruction (CDR) products | Rebuild the file from parsed content, so inconsistent metadata is either rejected or replaced by freshly written values | See D3-CM Content Modification |
| Forensic carving tools | Validators that use length fields and checksums to decide where a file starts and ends | See D3-FC File Carving |
| Image and document decoders in upload pipelines | Strict decoding that fails on CRC or length errors | Configure strict mode; many libraries can repair or ignore errors |
Decompression bounds are the neighbouring control: declared uncompressed sizes are metadata too, and checking them against the real output is where this technique meets D3-FCDC File Content Decompression Checking.
Deploying it
- Start with archives and images. ZIP-based formats and PNG carry explicit sizes and CRCs, so consistency checks are well defined and cheap to add.
- Order checks by cost. Signature and value checks first, then length and pointer checks, then full checksum recalculation, so most bad files fail early.
- Compare duplicated fields. For ZIP, compare local header, data descriptor and central directory values for every member; reject on any disagreement.
- Never trust a declared size for allocation. Cap reads and memory at your policy limit regardless of what the header says.
- Isolate the validator. It reads hostile metadata by design: run it with least privilege and time and memory limits.
- Log before enforcing. Run log-only for a few weeks (an example period, not a standard) to find partners whose tools write slightly inconsistent files.
- Enforce with quarantine. Failures go to D3-CQ Content Quarantine with the failing field, declared value and observed value recorded.
- Roll back per rule. Relax one rule for one source and fix the source, rather than turning consistency checks off.
How to verify it works
- CRC test: flip one bit inside a chunk of a harmless test PNG without updating its CRC; the pipeline should reject it.
- Length test: edit a chunk length in a test PNG so it points past the next chunk; confirm rejection without a crash or hang.
- Two-copy test: in a test ZIP, make the central directory size for one member differ from its local header; confirm rejection.
- Appended data test: append bytes after the end of a valid file; confirm your policy either rejects or strips them, and that the decision is logged.
- Resource test: declare a huge size in a small file and confirm the validator does not allocate memory to match the claim.
| Metric | Example target (NHI Mgmt Group suggestion, not a standard) | Why it matters |
|---|---|---|
| Allowed formats with length and checksum checks enabled | All formats that define them | Unchecked formats are the easy route |
| Inconsistency test cases rejected per release | 100% | Proves the checks run after changes |
| Consistency failures per week by source | Baselined; spikes reviewed | Clusters show a campaign or a broken partner tool |
| Validator crashes, hangs or memory limit hits | Zero; each investigated | These suggest crafted metadata reaching a parser bug |
| Median added latency per file | Within the pipeline's service target | Checksum work is the costly part and should not drive users to bypasses |
Limits, bypasses and failure patterns
- Self-consistent forgeries: CRCs are not signatures. Anyone who changes content can recalculate them, so consistency proves the file was written carefully, not honestly. Use D3-FH File Hashing against known values or digital signatures where provenance matters.
- Checking one copy only: validating the central directory while an extractor reads local headers, or the reverse, lets two tools see different files.
- Lenient decoders: libraries that ignore CRC errors or repair lengths by default turn the check off silently.
- Appended data policy gaps: Garfinkel notes that some formats tolerate appended data; decide explicitly whether your pipeline accepts it.
- Trusting declared sizes for resources: allocating memory from a header value is how small files cause large outages.
- Formats without integrity fields: where a format has no checksum, only length and pointer checks apply; do not report it as fully validated.
Framework mapping
| Framework | Reference | Relevance |
|---|---|---|
| NIST SP 800-53 Rev 5 | SI-10 Information Input Validation | Checking inputs are valid and well-formed |
| NIST SP 800-53 Rev 5 | SI-3 Malicious Code Protection | Rejecting crafted files before use |
| NIST SP 800-53 Rev 5 | SC-7 Boundary Protection | Validation at mail, web and import boundaries |
| MITRE ATT&CK mitigations | M1049 Antivirus/Antimalware | Closest mitigation (NHI Mgmt Group view); see M1049 Antivirus/Antimalware |
| MITRE D3FEND | D3-CV, D3-FFV, D3-FMCV | Ancestors and technique |
Practitioner checklist
- List the accepted formats that carry lengths, offsets and checksums.
- Recalculate checksums and compare; never accept a stored CRC on trust.
- Compare every duplicated metadata field, such as ZIP local and central records.
- Cap memory and reads at policy limits, not at declared sizes.
- Run strict decoders in isolation with resource limits.
- Decide and document the policy on appended data.
- Quarantine failures with declared and observed values logged.
- Keep bit-flip, length, two-copy and appended-data test files in release tests.
Standards and references
- MITRE D3FEND: File Metadata Consistency Validation (D3-FMCV)
- W3C: Portable Network Graphics (PNG) Specification, Third Edition
- PKWARE: APPNOTE.TXT, .ZIP File Format Specification
- Microsoft: [MS-CFB] Compound File Header
- van den Bos: Gathering Evidence, Model-Driven Software Engineering in Automated Digital Forensics
- Garfinkel: Carving Contiguous and Fragmented Files with Fast Object Validation
- OWASP: File Upload Cheat Sheet
Related NHI Mgmt Group resources: D3-FFV File Format Verification, D3-FISV File Internal Structure Verification, D3-FCDC File Content Decompression Checking, File Upload Security Guide.