Most connected devices, from routers and cameras to gateways and industrial controllers, run embedded Linux, and many ship with debug features, shared root passwords and a writable root filesystem that an attacker can modify permanently. Embedded Linux is usually built with the Yocto Project and OpenEmbedded, which give manufacturers fine control over what goes into the image but leave hardening to them. This guide covers the build (image features, compiler flags, CVE checking and SBOMs), the boot chain (hardware root of trust, U-Boot verified boot, dm-verity), the runtime (read-only root filesystem, kernel hardening, mandatory access control, service minimisation), credentials, and signed A/B updates. It is written for embedded and firmware engineers, product security teams and buyers who need to know what to ask suppliers; the broader device lifecycle is in the Embedded Device Security Guide.
Key takeaways
- Development defaults are not production settings. A new Yocto project's default local.conf enables features that leave root with a blank password; remove them and set unique credentials before any release build.
- Verify everything that boots. Chain trust from a key in read-only memory through the bootloader to a signed kernel and a dm-verity protected root filesystem, using different keys at each stage.
- Make the root filesystem read-only. A read-only, integrity-checked rootfs means malware cannot persist in system files and every device runs a known image.
- Know what you ship. Yocto generates an SPDX SBOM by default and can check every recipe against the NVD with the cve-check class; use both on every release.
- Updates must be signed and recoverable. A/B slots with mandatory bundle signatures and a watchdog-backed fallback let you patch the fleet without bricking it.
Threats to embedded Linux devices
| Threat | Example | ATT&CK | Primary control |
|---|---|---|---|
| Default or shared credentials | Same root or web admin password across the product line | T1078.001 Default Accounts | Unique per-device credentials, no root login over the network |
| Persistent modification of the OS image | Implant written to the root filesystem or kernel | T1601 Modify System Image | Read-only rootfs with dm-verity, signed kernel |
| Boot chain tampering | Modified bootloader or firmware | T1542 Pre-OS Boot, T1542.001 System Firmware | Hardware secure boot, U-Boot verified boot |
| Malicious or vulnerable update | Unsigned update package or compromised build pipeline | T1195.002 Compromise Software Supply Chain | Mandatory update signatures, protected signing keys, reproducible builds |
| Firmware destruction | Wiping or corrupting flash to brick devices | T1495 Firmware Corruption | Redundant slots, recovery image, write protection |
| Known vulnerable components | Old BusyBox, OpenSSL or kernel | Not technique specific | CVE checking in the build, regular updates from upstream stable branches |
Botnets such as Mirai have shown at scale what default credentials on embedded Linux lead to; see the Mirai and IoT Botnet Families Guide.
Hardening the Yocto build
Image features to remove
The Yocto Project's guidance on making images more secure says that IMAGE_FEATURES must not include allow-empty-password, allow-root-login or empty-root-password in production. A new project's default local.conf enables them through EXTRA_IMAGE_FEATURES, and leaving that line in place gives root a blank password.
| Feature | What it does | Production setting |
|---|---|---|
| allow-empty-password | Lets Dropbear and OpenSSH accept accounts with an empty password | Remove |
| allow-root-login | Lets Dropbear and OpenSSH accept root logins | Remove |
| empty-root-password | Disables the mechanism that forces a non-empty root password | Remove |
| tools-debug, dbg-pkgs | Install strace, gdb and debug symbol packages | Remove from release images |
| read-only-rootfs | Builds a read-only root filesystem | Add |
| overlayfs-etc | Places /etc under overlayfs so device-specific data can live elsewhere | Add where /etc needs per-device changes |
- Separate build configurations: keep development and release configurations in different distro or image recipes, so debug features cannot leak into a release by a forgotten local.conf line. Check the final image in CI for an empty root password and for debug tools.
- Accounts: set distinct passwords for root and other users with the extrausers class, and avoid the same password across devices; Yocto suggests a unique random password per device set as a separate step after deployment.
- Compiler and linker hardening: add
require conf/distro/include/security_flags.incto the distro configuration to enable the project's security flags (stack protection, fortify source, position-independent executables and similar), noting that some recipes override them. - Layers: the meta-security layer provides hardening tools, and meta-selinux provides SELinux. The DISTRO_FEATURES
pamandseccompenable PAM and seccomp support in applications. See the SELinux Guide and the seccomp Guide. - Minimise: remove unneeded packages and close unneeded network services; every service removed is a vulnerability you will never have to patch.
CVE checking and SBOM
- cve-check: add
INHERIT += "cve-check"to check every recipe against the NVD during the build. Reports are written per recipe to build/tmp/deploy/cve and per image to tmp/deploy/images, and unpatched CVEs raise build warnings. Set NVDCVE_API_KEY to avoid slow, rate-limited NVD downloads. - Triage with CVE_STATUS: record why a CVE does not apply (for example an incorrect CPE) with CVE_STATUS rather than silently suppressing it, and fix product mapping with CVE_PRODUCT and CVE_VERSION.
- Fixing: prefer upstream stable or LTS updates; on stable branches backport patches named after the CVE with a
CVE:tag and Upstream-Status. - SBOM: the create-spdx class is inherited by default and produces an SPDX JSON SBOM per image (IMAGE-MACHINE.spdx.json in tmp/deploy/images/MACHINE). Archive it with each release, and use SPDX_INCLUDE_KERNEL_CONFIG where customers need to see kernel options. See the SBOM Guide.
Verified boot and dm-verity
U-Boot's verified boot documentation describes the pattern: a master key whose public part sits in read-only memory verifies the first-stage image, which carries a secondary public key for the next stage, and so on, with a different key at each stage to limit the impact of a compromise. FIT images contain hashes, and U-Boot verifies a signed FIT against a public key held in its control device tree. The public key must be protected from tampering, in read-only memory or by on-chip cryptography.
- Hardware root of trust: enable the SoC's secure boot so the boot ROM verifies the first-stage bootloader, and fuse the key hash. This step is usually irreversible: test fully on development boards before fusing production units.
- Signed bootloader and FIT: sign the kernel, device tree and initramfs as a FIT image, and configure U-Boot to require signatures, with the public key in its control FDT.
- dm-verity for the root filesystem: dm-verity is a read-only device-mapper target that verifies each block on access against a hash tree; if a block cannot be verified up to the root hash, the read fails. The root hash must itself be trusted: pass it in the signed kernel command line or initramfs, or have the kernel check a PKCS#7 signature of the root hash (root_hash_sig_key_desc, with DM_VERITY_VERIFY_ROOTHASH_SIG enabled) against the built-in trusted keyring.
- Choose the corruption policy:
restart_on_corruptionorpanic_on_corruptionfor production, neverignore_corruption. Optional forward error correction (fec_roots, 2 recommended, about 0.8% overhead) repairs flash bit errors without weakening verification. - Disable bootloader escape hatches: no interactive U-Boot console, no unauthenticated boot from USB or network, and kernel command line not editable at boot.
See the Firmware and Boot Security Guide and the Linux Secure Boot and Shim Guide for the PC and server equivalent.
Runtime hardening
Read-only root filesystem
Add IMAGE_FEATURES += "read-only-rootfs" to the image recipe. Applications must then write elsewhere (for example /tmp or /var/run, or a separate data partition) or fail gracefully, and every package post-install script must complete at build time, because the build fails if any script still needs to run on first boot. Put persistent device data on a separate, ideally encrypted, data partition mounted nodev,nosuid,noexec.
Kernel configuration
| Setting | Purpose (Kernel Self Protection Project) |
|---|---|
| CONFIG_STRICT_KERNEL_RWX=y | Keeps kernel memory permissions strict |
| CONFIG_STACKPROTECTOR_STRONG=y | Strong stack canaries |
| CONFIG_HARDENED_USERCOPY=y | Bounds checking on copies between kernel and user space |
| CONFIG_MODULE_SIG_FORCE=y | Only signed kernel modules load |
| CONFIG_SECURITY_LOCKDOWN_LSM=y | Limits root's ability to modify the running kernel |
| CONFIG_DEVMEM not set | Removes direct access to physical memory |
| CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y | Zeroes memory allocations |
| CONFIG_RANDOMIZE_BASE=y | Kernel address randomisation |
| kernel.kptr_restrict = 2, kernel.dmesg_restrict = 1 | Hide kernel addresses from unprivileged users |
| kernel.modules_disabled = 1 | Stops module loading after boot, where all drivers are built in or loaded early |
Check each option against the SoC vendor kernel, which may lag mainline. See the Linux Kernel Hardening and sysctl Guide.
Services and access control
- Run services as dedicated users with systemd sandboxing options or equivalent, and drop capabilities they do not need. See the Linux Capabilities Guide.
- Mandatory access control: SELinux or SMACK policies tuned to the device's few services; on a single-purpose device a tight policy is practical in a way it rarely is on a general server.
- Web interfaces: Yocto's guidance singles out web admin interfaces because they often run with elevated privileges; test them for input validation and cross-site scripting, and never ship one with a default shared credential.
- Debug interfaces: disable or lock serial consoles and JTAG on production units, and remove getty on UARTs.
Where identity fits
Every embedded device carries machine identities: SSH host keys, TLS client certificates for the cloud backend, API tokens and, too often, a hard-coded password shared across the product line. Generate keys and certificates per device at manufacture or first boot, never in the image; store private keys in a TPM, secure element or the SoC's key storage where available; and give each device a unique identity the backend can revoke. Signing keys for bootloaders, FIT images, dm-verity root hashes and update bundles are the most valuable secrets the manufacturer holds: keep them in an HSM, separate from developer machines and CI runners. See the IoT Device Identity with TPM and DICE Guide and the Identity for IoT Fleets Guide.
Secure updates
Yocto's guidance is plain: keep the device updatable so future vulnerabilities can be patched. Open source update frameworks such as RAUC, SWUpdate and Mender implement the common pattern; RAUC's documentation is used here as an example.
- Redundant slots: install to the inactive slot of an A/B (or A/B/C) pair while the device keeps running from the active one.
- Mandatory signatures: RAUC makes bundle signing mandatory; use a self-signed certificate only in development and integrate production signing with your PKI.
- Bootloader handshake: the update client marks the new slot good or bad, and the bootloader falls back to the previous slot or a recovery system. A hardware watchdog should treat resets as failed boots.
- Anti-rollback: prevent installation of older, vulnerable signed images, using a version counter in secure storage where the hardware supports it.
- Integrity in transit and at rest: verity-format bundles allow verified streaming over HTTPS without storing the whole bundle first.
Implementation roadmap
- Baseline the current image: list image features, accounts, open ports, installed packages and kernel config; run cve-check and generate the SBOM.
- Fix credentials and debug features: remove the empty password features and debug tools, set per-device credentials and disable root network login. Test on development boards first.
- Enable build hardening: security_flags.inc, minimised package set and CI checks that fail a release build containing debug features.
- Introduce read-only rootfs: move writable state to a data partition, fix applications that write to system paths, and enable read-only-rootfs.
- Add verified boot and dm-verity: sign FIT images, enable dm-verity with a signed or kernel-embedded root hash, test failure behaviour, then fuse secure boot keys on a pilot batch.
- Deploy signed A/B updates: with fallback tested by deliberately installing a failing image in the lab.
- Operate: monthly CVE review, SBOM published with each release, and a vulnerability disclosure route. See the EU Cyber Resilience Act Guide for obligations that apply to products sold in the EU.
- Small teams: start with steps 1 to 3 and signed updates; a device that can be patched safely is the biggest single gain.
- Large product lines: a common hardened distro layer shared across products, an HSM-backed signing service, and fleet metrics per product.
Detection and measurement
- What to monitor: boot verification failures and dm-verity corruption events reported to the backend, failed and successful update installs and rollbacks, SSH or console logins on fielded devices, and unexpected outbound connections.
- Unpatched CVEs per release: count of unpatched, applicable CVEs in the cve-check report at release time.
- Verified boot coverage: percentage of shipped units with secure boot fused and dm-verity enforced.
- Update currency: percentage of the fleet on a supported release and median days from fix to fleet deployment.
- Debug leakage: release builds that failed CI checks for debug features or empty passwords (should trend to zero).
- Integrity events: dm-verity or boot verification failures per 10,000 devices, investigated individually.
Framework mapping
| Control | NIST SP 800-53 Rev. 5 | NIST CSF 2.0 | Other |
|---|---|---|---|
| Verified boot, dm-verity | SI-7 Software, Firmware, and Information Integrity | PR.PS-01 | ETSI EN 303 645 provision 5.7 (software integrity) |
| Signed updates and components | CM-14 Signed Components, SI-2 | PR.PS-02 | NIST IR 8259A software update |
| Minimised image and services | CM-7 Least Functionality | PR.PS-01 | CIS Controls v8.1 Control 4 |
| Per-device credentials | IA-5 Authenticator Management | PR.AA-01 | ETSI EN 303 645 provision 5.1 |
| Kernel memory protections | SI-16 Memory Protection | PR.PS-01 | Kernel Self Protection Project |
| SBOM and CVE checking | SR-4 Provenance, RA-5 | ID.RA-01 | EU Cyber Resilience Act |
| Build pipeline and signing keys | SA-10 Developer Configuration Management, SC-12 | PR.DS-01 | NIST SSDF |
Common failure patterns
- Shipping the development configuration: EXTRA_IMAGE_FEATURES left in local.conf, so release images accept root with an empty password.
- Verified boot that stops at the kernel: a signed kernel mounting a writable, unverified root filesystem, so an attacker persists in user space.
- Root hash not protected: dm-verity enabled but the root hash passed on an editable bootloader command line.
- Signing keys on a build server: FIT and update signing keys in CI variables or on a developer laptop, so one compromise signs malware for the whole fleet. The Toyota T-Connect key exposure shows how long an exposed credential can sit unnoticed in source code.
- Ignoring cve-check output: warnings accepted as noise, with no CVE_STATUS triage, so applicable vulnerabilities ship.
- Updates without fallback: single-slot updates that brick devices on power loss, which makes teams reluctant to patch at all.
Practitioner checklist
- Remove allow-empty-password, allow-root-login, empty-root-password and debug tools from release images, enforced in CI.
- Set unique per-device credentials and disable root network login.
- Enable security_flags.inc and minimise packages and services.
- Run cve-check on every build, triage with CVE_STATUS and archive the SPDX SBOM per release.
- Build a read-only rootfs with writable state on a separate data partition.
- Enable hardware secure boot, signed FIT images and dm-verity with a trusted root hash and restart or panic on corruption.
- Apply KSPP kernel options appropriate to the platform.
- Generate device keys per unit and protect signing keys in an HSM.
- Ship signed A/B updates with watchdog-backed fallback and anti-rollback.
Standards and references
- Yocto Project: Making Images More Secure
- Yocto Project: Checking for Vulnerabilities
- Yocto Project: Creating a Software Bill of Materials
- Yocto Project: Creating a Read-Only Root Filesystem
- Linux Kernel Documentation: dm-verity
- U-Boot: Verified Boot
- Kernel Self Protection Project: Recommended Settings
- RAUC: Basic Concepts
Related NHI Mgmt Group resources: Embedded Device Security Guide · Firmware and Boot Security Guide · Linux Kernel Hardening and sysctl Guide · SBOM Guide · IoT Device Identity with TPM and DICE Guide