Linux Fundamentals for Defenders: Users, Sudo, Services

Linux runs most servers, containers and cloud workloads, yet many security teams treat it as a black box and miss the ordinary mechanisms, such as sudo rules, setuid binaries, capabilities, SSH keys, systemd units and cron jobs, that attackers use to gain and keep access. This guide explains how Linux decides who a process is and what it may do, how authentication is assembled through PAM, where services and scheduled jobs are defined, how namespaces and cgroups underpin containers, and how package integrity is checked. It starts in plain language and then goes to practitioner depth with real file paths, directives and permission bits. It is written for SOC analysts, detection engineers and platform engineers who need a working model of Linux before they harden or monitor it.

Key takeaways

UID 0 is the security boundary that matters: the kernel bypasses its normal permission checks for a process with effective UID 0, so every path to root (sudo, setuid, capabilities, writable root-owned files) needs to be known and minimised.
Capabilities split root into pieces, but some pieces, especially CAP_SYS_ADMIN, are close to full root, so a "non-root" container or service with the wrong capability is not low privilege.
SSH keys and service users are non-human identities: an entry in authorized_keys is a standing credential that never expires unless you give it an expiry or replace keys with certificates.
systemd and cron define what runs, so unit files, timers and crontabs are both the main persistence locations and the place to apply sandboxing such as NoNewPrivileges and ProtectSystem.
Containers are processes with namespaces and cgroups, not virtual machines; a container running as root with broad capabilities shares the host kernel and can become a host compromise.

Users, groups and UID 0

Linux identifies users and groups by number. Accounts are defined in /etc/passwd (name, UID, primary GID, home directory, shell), password hashes in /etc/shadow (readable only by root) and groups in /etc/group. Directory-backed systems add accounts through SSSD or LDAP, but the kernel only ever sees numbers.

A process carries several identities, which the credentials(7) man page describes: the real UID/GID (who owns the process), the effective UID/GID (used for most permission checks), the saved set-user-ID (lets a setuid program drop and regain privilege) and the Linux-specific filesystem UID/GID used for file access, plus supplementary groups. When defenders see a process whose real UID is a normal user but whose effective UID is 0, a setuid program or sudo is in play.

UID 0: root. Any account with UID 0 in /etc/passwd, whatever its name, is root. A second UID 0 account is a classic backdoor and is easy to audit for.
System accounts: low UIDs (commonly below 1000) are reserved for services, which should have a non-login shell such as /usr/sbin/nologin and a locked password.
Privileged groups: membership of groups such as sudo or wheel, docker, disk or adm confers power. Membership of the docker group is effectively root on that host because the Docker daemon runs as root and will start containers with host mounts on request.
Local accounts as NHIs: service users, deployment users and automation accounts often outnumber humans on a server and are rarely reviewed. See the Service Account Security Guide.

File permissions, setuid and the sticky bit

Every file has an owner, a group and twelve mode bits. The inode(7) man page lists them: read, write and execute for owner (0700), group (0070) and others (0007), plus three special bits.

Bit	Octal	On a file	On a directory	Defender concern
Set-user-ID (setuid)	04000	Runs with the file owner's effective UID, so a root-owned setuid binary runs as root	Ignored on Linux	Every setuid root binary is a possible path to root (T1548.001); keep a baseline list and alert on new ones
Set-group-ID (setgid)	02000	Runs with the file's group	New files inherit the directory's group	Less dangerous, but audit setgid binaries owned by privileged groups
Sticky	01000	No practical effect	Only the file owner, directory owner or a privileged process can delete or rename entries	World-writable directories without it (other than by design) let users delete each other's files
World-writable (o+w)	0002	Anyone can modify	Anyone can create files	A root-run script or unit file that is world-writable is a direct escalation path

What to check first: files owned by root and executed by root (scripts called from cron or systemd, binaries in PATH) must not be writable by anyone else. Most real-world local escalation comes from this, not from kernel exploits.
Mount options: mounting /tmp, /var/tmp and removable media with nosuid and noexec removes whole classes of abuse. See the Linux Server Hardening Guide.
Beyond mode bits: POSIX ACLs and mandatory access control (SELinux, AppArmor) can further restrict or extend access, so ls -l is not always the full picture.

Capabilities: root split into parts

The capabilities(7) man page explains that Linux divides superuser power into distinct units that can be granted separately, so that a process needing one privileged action does not need full root. Capabilities are a per-thread attribute held in five sets: permitted (what the thread may use), effective (what the kernel checks), inheritable, bounding (a ceiling on what can be gained at execve) and ambient (preserved across execve of unprivileged programs). Executables can carry file capabilities in the security.capability extended attribute, which act like a narrower setuid.

Capability	What it allows	Risk
CAP_SYS_ADMIN	A large set of administrative operations including mount and namespace operations; the man page calls it "the new root"	Treat as root. Never grant to containers without a compelling reason
CAP_SYS_MODULE	Load and unload kernel modules	Kernel code execution
CAP_SYS_PTRACE	Trace any process and read its memory	Credential and secret theft from other processes
CAP_DAC_OVERRIDE / CAP_DAC_READ_SEARCH	Bypass file permission checks (write and read)	Read `/etc/shadow`, private keys, any file
CAP_SETUID	Change process UIDs arbitrarily	Become root directly
CAP_NET_ADMIN	Configure interfaces, firewall and routing	Disable host firewall, intercept traffic
CAP_NET_RAW	Raw and packet sockets	Sniffing and spoofing on the host network
CAP_BPF	Privileged BPF operations (split from CAP_SYS_ADMIN in Linux 5.8)	Kernel-level observation and tampering
CAP_NET_BIND_SERVICE	Bind ports below 1024	Low; the right way to let a web server use port 443 without root

Decision guidance: where a service needs one privileged action, grant the single capability (for example with systemd AmbientCapabilities=CAP_NET_BIND_SERVICE and a non-root User=) rather than running as root. Audit file capabilities with getcap -r / as part of the setuid baseline, because they do not show in a normal permission listing.

sudo, PAM and authentication

sudo and sudoers

sudo lets permitted users run commands as root or another user, controlled by /etc/sudoers and drop-in files under /etc/sudoers.d. The sudoers(5) manual gives several defender-relevant points:

Edit with visudo: it checks syntax before saving. Drop-in files are read in lexical order, so use zero-padded prefixes and review every included file.
ALL is dangerous: the manual warns that allowing ALL lets a user run any command, and that negating ALL to allow "all but a few" commands rarely works as intended. Editors, pagers and interpreters allowed through sudo give a shell escape; use sudoedit for file editing and the NOEXEC tag where a program should not start others.
NOPASSWD: scope it to specific commands, typically for automation accounts, never to ALL. Automation that needs root for one task should have one sudo rule for that task.
Logging: sudo logs allowed and denied attempts by default; log_output records session I/O, which is valuable for privileged sessions but can capture secrets.
Environment: keep env_reset on and set secure_path; the SETENV tag lets users bypass environment restrictions and should be rare.
Credential caching: the default timestamp is 5 minutes per user and terminal; a shorter timestamp_timeout narrows reuse. ATT&CK tracks abuse as T1548.003 Sudo and Sudo Caching.

PAM

Pluggable Authentication Modules assemble authentication from stacked modules configured under /etc/pam.d/, one file per service (sshd, sudo, login). The pam(8) page names four management groups: auth (prove identity), account (is the account allowed now), password (change credentials) and session (set up and tear down). pam.conf(5) defines control values: required (failure counts but the stack continues), requisite (failure returns immediately), sufficient (success returns immediately if no earlier required module failed), optional, and include / substack. A single misplaced sufficient line can let a weak module bypass MFA, and a malicious PAM module is a credential-capture and backdoor technique (T1556.003). Baseline /etc/pam.d and the module directory, and alert on change.

SSH keys and authorized_keys as non-human identities

The OpenSSH server reads public keys from ~/.ssh/authorized_keys by default (configurable with AuthorizedKeysFile). Each line is a standing credential: whoever holds the matching private key can log in as that account, with no expiry unless one is set. Automation, backup and deployment keys are among the most common unmanaged NHIs on Linux estates, and adding a key is a well known persistence technique (T1098.004 SSH Authorized Keys).

authorized_keys option	Effect (sshd(8))	Use it for
`from="pattern-list"`	Key accepted only from matching hosts or CIDR ranges	Automation keys used from known runners
`command="..."`	Forces one command regardless of what the client asks	Backup or replication keys
`restrict`	Disables port, agent and X11 forwarding, PTY allocation and ~/.ssh/rc	Default for every non-interactive key
`expiry-time="YYYYMMDD"`	Key rejected after the date	Temporary access, contractor keys
`cert-authority` with `principals=`	Trust certificates from a CA instead of individual keys	Short-lived SSH certificates at scale

sshd's StrictModes refuses the file if it, ~/.ssh or the home directory is writable by others. Monitor changes to every authorized_keys file, and for root in particular, and prefer centrally issued SSH certificates over scattered keys. Key lifecycle, discovery and rotation are covered in the SSH Key Management Guide.

Processes, systemd services and cron

Every running program is a process with a PID and the credentials above; /proc/<PID>/ exposes its command line, environment, open files and namespaces. On most modern distributions PID 1 is systemd, which starts services from unit files.

Unit file locations: vendor units in /usr/lib/systemd/system (or /lib/systemd/system), administrator units and overrides in /etc/systemd/system, and per-user units under ~/.config/systemd/user. New or modified units, and timers, are persistence (T1543.002 Systemd Service).
Service identity: systemd.exec(5) notes that User= defaults to root for system services, so a unit without it runs as root. DynamicUser= allocates a temporary user for the service's lifetime.
Sandboxing directives: NoNewPrivileges=yes stops the service and children gaining privilege through setuid or file capabilities; ProtectSystem=strict makes the file system read-only apart from API subtrees; ProtectHome= hides home directories; PrivateTmp= gives a private /tmp; CapabilityBoundingSet= removes every capability not listed.
cron: crontab(5) lists /etc/crontab and /etc/cron.d/ for system jobs (which include a username field) and /var/spool/cron/ for per-user crontabs. Scripts referenced from root jobs must be root-owned and not writable by others. Cron persistence is T1053.003.
Library preloading: /etc/ld.so.preload and the LD_PRELOAD variable load a library into every dynamically linked process; an unexpected entry is a strong rootkit indicator (T1574.006 Dynamic Linker Hijacking).

Namespaces and cgroups: what containers are

A container is an ordinary Linux process placed in its own namespaces and cgroups. The namespaces(7) page lists eight namespace types, each isolating one resource: cgroup (cgroup root), IPC, network (devices, stacks, ports), mount (mount points), PID (process IDs), time, user (user and group IDs) and UTS (hostname). Each process's memberships appear under /proc/<PID>/ns/. Control groups organise processes into a hierarchy whose resource use can be limited and accounted for: cgroups v2 (official since Linux 4.5) uses one unified hierarchy with controllers such as cpu, memory, pids and io.

Mechanism	Provides	Does not provide
Namespaces	A separate view of processes, network, mounts, hostname and IDs	A separate kernel; a kernel bug affects every container on the host
User namespace	Root inside the container mapped to an unprivileged UID outside	Protection if it is not enabled, which is still common
cgroups	Limits on CPU, memory, PIDs and I/O, stopping one container starving others	Access control or isolation of data
Capabilities, seccomp, LSMs	Restrict what the container's root can ask the kernel to do	Safety if the runtime is told to run privileged

Running a container privileged, as root without a user namespace, with CAP_SYS_ADMIN, or with the host's Docker socket mounted removes most of this isolation and makes escape to the host (T1611 Escape to Host) realistic. The man page also notes that since Linux 3.8 creating a user namespace needs no privilege, which is why some distributions restrict unprivileged user namespaces. See the Container Runtime Security Guide.

Package management and integrity

Distribution package managers (apt and dpkg on Debian and Ubuntu, dnf and rpm on Red Hat family systems) install software from signed repositories: the repository metadata or packages are signed and the client verifies them against trusted keys. That proves a package came from the repository, not that its upstream source was honest. The XZ Utils backdoor (CVE-2024-3094) showed the gap: a maintainer identity built up over about two years shipped release tarballs of versions 5.6.0 and 5.6.1 whose build scripts inserted a payload into liblzma, which could interfere with sshd authentication on systems where sshd loads liblzma through systemd. It was found because an engineer noticed slow SSH logins, and it reached mainly development and rolling distributions.

Verify installed files: rpm -Va compares installed files with package metadata; on Debian systems dpkg --verify does similar checks. Changed binaries in /usr/bin or /usr/sbin that no package update explains need investigation.
Repository trust: review which repositories and signing keys are configured; an extra repository is an extra supplier with root-level install rights.
Inventory: keep a package inventory or SBOM per host image so an advisory like XZ can be answered in minutes. See the Patch Management Guide.
Integrity monitoring: file integrity monitoring or IMA on critical paths catches changes outside package management.

What to monitor and how to measure it

Linux telemetry comes from the journal, auth logs, auditd and EDR sensors; collection and rule writing are covered in the Linux Logging and auditd for Defenders Guide and response in the Linux Detection and Response Guide. The fundamentals above give the watch list:

Identity files: changes to /etc/passwd, /etc/shadow, /etc/group, any new UID 0 account.
Privilege paths: changes to /etc/sudoers and /etc/sudoers.d, new setuid binaries or file capabilities, sudo denials.
Authentication stack: changes to /etc/pam.d, PAM module files, /etc/ssh/sshd_config and every authorized_keys file.
Persistence: new or modified systemd units and timers, crontabs, /etc/ld.so.preload.
Containers: containers started privileged or with host namespaces or the runtime socket mounted.

Metrics that show progress:

Number of setuid root binaries and file capabilities outside the approved baseline (target zero).
Percentage of sudo rules that name specific commands rather than ALL.
Percentage of authorized_keys entries with restrict or from=, or replaced by SSH certificates.
Percentage of custom systemd services running with a non-root User= and NoNewPrivileges=yes.
Number of running containers that are privileged or run as root without a user namespace.
Time to answer "which hosts have package X version Y" for a new advisory.

Framework mapping

Topic	NIST SP 800-53 Rev. 5	NIST CSF 2.0	MITRE ATT&CK v20
Users, sudo, setuid, capabilities	AC-6, AC-6(9)	PR.AA-05	T1548.001, T1548.003, T1078.003
PAM and SSH keys	IA-5	PR.AA-05	T1556.003, T1098.004
systemd, cron, preload	CM-6, CM-7	PR.PS-01	T1543.002, T1053.003, T1574.006
Containers	CM-7, SI-4	DE.CM-09	T1611
Package integrity	SI-7 Software, Firmware, and Information Integrity	PR.PS-02	T1195.002
Monitoring	AU-12, SI-4	PR.PS-04, DE.CM-09	T1685

In CIS Controls v8.1 the same material sits mainly under Control 2 (Inventory and Control of Software Assets), Control 4 (Secure Configuration), Control 5 (Account Management), Control 6 (Access Control Management) and Control 8 (Audit Log Management).

Common failure patterns

Broad sudo for convenience. ALL=(ALL) NOPASSWD: ALL for a deployment user turns any compromise of that account, or its SSH key, into root.
Writable scripts run by root. A cron job or unit runs a script in a directory a service user can write, giving silent escalation.
SSH keys that never die. Keys for departed staff and decommissioned automation remain in authorized_keys for years, with no source restriction or expiry.
Services without User=. Custom units run as root by default because nobody set an account or sandboxing directives.
Containers treated as VMs. Privileged containers or a mounted Docker socket are approved because "it is isolated", when the container shares the host kernel.
Trusting the package signature alone. Signed repositories do not catch a compromised upstream, as the XZ Utils case showed; inventory and behaviour monitoring are needed too.

Practitioner checklist

Audit for UID 0 accounts other than root and for service accounts with interactive shells.
Review every sudoers file: remove ALL grants, scope NOPASSWD to single commands, use sudoedit, keep env_reset and secure_path.
Baseline setuid, setgid and file-capability binaries and alert on additions.
Check that no root-executed script, unit file or binary is writable by non-root users; mount /tmp nosuid,noexec.
Inventory authorized_keys across the estate; add restrict and from= to automation keys, set expiry for temporary keys, and plan a move to SSH certificates.
Baseline /etc/pam.d and sshd_config and monitor them for change.
Give every custom systemd service a non-root User= (or DynamicUser=), NoNewPrivileges=yes, ProtectSystem=strict and a minimal CapabilityBoundingSet.
Block privileged containers and Docker socket mounts by policy, and enable user namespaces where the runtime supports them.
Run package verification on critical hosts and keep a per-image package inventory for advisory lookups.

Linux Fundamentals for Defenders Guide: Users, Permissions, Processes and Services