Linux Server Hardening Guide

A default Linux install is built to work out of the box, not to resist an attacker who already has a foothold, and most server compromises exploit the gap: password SSH, broad sudo rights, unmonitored changes and services that run with far more privilege than they need. Hardening closes that gap with a known baseline, applied by automation and checked continuously. This guide covers CIS Benchmarks and DISA STIGs, minimal builds and patching, SSH, sudo and PAM, SELinux and AppArmor, systemd sandboxing, kernel sysctl settings, nftables, auditd, AIDE, central logging and OpenSCAP scanning. It is written for Linux administrators, platform and SRE teams, and security engineers who own server baselines on RHEL-family, Debian and Ubuntu systems. Setting names and defaults are taken from upstream manual pages and vendor documentation; check them against your distribution release.

Key takeaways

• Start from a published baseline. CIS Benchmark Level 1 Server or the DISA STIG for your distribution gives an auditable starting point; tailor it and record every exception.
• SSH is the front door and the identity problem. Disable root and password logins, restrict who may connect, and manage the keys and service accounts behind them as credentials with owners.
• Least privilege needs enforcement, not just policy. Narrow sudo rules, SELinux or AppArmor in enforcing mode and systemd sandboxing limit what a compromised account or service can do.
• Detection is part of hardening. auditd rules for identity files and privileged commands, AIDE integrity checks and central log forwarding make tampering visible.
• Automate and measure drift. Apply the baseline with configuration management and scan with OpenSCAP so compliance is a number, not an assumption.

Choosing a baseline: CIS Benchmarks and DISA STIGs

CIS describes its Benchmarks as a "consensus-based effort of cybersecurity experts globally", with Linux benchmarks for RHEL, Rocky, AlmaLinux, Oracle Linux, Debian, Ubuntu, SUSE and Amazon Linux among others. DISA Security Technical Implementation Guides (STIGs) are the US Department of Defense configuration standards, with findings rated CAT I (high), CAT II (medium) and CAT III (low).

CIS also publishes hardened images on cloud marketplaces. Whichever you choose, keep a tailoring file listing disabled rules and the reason, because unexplained exceptions are what auditors and attackers both find.

Minimal build and patching

• Install the minimum: start from the minimal or server profile, remove compilers, X11 and unused network services, and confirm what listens with ss -tulpn. Disable units you do not need with systemctl disable --now and systemctl mask.
• Separate and restrict filesystems: mount /tmp, /var/tmp and /dev/shm with nodev,nosuid,noexec where workloads allow, and keep /var/log and /var/log/audit on their own partitions so log floods cannot fill root.
• Set a restrictive default umask: 027 for system accounts, and keep sensitive files at mode 600 or 640 (chmod 600 for private keys and secrets files, root-owned with chown root:root).
• Review SUID and SGID binaries: list them with find / -xdev -perm -4000 -o -perm -2000, remove the bit where not required and alert on new ones. Abuse is ATT&CK T1548.001 Setuid and Setgid.
• Patch on a schedule: unattended-upgrades on Debian and Ubuntu and dnf-automatic on RHEL-family systems apply security updates; plan reboots or kernel live patching so the running kernel matches the installed one. See the Patch Management Guide.
• Trust the package path: install only from signed repositories you control or mirror. The XZ Utils backdoor showed how a compromised upstream library could reach sshd through ordinary distribution updates.

SSH hardening

OpenSSH defaults favour compatibility. According to the sshd_config(5) manual for OpenSSH 9.9, PasswordAuthentication, KbdInteractiveAuthentication, AllowTcpForwarding and AllowAgentForwarding all default to yes and PermitRootLogin defaults to prohibit-password. A typical server baseline:

• Check the effective configuration: sshd uses the first value it reads for most keywords, and Include /etc/ssh/sshd_config.d/*.conf drop-ins, often written by cloud-init, can re-enable passwords. Run sshd -T to print what is actually in force and sshd -t before reloading.
• Rate limiting: OpenSSH 9.8 and later apply PerSourcePenalties by default; fail2ban jails on the sshd log add longer bans for hosts that must expose port 22. Neither replaces turning passwords off.
• Keys are credentials: ATT&CK T1098.004 SSH Authorized Keys notes adversaries "may modify the SSH authorized_keys file to maintain persistence", and T1021.004 covers lateral movement over SSH with valid keys. Inventory keys, assign owners, prefer SSH certificates with short validity and remove keys at leaver time. Rotate host keys if they leak, as in the GitHub RSA SSH host key exposure. See the SSH Key Management Guide.

Accounts, sudo and PAM

• Edit with visudo, scope narrowly: put rules in /etc/sudoers.d/ and grant specific commands to groups, not ALL. The sudoers manual warns that "Wildcards in command line arguments should be used with care" and that negating ALL to exclude a few commands "rarely works as intended". Editors, pagers and shells allowed through sudo give a root shell; use sudoedit for file edits.
• Logging defaults: set Defaults use_pty, Defaults logfile="/var/log/sudo.log" and, for administrators of sensitive systems, log_input and log_output (off by default). sudo 1.9 can send event and I/O logs to a central sudo_logsrvd through log_servers.
• Sudo caching: credentials are cached for 5 minutes per terminal by default (timestamp_timeout). Shorten it on shared bastions; ATT&CK tracks abuse as T1548.003 Sudo and Sudo Caching. Avoid NOPASSWD: ALL except for tightly scoped automation.
• Lockout and password quality: pam_faillock reads /etc/security/faillock.conf (for example deny=5, unlock_time=900). It does not lock root unless even_deny_root is set, and its records in /var/run/faillock may not survive a reboot unless dir= points elsewhere. Pair it with pam_pwquality for any remaining passwords. Change PAM through authselect on RHEL-family systems and pam-auth-update on Debian and Ubuntu rather than editing stacks by hand.
• Service accounts: give each daemon its own account with a /usr/sbin/nologin shell, no password, no SSH access and no sudo unless a reviewed rule needs it. Remove or lock default and vendor accounts. See the Service Account Security Guide.
• Secrets on disk: database passwords and API tokens in .env files or unit files are readable by anyone who compromises the service user. Inject them from a vault or with systemd LoadCredential=. See the Secrets Management Guide.

Mandatory access control and service sandboxing

SELinux (RHEL, Fedora, Rocky, AlmaLinux) and AppArmor (Ubuntu, Debian, SUSE) confine processes even when they run as root. Keep SELinux at SELINUX=enforcing in /etc/selinux/config (check with getenforce) and fix denials with booleans (setsebool -P) or file contexts (semanage fcontext) rather than setenforce 0. On AppArmor, aa-status lists profiles in enforce and complain mode; move custom profiles from complain to enforce once logs are clean.

systemd adds per-service isolation without writing a full policy. Apply it in a drop-in with systemctl edit <unit> and score units with systemd-analyze security.

Kernel, network and firewall settings

Put settings in /etc/sysctl.d/60-hardening.conf and load them with sysctl --system. Values below follow the kernel documentation and common CIS recommendations; test each against your workloads.

Host firewall with nftables

nftables replaces iptables on current distributions, and firewalld on RHEL-family systems uses it as its backend. The nftables project's server example uses one inet table for IPv4 and IPv6, an input chain with policy drop, ct state established and related accepted and invalid dropped, loopback accepted, ICMPv6 neighbour discovery allowed ("connectivity breaks" otherwise), and an explicit list such as tcp dport { 22, 443 } accept; forward is dropped unless the host routes. Restrict port 22 to bastion or management ranges with a source set rather than opening it to the world, log dropped packets at a rate limit, and load rules from a version-controlled file with nft -f. See the Firewall Management Guide.

auditd, file integrity and central logging

The audit.rules(7) manual prefers the explicit syscall form over the older -w watch syntax, asks for an arch on every syscall rule for performance, and recommends keys so ausearch -k and aureport can find events. Place rules in /etc/audit/rules.d/, load them with augenrules --load, and end with -e 2 to make the rule set immutable until reboot.

• Identity and privilege files: -a always,exit -F arch=b64 -F path=/etc/shadow -F perm=wa -F key=identity, with similar rules for /etc/passwd, /etc/group, /etc/sudoers, /etc/sudoers.d and /etc/ssh/sshd_config.
• Privileged commands by real users: -a always,exit -F arch=b64 -S execve -F euid=0 -F auid>=1000 -F auid!=unset -k priv_exec, which ties root activity back to the login user.
• Kernel modules: -S init_module,finit_module,delete_module with a modules key.
• SSH keys: watch authorized_keys paths; ATT&CK's detection for T1098.004 looks for exactly these modifications.
• File integrity with AIDE: aide --init, move /var/lib/aide/aide.db.new.gz to aide.db.gz, then run aide --check. Red Hat advises at least weekly and ideally daily checks, aide --update after approved changes, and keeping the database, configuration and binary on "read-only media" or off-host. AIDE detects; IMA can also prevent execution of altered files.
• Central logging: forward journald, /var/log/secure or auth.log, sudo logs and audit records to a SIEM over TLS, so an attacker with root cannot erase the only copy. See the Log Management and Retention Guide.

Automating and verifying the baseline

Hardening applied by hand drifts within weeks. Define the baseline as code (Ansible roles such as those in the DevSec Hardening Framework, or Puppet, Chef or Salt), build golden images from it, and scan running hosts. On RHEL the SCAP Security Guide content lives at /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml: oscap info lists profiles, oscap xccdf eval --profile <id> --report report.html <data stream> scans (oscap-ssh does the same remotely), and oscap xccdf generate fix produces an Ansible playbook without changing the host. Red Hat warns that remediation can leave a system non-functional and offers no automated revert. On Ubuntu Pro, usg audit and usg fix wrap the same OpenSCAP technology, with tailoring files for exceptions. Lynis gives a quick, agentless second opinion.

Implementation roadmap

1. Inventory and classify servers by distribution, role and exposure; record who owns each and which baseline applies.
2. Scan before changing: run OpenSCAP or CIS-CAT in report mode to get a baseline score per role.
3. Tailor: disable rules that break the role, with a written reason and an owner, and version the tailoring file.
4. Pilot: apply through configuration management to non-production hosts, run application tests, and keep an existing root session open when changing SSH or PAM. Put AppArmor profiles in complain mode first and review SELinux denials with ausearch -m avc before enforcing.
5. Roll out in rings: internet-facing and bastion hosts first, then internal tiers. Roll back by reverting the configuration commit and re-running the play, which is why manual fixes are not allowed.
6. Bake into images so new servers start compliant, then scan continuously and alert on drift.

Decision points

• CIS or STIG: choose STIG when a contract or regulator requires it; otherwise CIS Level 1 Server, with Level 2 for exposed or sensitive hosts.
• SELinux or AppArmor: use the one your distribution ships and supports. Switching adds risk without benefit.
• Keys or certificates for SSH: keys with central authorized_keys suit small estates; an SSH CA with short-lived certificates scales better and makes leaver removal automatic.
• Small teams can rely on CIS hardened images, USG or SSG playbooks and a managed SIEM. Large estates need a baseline owner, per-role tailoring, continuous scanning and exception expiry dates.

Detection and measurement

Alert on: SSH logins by password or as root, new keys in authorized_keys, changes to sudoers or sshd_config, new SUID files, SELinux switched to permissive, auditd stopped or rules changed, AIDE differences outside change windows, and kernel module loads on servers that should not load modules. See the SIEM Guide.

• Baseline compliance: average OpenSCAP or CIS-CAT score per role, and hosts below the agreed threshold.
• SSH posture: hosts with PasswordAuthentication or root login enabled according to sshd -T (target zero).
• Privilege sprawl: sudo rules granting ALL or NOPASSWD, trended down.
• MAC coverage: hosts with SELinux enforcing or AppArmor profiles enforced (target 100 per cent).
• Logging coverage: hosts sending audit and auth logs to the SIEM in the last 24 hours.
• Patch latency: median days from security update release to install, and hosts running a kernel older than the installed one.

Common failure patterns

• Password logins left on: one forgotten host that still accepts passwords becomes the entry point. In the PHP Git server compromise, password-based logins to a self-hosted server let an attacker push a backdoor.
• Drop-in overrides: sshd_config looks hardened, but a cloud-init file in sshd_config.d wins because it is read first.
• SELinux disabled to fix one application and never re-enabled.
• Shared admin accounts and keys: one key copied to every server for "ops", with no owner and no expiry.
• Hardening once, by hand: no configuration management, so new servers and rebuilt ones miss half the baseline.
• Logs only on the host: audit trails that root can delete are gone when needed.

Framework mapping

Practitioner checklist

• Pick CIS Level 1 Server, Level 2 or the STIG per server role and version a tailoring file with reasons.
• Set PermitRootLogin no, PasswordAuthentication no, KbdInteractiveAuthentication no and AllowGroups, and verify with sshd -T.
• Inventory SSH keys and service accounts, assign owners and move to SSH certificates where practical.
• Replace broad sudo rules with command-specific ones, and enable use_pty and a sudo log file.
• Keep SELinux or AppArmor enforcing and add systemd sandboxing to every custom service.
• Deploy the sysctl and nftables baselines from version control.
• Load auditd rules for identity files, sudoers, sshd_config, privileged execution and modules, with -e 2.
• Initialise AIDE, store its database off-host and run checks at least daily.
• Forward auth, sudo and audit logs to the SIEM and alert when a host goes quiet.
• Scan with OpenSCAP or CIS-CAT on a schedule and report drift per role.

Standards and references

• CIS: CIS Benchmarks
• OpenSSH (man7.org): sshd_config(5)
• Sudo (man7.org): sudoers(5)
• Linux kernel documentation: Documentation for /proc/sys/kernel/
• Linux Audit (man7.org): audit.rules(7)
• systemd (man7.org): systemd.exec(5)
• Red Hat: Checking integrity with AIDE
• nftables wiki: Simple ruleset for a server

Related NHI Mgmt Group resources: SSH Key Management Guide · Service Account Security Guide · Secrets Management Guide · Endpoint Security Guide · Privileged Access Management Guide