Hadoop Security Guide: Kerberos, Ranger and Data Lake Controls

An Apache Hadoop cluster in its default configuration trusts whatever user name a client claims, so anyone who can reach it on the network can read data and run code. The Apache Hadoop documentation is blunt about this: with hadoop.security.authentication left at simple, the only protection is restricting network access. This guide covers how to move a cluster to secure mode with Kerberos, how to authorise access with HDFS permissions, service-level ACLs and Apache Ranger, how to encrypt data on the wire and at rest with HDFS transparent encryption and the Hadoop KMS, and how to monitor the result. It is for platform engineers and security architects running self-managed Hadoop, Hive, Spark and HBase estates.

Key takeaways

• Secure mode is not optional. Without Kerberos, HDFS and YARN accept the client's asserted user name, and exposed YARN ResourceManagers have been used to run cryptominers on unauthenticated clusters.
• Kerberos is only the start. Authentication must be paired with service-level authorisation, HDFS permissions and ACLs, and a central policy layer such as Apache Ranger for Hive, HBase and other engines.
• Keytabs are the cluster's crown jewels. Every daemon and many pipelines run on keytab files that work like passwords; their storage, ownership and rotation decide whether a single host compromise becomes a cluster compromise.
• Encrypt in three places. Set hadoop.rpc.protection and dfs.data.transfer.protection to privacy, turn on HTTPS for every web UI, and put sensitive data in HDFS encryption zones whose keys live in a KMS with its own ACLs.
• Proxy users and superusers need the tightest scoping. A hadoop.proxyuser entry with wildcard hosts and groups lets one service impersonate anyone, which defeats the rest of the model.

The Hadoop security model and why the defaults are unsafe

Hadoop grew as a trusted-network system. Its security features, as the project's "Hadoop in Secure Mode" page lists them, are authentication, service-level authorisation, authentication for web consoles and data confidentiality. None of them is on by default. The defaults that matter most are:

• Authentication: hadoop.security.authentication defaults to simple, meaning no authentication. A client can set any user name, including hdfs, the HDFS superuser.
• Authorisation: hadoop.security.authorization (RPC service-level authorisation) is off until set to true.
• Wire protection: hadoop.rpc.protection defaults to authentication (no integrity or encryption), and dfs.encrypt.data.transfer defaults to false.
• Web consoles: dfs.http.policy defaults to HTTP_ONLY, so NameNode, DataNode and JournalNode UIs and WebHDFS run in clear text unless HTTPS and SPNEGO are configured.
• Ecosystem services: Spark, Hive, HBase, Oozie, Livy and notebook services each have their own authentication switches. Apache Spark's security notes, for example, state that the standalone REST submission API (port 6066 by default) does not use authentication and recommend setting spark.master.rest.enabled to false where it is not needed.

Treat the cluster as a set of exposed services, each needing its own controls. The Data Security Guide covers the wider data protection programme; this guide stays inside the Hadoop stack.

Authentication: Kerberos, principals and keytabs

In secure mode every Hadoop service and every user authenticates with Kerberos. The cluster needs a KDC (MIT Kerberos, or Active Directory acting as one), correct forward and reverse DNS for every host, and a principal and keytab for each service instance.

Service principals and daemon accounts

The Hadoop documentation recommends running HDFS and YARN daemons as different Unix users (hdfs, yarn, and mapred for the JobHistory server) that share a group such as hadoop. Service principals take the form ServiceName/_HOST@REALM, for example nn/[email protected] for the NameNode and dn/[email protected] for DataNodes. The _HOST placeholder is replaced with each host's fully qualified name at runtime, so one configuration file can be pushed to every node while each node still has its own key.

Mapping principals to users

hadoop.security.auth_to_local converts Kerberos principals to local user names using RULE: expressions and a final DEFAULT. A loose rule can map ordinary principals to hdfs, the superuser. Test every rule with hadoop kerbname <principal> before deployment and keep rules anchored to your realm. Group membership comes from hadoop.security.group.mapping (shell-based or LDAP) and must match your directory.

DataNode authentication

The DataNode data transfer protocol does not use Hadoop RPC. Older secure clusters solved this by starting DataNodes as root through jsvc so they could bind privileged ports (for example 1004 and 1006) and then drop to HDFS_DATANODE_SECURE_USER. Since Hadoop 2.6.0, SASL can authenticate the data transfer protocol instead: set dfs.data.transfer.protection, use a non-privileged dfs.datanode.address and set dfs.http.policy to HTTPS_ONLY. SASL removes the need to run any part of the DataNode as root and is the preferred design for new clusters.

Keytab handling

• Ownership: each keytab readable only by its daemon user (mode 0400 or 0600), on local disk, never in HDFS, shared NFS, configuration repositories or container images.
• Scope: one principal per service per host; never copy the hdfs headless keytab to edge nodes or ETL servers.
• Rotation: rotate service keys on a schedule and immediately after any host compromise, using your KDC's key version numbers (kvno) so running services pick up the new key on restart.
• Pipelines: batch jobs and schedulers should use their own principals rather than borrowing a human user's keytab. See the Service Account Security Guide and the Kerberos Security Guide.

Jobs use delegation tokens from the NameNode, ResourceManager or KMS rather than keytabs in every container. Tokens are bearer credentials until they expire, another reason to run containers as the submitting user.

Authorisation: HDFS permissions, service ACLs and proxy users

HDFS permissions and ACLs

HDFS uses a POSIX-like owner, group and other model with an optional sticky bit on directories, plus POSIX ACLs (enabled by default through dfs.namenode.acls.enabled). The superuser is whoever runs the NameNode, plus members of dfs.permissions.superusergroup (default supergroup). Three practical rules follow:

• Superuser group: set dfs.permissions.superusergroup to a dedicated, tightly controlled group, and make sure no general-purpose directory group has that name.
• Umask: the default fs.permissions.umask-mode of 022 creates world-readable files. Use 027 or 077 for clusters holding personal or regulated data, and set restrictive modes on landing and staging directories.
• Shared directories: set the sticky bit on /tmp and other shared paths.

Service-level authorisation

With hadoop.security.authorization=true, hadoop-policy.xml controls which users and groups may connect to each protocol, for example security.client.protocol.acl for HDFS clients. Each ACL has a .blocked counterpart and a host list (security.client.protocol.hosts); if no host list is defined, the documented default is *. Restrict administrative protocols to the admin group and reload changes with hdfs dfsadmin -refreshServiceAcl and yarn rmadmin -refreshServiceAcl.

Proxy users

Services such as Oozie, Hive Server 2, Livy, Knox and Hue act on behalf of end users through Hadoop's proxy user (impersonation) mechanism, configured with hadoop.proxyuser.<service>.hosts, .groups and .users. A setting of * for hosts and groups allows that service account, from any host, to impersonate any user, including administrators. Limit hosts to the service's own servers (IP addresses or CIDR ranges) and groups to the user population that actually uses the service. Review it like any privileged grant. See the Kerberos Delegation Security Guide.

Central policy with Apache Ranger and perimeter access with Knox

HDFS permissions do not express table, column or row access, and each engine (Hive, HBase, Kafka, Solr, YARN queues) has its own model. Apache Ranger centralises this: its stated goals are centralised security administration through a UI or REST API, fine-grained authorisation, standardised authorisation across components, role-based and attribute-based access control, and centralised auditing. Plugins inside each service download policies from Ranger Admin, cache them locally and enforce them in-process.

Making Ranger effective

1. Close the file bypass: once Hive tables are governed by Ranger, set the warehouse directories so that only the hive service user can read them. Otherwise a user denied a column in SQL can read the Parquet or ORC files with hdfs dfs -cat.
2. Use tags for sensitive data: in Ranger's tag model, Atlas publishes classifications through Kafka, ranger-tagsync loads them into Ranger's tag store, and tag policies are evaluated before resource policies. A deny from any tag policy denies access, so one "PII deny unless approved" policy can protect every tagged table.
3. Order masking items carefully: for row filters and masks, the first matching policy item wins. Exemptions (an empty filter, or "Unmasked") must be placed above the general rule. Insert, update, delete and export are denied to a user who has a filter or mask on that table.
4. Treat Ranger Admin as tier 0: put it behind SSO and MFA, restrict its admin role and audit policy changes.

Apache Knox is a reverse proxy that gives one authenticated entry point for REST and UI access to services such as WebHDFS, the YARN ResourceManager, Hive over JDBC/HTTP, Livy and the Ranger Admin console. It supports LDAP/AD authentication, KnoxSSO, and federation through SAML or OpenID Connect via Pac4J, plus ACLs by user, group and IP.

Encryption in transit and at rest

In transit

Set hadoop.rpc.protection=privacy for RPC, dfs.data.transfer.protection=privacy (or dfs.encrypt.data.transfer=true) for block transfer with dfs.encrypt.data.transfer.cipher.suites=AES/CTR/NoPadding and a 256-bit dfs.encrypt.data.transfer.cipher.key.bitlength, and HTTPS_ONLY for HDFS, YARN (yarn.http.policy) and the MapReduce JobHistory server (mapreduce.jobhistory.http.policy). Other ecosystem services need their own TLS settings. Benchmark throughput before and after so the performance cost is understood rather than used as a reason to switch it off.

At rest: HDFS transparent encryption

HDFS transparent encryption works through encryption zones: directories whose contents are encrypted on write and decrypted on read by the client. Each zone has a zone key held in a Key Management Server (KMS). When a file is created, the NameNode asks the KMS for a new data encryption key encrypted with the zone key (an EDEK) and stores it in the file metadata. On read, the client sends the EDEK to the KMS, which checks the client's permission on the zone key before returning the decrypted key. As the documentation notes, a compromised HDFS superuser then sees only ciphertext and encrypted keys, because key access is controlled separately on the KMS.

• KMS ACLs: the Hadoop KMS has two layers, KMS ACLs in kms-acls.xml (operations such as CREATE, DELETE, ROLLOVER, GET, GET_KEYS, GET_METADATA, GENERATE_EEK, DECRYPT_EEK) and per-key ACLs. Each operation also has a hadoop.kms.blacklist.* property; blacklist the hdfs user from DECRYPT_EEK so the HDFS superuser cannot decrypt file data.
• Separation of duties: key administrators create and roll keys; HDFS administrators create zones with hdfs crypto -createZone -keyName <key> -path <empty dir>; data users get DECRYPT_EEK only on the keys for zones they need.
• KMS authentication: set hadoop.kms.authentication.type to kerberos and serve the KMS over HTTPS.
• Existing data: zones must be empty when created, so copy data in and securely remove old copies.
• Passwords in configuration: keystore and database passwords in *-site.xml files should move to the Hadoop Credential Provider API (hadoop credential create ... -provider jceks://...), which is consulted before clear-text configuration values.

Disk or volume encryption on DataNodes (see the Encryption Guide) protects against stolen drives but not against anyone with HDFS access. Use encryption zones for regulated data and keep KMS keys under the controls in the Cryptographic Key Management Guide.

YARN, compute isolation and network exposure

YARN runs arbitrary user code, which makes it the most attractive target in the cluster. In January 2024 Aqua Security reported a campaign that used Hadoop YARN ResourceManagers configured to let unauthenticated users create and run applications to deploy rootkits and a Monero miner. Apache Spark also had a command injection in its UI permission check (CVE-2022-33891, in CISA's Known Exploited Vulnerabilities Catalog) when spark.acls.enable was set, fixed in Spark 3.2.2 and 3.3.0.

• Container execution: use LinuxContainerExecutor, with the container-executor binary owned by root, group-owned by the NodeManager group and mode 6050, and container-executor.cfg setting banned.users=hdfs,yarn,mapred,bin and min.user.id=1000 so jobs cannot run as service or system accounts.
• Queue ACLs: set yarn.acl.enable=true and define submit and administer ACLs per queue, either in the scheduler configuration or through Ranger's YARN plugin.
• Network: place masters and workers in a dedicated segment, allow client access only through edge nodes and Knox, and never expose ResourceManager (8088), NameNode (9870) or Spark master ports to the internet. See the Network Segmentation Guide.
• Edge nodes: require MFA for SSH, no shared accounts, session logging; harden all nodes with the Linux Server Hardening Guide.

Implementation roadmap

1. Inventory and exposure check (weeks 1 to 2): list every service, port and web UI; scan from outside the cluster network; record current values of the settings in the table above; list every hadoop.proxyuser entry.
2. Network containment first: close internet exposure and route user access through edge nodes and Knox; this is the fastest risk reduction.
3. Kerberise a non-production cluster: create principals and keytabs, configure auth_to_local, enable Kerberos and SPNEGO, and run every production job against it. Expect failures in schedulers and BI connectors.
4. Production cut-over in a maintenance window: Kerberos is a cluster-wide switch, not a ring rollout. Keep the previous configuration set and keytab-free rollback plan versioned so you can revert in one restart if critical jobs fail.
5. Authorisation: enable service-level ACLs, tighten umask and superuser group, scope proxy users, then deploy Ranger plugins in a mode that audits before enforcing (start with permissive policies that mirror current access, review the Ranger audit for a month, then narrow).
6. Encryption: enable RPC and data transfer privacy, HTTPS on UIs, then stand up the KMS and migrate regulated datasets into encryption zones one zone at a time.
7. Operate: keytab rotation, quarterly review of Ranger policies and proxy users, patching against vendor and Apache advisories, and the monitoring below.

Small versus large estates: a small cluster can use OS group lookup, a few Ranger policies and one KMS; large multi-tenant estates need LDAP group mapping, Atlas-driven tag policies, HA KMS and automated keytab management. If the team cannot sustain Kerberos operations, migrate to a managed lakehouse rather than run an unsecured cluster (see the Data Lake Security Guide).

Detection and measurement

Collect the NameNode audit log (every file system operation with user, IP, command and path), Ranger audit events (allowed and denied access with the policy that decided), KMS audit logs, ResourceManager application logs, KDC logs and Knox audit logs into your SIEM. Useful detections:

• Any request authenticated as hdfs, yarn or another service principal from a host that does not run that service.
• YARN applications submitted by unexpected users, or whose containers launch shells, download binaries or run long at full CPU (MITRE ATT&CK T1496.001 Compute Hijacking).
• Bulk reads of warehouse directories by users who normally query through Hive or Spark SQL (T1213 Data from Information Repositories), and bulk deletes or -skipTrash removals (T1485 Data Destruction).
• KMS DECRYPT_EEK spikes, key deletions and rollovers outside change windows, and changes to Ranger policies, proxy user settings and auth_to_local rules.
• Kerberos ticket anomalies from the KDC, such as many service ticket requests for Hadoop principals from one host (T1558 Steal or Forge Kerberos Tickets), and keytab files read by unexpected processes (T1552.001 Credentials In Files).

Framework mapping

Exposed YARN and Spark endpoints map to ATT&CK T1190 Exploit Public-Facing Application; encryption zones map to D3FEND D3-FE File Encryption.

Common failure patterns

• Kerberos on HDFS, not on everything else: HDFS is secured but Spark standalone, Livy, notebooks or the YARN REST API still accept unauthenticated work, which is exactly the gap cryptomining campaigns use.
• Wildcard proxy users: hadoop.proxyuser.hive.hosts=* and groups=* left from installation turn the Hive service account into a master key.
• Ranger policies with open files underneath: column masking in Hive while the warehouse directory is readable by a broad group.
• Keytabs as shared secrets: the hdfs keytab copied to ETL servers or stored in a configuration repository, where one leak gives superuser access. The GoTo breach shows the same principle for encryption keys: protection disappears when the key travels with the data.
• KMS without separation: the hdfs user not blacklisted from DECRYPT_EEK, or HDFS and key administration held by the same people, which removes the point of encryption zones.
• Privacy mode switched off for performance and never revisited.

Practitioner checklist

• Set hadoop.security.authentication=kerberos and hadoop.security.authorization=true on every cluster, and enable authentication on Spark, Hive, HBase, Livy and notebook services.
• Run HDFS, YARN and MapReduce daemons as separate users, with one principal and keytab per service per host, keytabs mode 0400 on local disk.
• Test every auth_to_local rule with hadoop kerbname and anchor rules to your realm.
• Remove wildcards from hadoop.proxyuser.* settings and review them quarterly.
• Set RPC and data transfer protection to privacy with AES, and HTTPS_ONLY with SPNEGO on all web UIs.
• Deploy Ranger plugins, close direct file access to governed warehouse directories, and drive sensitive-data policies from tags.
• Put regulated data in HDFS encryption zones, blacklist hdfs from DECRYPT_EEK, and separate key and HDFS administration.
• Use LinuxContainerExecutor with banned.users and min.user.id, and queue ACLs.
• Expose no cluster ports to the internet; route users through edge nodes and Knox.
• Forward NameNode, Ranger, KMS, YARN, KDC and Knox audit logs to the SIEM and alert on the detections above.

Standards and references

• Apache Hadoop: Hadoop in Secure Mode
• Apache Hadoop: Transparent Encryption in HDFS
• Apache Hadoop: Hadoop Key Management Server (KMS)
• Apache Hadoop: Proxy user, superusers acting on behalf of other users
• Apache Ranger: Tag Based Policies
• Apache Knox: Knox Gateway
• Apache Spark: Spark Security (published vulnerabilities)
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Data Lake Security Guide · Kerberos Security Guide · Database Security Guide · Service Account Security Guide · Databricks Security Guide