By default an Amazon EMR cluster is single tenant: every job on every node acts with the same EC2 instance profile, so one user's Spark job can read whatever any job on that cluster is allowed to read. EMR offers the controls to change that, including runtime roles, Lake Formation, Kerberos, security configurations for encryption and block public access, but they are opt-in and several have release, application and high-availability constraints. This guide explains the EMR identity model, the three IAM roles every cluster uses, how runtime roles and Lake Formation give per-job permissions, how to choose a Kerberos architecture, how to encrypt data at rest and in transit, and how EMR Serverless and EMR Studio differ. It is for data platform engineers, AWS security teams and auditors. Open-source Hadoop controls are covered in the Hadoop Security Guide.
This is an independent guide from NHI Mgmt Group, based on AWS's public documentation as of November 2026. It is not sponsored or reviewed by AWS. Product features change, so confirm settings against AWS's current documentation.
Key takeaways
- The instance profile is the default identity for every job. Without runtime roles or fine-grained access control, all jobs share the EC2 instance profile's permissions, so it ends up with the union of everything any job needs.
- Runtime roles give per-job permissions. From emr-6.7.0, Spark and Hive steps can run with a role passed as --execution-role-arn; such jobs cannot reach the instance metadata service or the instance profile.
- Restrict who can pass which runtime role. AWS states the elasticmapreduce:ExecutionRoleArn condition must be included when granting AddJobFlowSteps or GetClusterSessionCredentials.
- Security configurations carry encryption and Kerberos. They are reusable objects applied at cluster creation (EMR 4.8.0 and later) and cover S3, local disk, EBS and in-transit encryption plus authentication and authorisation settings.
- Block public access is on by default; keep it that way. It stops clusters in public subnets launching with security group rules open to 0.0.0.0/0 or ::/0, with only port 22 excepted by default.
How identity and access work in Amazon EMR
EMR has two planes. The control plane is the EMR API (RunJobFlow, AddJobFlowSteps, ModifyCluster and so on), authorised by IAM and logged in AWS CloudTrail. The data plane is the cluster itself: open-source applications (Spark, Hive, Presto, HDFS, YARN) running on EC2 instances, which authenticate users through IAM Identity Center, LDAP or Kerberos and access AWS services with an IAM role.
| Identity | Used by | Scope | Hardening point |
|---|---|---|---|
| EMR service role | The EMR service, to provision EC2 instances and manage the cluster | Account level | Use AmazonEMRServicePolicy_v2 so block public access can revoke non-compliant rules |
| EC2 instance profile | Every cluster instance and any application process without a runtime role | Whole cluster | Keep minimal; it is the fallback identity for all jobs |
| Runtime role | An individual Spark or Hive step, query or Studio session | Per job | Least privilege per team or pipeline; restricted with ExecutionRoleArn conditions |
| Workforce identities | Humans via IAM Identity Center (recommended by AWS), LDAP or Kerberos | Per user | Trusted identity propagation carries user identity to data access |
| EMR Serverless job runtime role | An EMR Serverless job run | Per job run | Pass only specific roles with iam:PassedToService |
AWS describes clusters as single tenant by default. Enabling runtime roles, IAM Identity Center, Kerberos or LDAP makes a cluster multi-tenant, and when runtime roles or fine-grained access control are enabled EMR uses iptables to block jobs from reaching the instance profile.
Runtime roles
Runtime roles need emr-6.7.0 or later for EMR steps (emr-6.9.0 for SageMaker AI Studio connections through Livy, Spark, Hive or Presto) and a security configuration with AuthorizationConfiguration, IAMConfiguration, EnableApplicationScopedIAMRole set to true. AWS recommends enabling in-transit encryption in the same configuration. Setup has three parts:
- Trust: the runtime role's trust policy trusts the instance profile role for sts:AssumeRole and sts:TagSession, and the instance profile is allowed to assume only the listed runtime roles.
- Authorisation: callers get AddJobFlowSteps with an elasticmapreduce:ExecutionRoleArn condition naming the roles they may use, so a data scientist cannot submit a step as the finance pipeline role.
- Attribution: set PropagateSourceIdentity to true under ApplicationScopedIAMRoleConfiguration (with sts:SetSourceIdentity on both roles) so CloudTrail records which caller ran each runtime role session.
Constraints matter for design: runtime roles do not control HDFS or Hive Metastore access, are not supported on high-availability clusters (including clusters using Identity Center trusted identity propagation), and do not support Docker or container workloads. AWS recommends turning off SSH and allowing only the AddJobFlowSteps API to reach runtime role clusters. From EMR 7.5.0 the Spark and YARN web UIs prompt for credentials generated with get-cluster-session-credentials.
Lake Formation fine-grained access
With EMR 6.7.0 and later, Spark, Hive and Presto jobs can be governed by AWS Lake Formation permissions on Glue Data Catalog databases, tables and columns, with row and cell-level filtering. Prerequisites are runtime role authorisation on the cluster, the Glue Data Catalog as metastore and Lake Formation permissions defined. EMR obtains temporary credentials from Lake Formation, reads from S3 and filters results to what the runtime role is permitted. This moves table and column policy out of each cluster and into one governed catalogue. See the Data Lake Security Guide.
Admin roles and privileged access
EMR administration is IAM administration. The actions below change who can run code with which identity, or how exposed a cluster is.
| Capability | Example IAM actions | Why it is privileged |
|---|---|---|
| Create clusters | elasticmapreduce:RunJobFlow plus iam:PassRole on the service role and instance profile | Chooses the instance profile, security configuration, subnet and bootstrap actions |
| Submit work | elasticmapreduce:AddJobFlowSteps | Runs arbitrary code; with runtime roles, choose the identity via ExecutionRoleArn |
| Get UI credentials | elasticmapreduce:GetClusterSessionCredentials | Requires the ExecutionRoleArn condition according to AWS |
| Change exposure | emr put-block-public-access-configuration, ec2:AuthorizeSecurityGroupIngress | Can open cluster ports to the internet |
| Manage security configurations | elasticmapreduce:CreateSecurityConfiguration, DeleteSecurityConfiguration | Define encryption, Kerberos and runtime role settings |
| Interactive access | EC2 key pairs for SSH, EMR Studio access | Shell on the primary node bypasses API-level controls |
- PassRole is the escalation path: anyone who can create a cluster and pass a powerful instance profile, or submit steps with a powerful runtime role, gets that role's access. Scope iam:PassRole to named roles, as AWS shows for EMR Serverless with an iam:PassedToService condition. See the AWS IAM Policies and Roles Guide.
- Tag-based control: AWS notes that identity-based policies can be combined with tagging for cluster-level control, so teams can manage only clusters carrying their tag.
- Bootstrap actions can run as root: they run as the hadoop user by default and can use sudo for root privileges, so treat the S3 locations of bootstrap scripts and custom AMIs as part of the trusted computing base, writable only by the platform team.
Integrations, API tokens and service accounts
Kerberos
Kerberos (EMR 5.10.0 and later) authenticates cluster applications and users to each other. EMR documents these architectures:
| Architecture | Strengths | Considerations |
|---|---|---|
| Cluster-dedicated KDC on the primary node | EMR owns the KDC; independent of central directories; minimal load | Kerberised clusters cannot authenticate to each other; Linux users and HDFS directories created per principal; SSH needs an EC2 key file plus kinit |
| Cluster-dedicated KDC with cross-realm trust (commonly Active Directory) | AD users sign in with kinit, no EC2 key file; EMR creates Linux users for AD principals | Needs an AD account able to join principals; trusts are not transitive; realms must differ |
| External KDC: MIT KDC or another cluster's primary node (EMR 5.20.0 and later) | One place to manage principals; several clusters share a realm | Every node needs a route to the KDC; size the KDC for peak node count; harder troubleshooting |
| External KDC on another cluster with AD cross-realm trust | Principals managed in AD; only one cluster holds AD trust credentials | Every node needs routes to the KDC and domain controllers |
AWS Directory Service for Microsoft Active Directory is not supported as a cross-realm trust or external KDC. Concurrent clusters sharing an external KDC in one realm can conflict on service principal names, so use different realms or non-overlapping subnets. The AD join account used for cross-realm trust is a non-human credential: give it only the right to join principals and store its password in a secrets manager. Protocol background is in the Kerberos Security Guide.
EMR Serverless
EMR Serverless has no clusters to harden, so identity is the main control. Each job run assumes a job runtime role for S3, Glue and other services. AWS warns that the console's "All buckets" option attaches AmazonS3FullAccess and recommends "Specific buckets"; its own sample grants Glue actions on all resources, which should be narrowed to the catalogue, database and table ARNs the job uses. Submitters need iam:PassRole on the specific role with iam:PassedToService set to emr-serverless.amazonaws.com.
EMR Studio
EMR Studio uses either IAM authentication mode (IAM principals, or federation with a SAML 2.0 or OIDC identity provider, with ABAC for access) or IAM Identity Center authentication mode. The mode applies to every user of a Studio. AWS recommends using the organisation management account only to enable Identity Center and provision identities, and creating Studios from a member account. See the AWS IAM Identity Center Guide.
Encryption
| Data | Options in a security configuration | Notes |
|---|---|---|
| EMRFS data in S3 | SSE-S3, SSE-KMS, CSE-KMS, CSE-C (custom provider), with per-bucket overrides | SSE-C is not supported; S3 Bucket Keys change the KMS encryption context to the bucket ARN; Hue S3 File Browser writes bypass EMRFS encryption |
| Local disk | EBS encryption (EMR 5.24.0 and later, KMS) or LUKS; instance store via LUKS or always-on NVMe encryption | EBS encryption covers root and attached volumes and is AWS's recommended option; also enables Hadoop RPC Privacy and HDFS block transfer encryption |
| In transit | EMR-managed certificates, a zipped certificate bundle in S3, or a custom provider | Use TLS 1.2 or higher; EMR 7.0.0 and later mostly enforce it via Java 17; some endpoints need Kerberos for SASL |
Key policy design matters as much as the mode: KMS keys used by EMR must allow the instance profile or runtime roles that read the data. See the AWS KMS Security Guide.
Logging and detection
AWS CloudTrail records EMR API calls; with PropagateSourceIdentity, runtime role sessions carry the caller's source identity. Route trails organisation-wide as described in the AWS CloudTrail Guide.
| Signal | Source | What to look for |
|---|---|---|
| Cluster launches | CloudTrail RunJobFlow | Clusters without a security configuration, in public subnets or with an unexpected instance profile |
| Steps submitted with runtime roles | CloudTrail AddJobFlowSteps, AssumeRole with source identity | Callers using roles outside their team |
| Block public access changes | CloudTrail PutBlockPublicAccessConfiguration | Disabling or new port exceptions |
| Security group violations | AWS Health events from block public access | Rules EMR could not revoke |
| Lake Formation data access | CloudTrail Lake Formation events | Unusual tables or volumes per runtime role |
| Instance profile credential use outside cluster | CloudTrail calls by the instance profile role from unexpected IPs | Stolen instance credentials |
Metrics:
- Share of clusters launched with a security configuration enforcing at-rest and in-transit encryption.
- Share of multi-user clusters using runtime roles or Lake Formation rather than the instance profile alone.
- Number of block public access port exceptions per Region (target zero beyond approved ones).
- Instance profiles with s3:* or wildcard resources, trended down.
- Principals able to pass EMR roles without a resource restriction.
Hardening baseline
- Confirm block public access in every Region, remove unneeded exceptions (including port 22 if you use other access paths) and attach AmazonEMRServicePolicy_v2 to the service role so violations are revoked.
- Launch in private subnets and reach web interfaces through SSH tunnels or authenticated UIs rather than public ports.
- Create standard security configurations per environment: EBS encryption, SSE-KMS for EMRFS, in-transit encryption with TLS 1.2 or higher, and runtime roles enabled. Test on a non-production cluster, then require them through IaC or service control policies.
- Shrink the instance profile to the bootstrap and logging needs of the cluster, and move data access to runtime roles.
- Restrict role use with ExecutionRoleArn conditions and narrow iam:PassRole for cluster creators and EMR Serverless submitters.
- Adopt Lake Formation for table and column permissions where several teams share data.
- Pick a Kerberos architecture where users need interactive Hadoop access, preferring cross-realm trust to centralise identities.
- Turn off SSH on runtime role clusters and remove EC2 key pairs from long-running clusters where not needed.
- Enable source identity propagation and CloudTrail alerting.
Rollback: security configurations apply at cluster creation, so roll back by launching from the previous configuration; transient clusters make this cheap. Keep both configurations available during migration.
Choosing an approach: transient single-pipeline clusters can rely on a scoped instance profile. Shared long-running clusters need runtime roles or Lake Formation. Teams that want no infrastructure to patch should consider EMR Serverless, where the job runtime role is the main control. Interactive Hadoop users with corporate accounts point to Kerberos with AD cross-realm trust or IAM Identity Center.
Common misconfigurations
- One fat instance profile: a shared cluster whose instance profile can read every data bucket gives every user every dataset.
- Runtime roles without ExecutionRoleArn conditions: users can submit steps as any runtime role the instance profile can assume.
- Assuming runtime roles protect HDFS: they govern AWS resource access, not HDFS or Hive Metastore, so on-cluster data needs other controls.
- Block public access exceptions left open: each exception leaves a port exposed to the internet on public subnet clusters.
- Encryption in name only: SSE for S3 without local disk or in-transit encryption leaves shuffle data and node-to-node traffic exposed, and Hue S3 File Browser writes bypass EMRFS encryption.
- EMR Serverless roles with AmazonS3FullAccess: the console's "All buckets" default grants every bucket in the account.
Framework mapping
| EMR control | NIST SP 800-53 Rev 5 | NIST CSF 2.0 | Threat addressed |
|---|---|---|---|
| Runtime roles, ExecutionRoleArn, narrow PassRole | AC-6, AC-3 | PR.AA-05 | ATT&CK T1078.004 Cloud Accounts |
| Instance metadata blocked for runtime role jobs | AC-6 | PR.AA-05 | ATT&CK T1552.005 Cloud Instance Metadata API |
| Lake Formation fine-grained access | AC-3 | PR.AA-05 | ATT&CK T1530 Data from Cloud Storage |
| Security configuration encryption | SC-8, SC-28, SC-13 | PR.DS-01 | Disclosure of data at rest and in transit |
| Block public access, private subnets, SSH off | SC-7, CM-7 | PR.PS-01 | ATT&CK T1190, T1021.004 |
| Kerberos and Identity Center authentication | IA-2, IA-5 | PR.AA-01 | Unauthenticated cluster access |
| CloudTrail with source identity | AU-2, AU-6 | DE.CM-01 | Unattributed data access |
Practitioner checklist
- Verify block public access is enabled in every Region and review its port exceptions.
- Attach AmazonEMRServicePolicy_v2 to the EMR service role.
- Require a security configuration with EBS, EMRFS and in-transit encryption for every cluster.
- Enable runtime roles on shared clusters and keep the instance profile minimal.
- Add elasticmapreduce:ExecutionRoleArn conditions to every AddJobFlowSteps and GetClusterSessionCredentials grant.
- Restrict iam:PassRole for EMR and EMR Serverless roles to named roles and services.
- Use Lake Formation for shared catalogue data and remember HDFS needs separate controls.
- Choose a Kerberos architecture deliberately and vault the AD join credential.
- Disable SSH where steps are submitted only through the API.
- Enable PropagateSourceIdentity and alert on cluster launches without security configurations.
Standards and references
- AWS: Security in Amazon EMR
- AWS: Runtime roles for Amazon EMR steps
- AWS: Using Amazon EMR block public access
- AWS: Kerberos architecture options with Amazon EMR
- AWS: Encryption options for Amazon EMR
- AWS: Integrate Amazon EMR with AWS Lake Formation
- AWS: Job runtime roles for Amazon EMR Serverless
- AWS: Choose an authentication mode for Amazon EMR Studio
Related NHI Mgmt Group resources: Hadoop Security Guide · Apache Spark Security · AWS IAM Policies and Roles Guide · Amazon S3 Security Guide