Join our Newsletter — 33% off our NHI Course
Home› Guides› Ray and AI Compute Cluster Security Guide: Lock…
Guide AI Platforms

Ray and AI Compute Cluster Security Guide: Lock down dashboards, job APIs and cluster credentials

PDF for newsletter subscribers · free to read online

← All guides
By NHI Mgmt Group, edited by Lalit Choda Updated 8 November 2026 15 min read
On this page

Anyone who can reach a Ray cluster's dashboard, Jobs API or Ray Client port can run arbitrary code on every node, and with it read every credential the cluster holds. Ray was built as a developer tool for trusted networks, so keeping it off untrusted networks and turning on its token authentication are customer-side controls, especially on GPU clusters full of cloud keys, model hub tokens and AI API keys. This guide covers how access works in Ray and KubeRay, the token authentication Ray now ships, the credentials and service accounts that make a compromised cluster expensive, what to log, and a hardening baseline. It is written for platform engineers, ML infrastructure teams and security engineers who run or review Ray, and applies by analogy to other distributed AI compute schedulers.

This is an independent guide from NHI Mgmt Group, based on the Ray project's public documentation as of November 2026. It is not sponsored or reviewed by Anyscale. Product features change, so confirm settings against the Ray project's current documentation.

Key takeaways

  • Reachability equals code execution. Ray's own security documentation states that any client that can reach the dashboard, Jobs or Client ports can run arbitrary code, explicitly through jobs, indirectly through dashboard REST APIs, or implicitly through cloudpickle deserialisation.
  • The network is the primary control. Ray expects isolation to be enforced outside the cluster: private networks, security groups, NetworkPolicies and authenticating proxies come first.
  • Token authentication is defence in depth, not a perimeter. Available from Ray 2.52.0 and on by default for all clusters from Ray 2.61, the token is a shared, non-expiring secret sent as an HTTP header, so it needs TLS and careful storage.
  • Ray has no internal isolation between users or jobs. Separate teams, tenants and trust levels need separate clusters, each with its own narrowly scoped workload identity.
  • The blast radius is the credentials on the nodes. Nodes can hold cloud keys, AI API, model hub and SaaS tokens, SSH keys and Kubernetes secrets. Short-lived workload identity and scoped API keys limit what an attacker can take.

How identity and access work in Ray

A Ray cluster has a head node and worker nodes. Core components communicate over gRPC for distributed memory, heartbeats, metrics and logs. Three additional services sit on top, and Ray's documentation is explicit that each gives complete access to the cluster and its compute:

  • Ray dashboard: cluster introspection and debugging, served over HTTP on port 8265 by default. Its REST APIs can trigger code execution.
  • Ray Jobs: runs alongside the dashboard and accepts job submissions (the /api/jobs/ endpoints). A job is an entrypoint command plus a runtime environment, so a submitted job is a shell command on the cluster.
  • Ray Client: interactive development against a remote cluster (port 10001 by default). Objects cross the wire with cloudpickle, so Python's pickle trust model applies: deserialising untrusted data is code execution.

Turn on token authentication: without it, these services accept any caller that reaches them. Users of one cluster share the same access and jobs run side by side, so use separate clusters where teams or jobs need isolation. In identity terms, every caller who reaches a port is effectively root on the cluster and inherits the cluster's workload identity: the cloud instance role, the Kubernetes service account and any secrets in environment variables or mounted files.

Why scanners are not enough

Anyscale, Ray's maintainer, states that Ray must only be reachable from trusted machines. Many scanners and software composition tools do not report an exposed Ray cluster, so do not rely on vulnerability scanning to find one; check exposure directly, and apply Ray updates listed on the Ray security page promptly.

Token authentication

Setting or behaviourWhat it doesSecurity note
RAY_AUTH_MODE=tokenEnables token authentication on nodes and clientsAvailable from Ray 2.52.0; default for local clusters in 2.59 and all clusters in 2.61
RAY_AUTH_MODE=disabledOpts out of authenticationTreat as a policy exception; alert if seen in cluster configs or images
RAY_AUTH_TOKENSupplies the token value directlyDiscouraged in the docs: any code in the process environment can read it
RAY_AUTH_TOKEN_PATHPoints to a token filePreferred; mount from a secret store with restrictive file permissions
~/.ray/auth_tokenDefault token location, plaintextProtect on shared machines; never commit to version control
ray get-auth-token --generatePrints or creates the tokenGeneration is a privileged action: run it in the deployment pipeline, not on laptops
Dashboard sign-inToken entered once and stored as a cookie for up to 30 daysLong-lived browser credential; shared or kiosk browsers must not be used
TransportToken sent as an HTTP headerOver plain HTTP it can be intercepted; use a TLS proxy, SSH tunnel, kubectl port-forward or VPN
Lifetime and rotationTokens do not expire; one token per cluster lifetimeRotation means restarting the cluster with a new token and updating every client

The token check is the lookup order RAY_AUTH_TOKEN, then the file at RAY_AUTH_TOKEN_PATH, then ~/.ray/auth_token. A stale environment variable therefore silently overrides a rotated token file, which is a common cause of confusing failures after rotation. Note also what token mode is not: it is a single shared secret, so it proves "holds the cluster token", not which person or service submitted a job. Attribution still needs an authenticating proxy or the Kubernetes RBAC option that KubeRay documents.

Admin roles and privileged access

Ray has no internal role model, so "admin" is defined by who controls the layers around it. Map these explicitly:

LayerPrivileged actionWho should hold it
Cluster launcher (ray up) or KubeRay RayCluster / RayJob resourcesCreate clusters, choose images, mount secrets, set the service accountPlatform team through CI/CD; developers through a reviewed template
KubeRay operatorReconciles Ray resources cluster-wideOperator service account scoped to its namespaces, not cluster-admin
Ray auth tokenFull code execution on that clusterOnly the users and pipelines of that cluster; stored in a secrets manager
Cloud IAM role on nodesWhatever the role allows: storage, model registries, other servicesScoped per cluster to the buckets and APIs the workload needs
Network path to 8265 and 10001Reach the dashboard, Jobs and ClientIdentity-aware proxy or bastion; never a public load balancer
  • Do not run Ray as root: running as root turns code execution into host compromise. Use a non-root container user and drop Linux capabilities.
  • Cluster-admin is a red flag: an operator with administrator rights widens any compromise. Give the Ray pods' service account no Kubernetes API permissions unless autoscaling or a specific integration needs them. See the Kubernetes RBAC Guide.
  • Human access through an identity-aware proxy: put dashboard access behind SSO with MFA so each session is attributable. See the Identity-Aware Proxy Guide.

Integrations, API tokens and service accounts

A Ray cluster is a concentration point for non-human identities. Training and batch inference jobs pull data from object storage, models from hubs, and call external AI APIs, so the nodes collect credentials. Plan against the full inventory below.

Credential on the clusterTypical locationBetter pattern
Cloud access keys (AWS, Google Cloud, Azure, GPU clouds)Environment variables, ~/.aws/credentials, job runtime_envInstance role or Kubernetes workload identity with short-lived tokens, scoped per cluster
Instance metadata credentialsIMDS endpoint reachable from every podIMDSv2 with hop limit, block metadata from worker pods that do not need it
Hugging Face tokensEnvironment, ~/.cache/huggingfaceRead-only, fine-grained tokens; write tokens only in a separate publishing pipeline
OpenAI and other AI API keysEnvironment variables in job specsProject-scoped keys with spend limits, injected per job from a secrets manager
Database and SaaS tokens (Stripe, Slack)Config files, job argumentsKeep out of compute clusters; call through a service that holds the credential
SSH keysHead node home directories (cluster launcher)Short-lived SSH certificates or no SSH; use port-forwarding through the control plane
Kubernetes service account tokenMounted in every pod by defaultDisable automounting where not needed; bound, audience-scoped tokens elsewhere
  • Model hub write access: stolen Hugging Face write tokens could let an attacker replace an organisation's models with malicious versions. A cluster that only needs to download models should never hold a write token.

For the wider pattern of replacing static keys on GPU and training infrastructure with federated workload identity, see the AI Infrastructure Workload Identity Guide and the Kubernetes Service Account Token Guide.

Logging and detection

Detection for Ray depends on runtime telemetry and on the identity systems around the cluster. The dashboard's own job history records submitted commands, so review it.

SignalSourceWhat suspicious looks like
Job submissionsRay Jobs history, dashboard logs, proxy access logsSubmissions from unknown source IPs, entrypoints with base64 strings, curl or wget piped to a shell, reads of /etc/shadow
Requests to /api/jobs/ without a proxy identityLoad balancer or proxy logsAny direct request bypassing the authenticating proxy
Outbound connections from nodesVPC flow logs, DNS logs, egress proxyMining pools, tunnelling services, callback domains used for exploit confirmation, new external IPs
GPU and CPU utilisationCloud monitoring, DCGM or node exporterSustained full utilisation with no scheduled job, processes with masqueraded names
PersistenceHost and container runtime telemetryNew cron entries, unexpected processes on every node at once (a sign of node affinity scheduling abuse)
Credential useCloudTrail, Google Cloud audit logs, Azure activity log, AI provider usage dashboardsNode role credentials used from outside the cluster's network, AI API spend spikes, Hugging Face token use from new locations
Kubernetes API activityKubernetes audit logRay pod service accounts listing secrets or creating pods outside the operator's pattern

Ingest Ray job history and proxy logs into the SIEM rather than leaving them on the head node, where an attacker with code execution can delete them. See the Kubernetes Audit Log Detection Guide.

Metrics that show the controls are working

  • Externally reachable Ray ports: count of clusters with 8265, 10001 or the GCS port reachable from outside the trusted network. Target zero, measured by an external scan.
  • Clusters with authentication enabled: percentage of clusters running with RAY_AUTH_MODE=token and behind an authenticating proxy. Target 100 per cent.
  • Static credentials on nodes: number of long-lived cloud keys or AI API keys found in cluster images, environment variables and job specs. Trend to zero.
  • Attributable job submissions: percentage of jobs whose submitter maps to a named person or pipeline identity.
  • Token age: days since each cluster's token was rotated, with a ceiling set by policy and a forced rotation on staff departure.

Hardening baseline

Roll the baseline out in this order. Each step reduces exposure on its own, so do not wait for the full set.

  1. Find every cluster. Inventory Ray clusters across cloud accounts and Kubernetes namespaces (KubeRay RayCluster, RayJob and RayService resources, plus VMs started with ray start). Scan your external ranges for 8265 and 10001. Anyscale publishes a Ray open ports checker for this purpose.
  2. Remove internet exposure. Bind the dashboard to a private interface rather than 0.0.0.0, delete public load balancers and Ingress objects for the dashboard, and restrict security groups to the head node's private network. Copied examples and boilerplate can expose the dashboard on all interfaces: review copied templates.
  3. Treat exposed clusters as compromised. If a cluster was reachable without authentication, rotate every credential it held, review job history and cloud audit logs, and rebuild nodes from clean images. Use the Cryptojacking Incident Response Playbook for the response.
  4. Put humans behind an authenticating proxy. Access to the dashboard through an identity-aware proxy with SSO and MFA, or through ray dashboard SSH forwarding or kubectl port-forward for administrators.
  5. Enable token authentication. Upgrade to a Ray version that supports it, set RAY_AUTH_MODE=token, distribute the token with RAY_AUTH_TOKEN_PATH from a secret, and test job submission and Ray Client from each pipeline before enforcing. Roll back by reverting the cluster template, not by setting disabled in production.
  6. Encrypt in transit. Terminate TLS at the proxy for dashboard and Jobs traffic and enable TLS for gRPC where the KubeRay TLS guidance fits your environment. Ray's documentation is clear that TLS does not replace network isolation.
  7. Isolate with NetworkPolicies and egress control. Allow only intra-cluster traffic between Ray pods, ingress to 8265 and 10001 from the proxy only, and egress to named destinations (object storage, model hub, AI APIs) through an egress proxy. See the Agent Egress Control and Network Isolation Guide.
  8. Replace static secrets. Move to workload identity for cloud access, block or harden instance metadata, scope Hugging Face and AI API tokens to read-only and per project, and disable service account token automounting where unused.
  9. Harden the runtime. Non-root user, read-only root filesystem where practical, no privileged containers, signed and pinned images, and a dependency policy for runtime_env packages.
  10. Separate clusters by tenant and trust. One cluster per team or sensitivity level, ephemeral clusters for untrusted experiments, each with its own token and role. For running model-generated or user-supplied code, see the AI Agent Sandboxing and Code Execution Guide.

Test each change in a non-production cluster first: submit a known job through every path (CLI, SDK, Ray Client, any orchestration tool such as Airflow or a notebook), confirm an unauthenticated request is refused, and only then apply the template to production clusters.

Choosing an access pattern

PatternFitsTrade-off
Private network plus SSH or kubectl port-forwardSmall teams, a handful of clustersSimple and strong; attribution relies on SSH or Kubernetes identity, awkward for many users
Identity-aware proxy plus token authShared platforms with many developersPer-user attribution and MFA; the proxy must hold or inject the cluster token, so it becomes a privileged component
Kubernetes RBAC authentication (KubeRay)Organisations already governing access through KubernetesReuses existing RBAC and audit; depends on mature Kubernetes identity governance
Managed Ray serviceTeams that do not want to operate the control planeProvider handles dashboard exposure and auth; you still own node credentials and network egress

Small organisations get most of the benefit from the first row plus token authentication. Large platforms with many tenants need the second or third row, because a shared token alone cannot tell you who ran a job.

Common misconfigurations

  • Dashboard on a public IP: anyone who reaches it can submit jobs. Keep it on a private network behind authentication.
  • Relying on the scanner: a clean vulnerability report says nothing about exposure. Check reachability, not CVE lists.
  • Cloud keys in environment variables: the first thing a submitted job can print.
  • One shared cluster for everyone: Ray does not isolate jobs, so a single compromised notebook or dependency exposes every team's data and credentials.
  • Ray pods with cluster-admin or broad Kubernetes permissions: turns a Ray compromise into a Kubernetes compromise. See the Kubernetes Cluster Compromise Playbook.
  • Unrestricted egress: lets miners reach pools, reverse shells reach their operators and stolen data leave. Default-deny egress makes most post-exploitation steps fail.

Framework mapping

Risk in RayThreat referenceControls
Unauthenticated Jobs API on a reachable dashboardATT&CK T1190 Exploit Public-Facing Application; ATLAS AML.T0049NIST SP 800-53 SC-7, AC-3, CM-7; CSF 2.0 PR.IR-01; ISO/IEC 27001:2022 A.8.20, A.8.22; CIS Controls v8.1 Control 12
Arbitrary Python execution through jobs or Ray ClientATT&CK T1059.006 PythonNIST SP 800-53 AC-6, CM-7; ISO/IEC 27001:2022 A.8.5
Secrets in files, environment and metadataATT&CK T1552.001, T1552.005; ATLAS AML.T0055 Unsecured CredentialsNIST SP 800-53 IA-5; CSF 2.0 PR.AA-05; CIS Controls v8.1 Control 3 and 6
Stolen cloud credentials used elsewhereATT&CK T1078.004 Cloud AccountsNIST SP 800-53 AC-6, SI-4; CSF 2.0 DE.CM-01
GPU cryptomining and AI spend abuseATT&CK T1496.001 Compute Hijacking; ATLAS AML.T0034 Cost HarvestingNIST SP 800-53 SI-4, AU-2; ISO/IEC 27001:2022 A.8.16; CIS Controls v8.1 Control 13
Token sent in clear over HTTPInterception on the network pathNIST SP 800-53 SC-8

Practitioner checklist

  • Inventory every Ray cluster, including KubeRay resources and VM clusters, and scan external ranges for ports 8265 and 10001.
  • Remove all public exposure of the dashboard, Jobs API and Ray Client; bind to private interfaces only.
  • Treat any cluster that was reachable without authentication as compromised: rotate its credentials and review job history.
  • Enable RAY_AUTH_MODE=token, distribute the token by file from a secrets manager, and give each cluster its own token.
  • Put human access behind an identity-aware proxy with SSO and MFA, terminating TLS.
  • Replace static cloud keys with workload identity, harden instance metadata, and scope model hub and AI API tokens to read-only and per project.
  • Run Ray as non-root, remove Kubernetes API permissions from Ray pods, and apply NetworkPolicies with default-deny egress.
  • Use separate clusters for separate teams and trust levels; never mix untrusted experiments with production credentials.
  • Ship job history, proxy logs, flow logs and cloud audit logs to the SIEM and alert on unattributed job submissions and GPU use without scheduled jobs.
  • Review cluster templates and Helm values for 0.0.0.0 bindings and RAY_AUTH_MODE=disabled before they are reused.

Standards and references

Related NHI Mgmt Group resources: AI Infrastructure Workload Identity Guide · MLflow and Kubeflow Security Guide · vLLM, Triton and NVIDIA NIM Inference Server Security Guide · Kubernetes NHI Security Guide · LLMjacking Guide

PDF for subscribers

Take this guide with you

Every section, table and checklist in one PDF, with links to the sources and related guides. Subscribe to our weekly NHI & AI Identity Journal and we’ll email you a download link.

Free weekly newsletter

Subscribe to the NHI & AI Identity Journal

The latest on NHI and Agentic AI security – articles, research, breaches, news and events every week.

Bonus 33% off our NHI Course when you subscribe.

NHIMG Editorial Note
Researched and written by the NHI Mgmt Group editorial team using AI tools, under the editorial direction of Lalit Choda. Facts are drawn from the sources listed on this page and checked by our automated quality process. Last updated 8 November 2026. Spotted an error? Use our contact page.
NHI Mgmt Group, the independent authority on Non-Human Identity, IAM and Agentic AI security. nhimg.org

PDF Downloads for Newsletter Subscribers

Get the guide

Enter your details and we’ll email you a download link.

The full guide is free to read online. The PDF is a thank you for subscribing.

Fill in all fields marked * and tick the PDF box to continue.