Splunk Universal Forwarder and Deployment Server Hardening Guide: Securing the Agents That Feed Your SIEM

A Splunk deployment server can push configuration and scripts to every universal forwarder it manages, which makes it one of the most powerful and least watched software deployment systems in many estates. Universal forwarders run on domain controllers, payment servers and cloud hosts, often with broad read access, and they trust whatever the deployment server (now called the agent manager in current documentation) sends them. HTTP Event Collector (HEC) tokens and the shared pass4SymmKey secrets add more machine credentials that few teams inventory. This guide covers the identity model of the Splunk data collection tier, privileged roles, tokens and keys, TLS between components, detection of tampering, and a hardening baseline with exact server.conf and deploymentclient.conf settings. It is for Splunk administrators, SOC engineers and security architects. It complements the Splunk Enterprise Security Guide, which covers detection content and ES itself.

Key takeaways

• The deployment server is a code execution channel. Anyone who can change a server class or app on it can run scripted inputs on every client, so treat it like a software deployment tool (ATT&CK T1072), not a log utility.
• Turn on deployment authentication. Since 8.1.10.1, 8.2.6.1 and 9.0.0, deployment servers and clients can mutually authenticate phone home and bundle downloads; before 9.0, bundles could be downloaded without authentication (CVE-2022-32157).
• Most TLS checks are off by default. sslVerifyServerCert, sslVerifyServerName and requireClientCert all default to false in server.conf, and the certificates shipped with the product must be replaced.
• Run forwarders least privileged. On Linux 9.0 and later the forwarder can run as splunkfwd with only CAP_DAC_READ_SEARCH, which removes root from the most widely deployed Splunk component.
• Tokens and keys are non-human identities. HEC tokens, pass4SymmKey values and the admin credentials on each forwarder need owners, scoping, vault storage and rotation.

How identity and access work in the Splunk collection tier

Splunk components authenticate to each other in several different ways, and each one is a separate trust decision.

The local admin account on each universal forwarder is another identity. Current releases have no default password (an admin must be created at install), but estates built from old images or scripted installs often reuse one password across thousands of forwarders.

Admin roles and privileged access

• Deployment server administrators: whoever can edit serverclass.conf or drop an app into deployment-apps controls code on every client. Limit this to a small group, require MFA through SAML or your identity provider for Splunk Web, and route changes through version control and review. See the Service Account Security Guide for the account model.
• Host administrators on the deployment server: root or local administrator on that host is equivalent to Splunk admin on it. Put the host in your highest administrative tier, alongside other software deployment and configuration management servers.
• Forwarder local admin: use a unique, vaulted credential per forwarder or per build, and set allowRemoteLogin (default requireSetPassword, marked deprecated) so remote splunkd logins are refused where they are not needed.
• Forwarder operating system account: on Linux run as splunkfwd with AmbientCapabilities=CAP_DAC_READ_SEARCH and NoNewPrivileges=yes, set through splunk enable boot-start -systemd-managed 1 -user <user> -group <group>. On Windows choose a low-privilege or virtual account where the inputs allow it, rather than Local System by habit.
• Splunk Cloud Platform: Splunk Cloud does not use customer deployment servers in the cloud tier (the SVD-2022-0607 advisory states it was not affected), but customers still run deployment servers on premises to manage their forwarders, so the controls here still apply.

Integrations, API tokens and service accounts

Store all of these in a secrets manager with an owner and rotation date. See the Secrets Management Guide. When rotating a HEC token, create the new token, move the client, confirm data flows, then disable the old one.

Deployment server and client controls

1. Upgrade first. Deployment servers and all managed forwarders must be at 9.0 or later (or 8.1.10.1 and 8.2.6.1 on older lines) before mutual authentication can be enabled. Once enabled, older clients stop being managed.
2. Install non-default certificates on the deployment server and clients. The authentication procedure requires that they are not the certificates shipped with Splunk.
3. Set pass4SymmKey in the [deployment] stanza of server.conf on the server and every client.
4. Require authentication in $SPLUNK_HOME/etc/system/local/restmap.conf on the deployment server: requireAuthentication = true with authKeyStanza pointing at the stanza that holds the key, under [broker:broker] and [streams:agent]. This can only be done by editing the file, and needs a restart, not a reload.
5. Pin where content can land. In deploymentclient.conf set serverRepositoryLocationPolicy = rejectAlways and serverEndpointPolicy = rejectAlways so the server cannot redirect installs or downloads to arbitrary paths or endpoints. Keep changes in etc/system/local, never default.
6. Verify the client trusts the right server. Set sslVerifyServerCert = true and sslVerifyServerName = true in the client's [sslConfig]; sslVerifyServerName only works when sslVerifyServerCert is on.
7. Test in a pilot server class. Move a small group of forwarders first and watch client splunkd.log for errors such as "Received message that did not originate from deployment server, ignoring", which shows the client is rejecting unauthenticated messages.
8. Rollback: remove the restmap.conf settings and restart the deployment server to return to unauthenticated operation while you fix clients. Keep the previous configuration in version control.

Do not use the deployment server for indexer cluster peers (use the cluster manager and configuration bundles) or search head cluster members (use the deployer). Hierarchical deployment servers (reloadDSOnAppInstall) are discouraged in the documentation.

Logging and detection

Splunk records its own activity in internal indexes (commonly _internal for component logs such as splunkd.log and _audit for user and configuration actions). Forward and protect these like any other security log, and alert on the events below.

Because the SIEM watches itself here, also send these alerts, or at least deployment server file integrity events, to a second destination that a Splunk administrator cannot silence. See the SIEM Guide.

Testing the controls

• Unauthenticated bundle download: from a host that is not an enrolled client, confirm that requests to the deployment server for a bundle are refused once authentication is on.
• Rogue server: point a test forwarder at a deployment server with a certificate from another issuer and confirm the handshake fails with verification enabled.
• Token scope: send a test event with each HEC token to an index outside its allowed list and confirm it is rejected.
• Silence alert: stop a pilot forwarder and time how long it takes for the missing data alert to reach the SOC queue.

Run these after every Splunk upgrade, because upgrades can reset local settings or change defaults.

Metrics:

• Share of forwarders at a version that supports deployment authentication, and share actually enrolled with it enabled.
• Share of Linux forwarders running as a non-root user.
• Number of HEC tokens with a named owner and a single allowed index, as a share of all tokens.
• Age of the oldest pass4SymmKey and forwarder admin credential.
• Mean time to detect a silent critical forwarder.
• Changes to deployment apps that went through review, as a share of all changes (target: 100 percent).

Hardening baseline

• Patch: track Splunk security advisories; CVE-2026-20253, a missing authentication flaw in a Splunk Enterprise PostgreSQL sidecar service endpoint, was added to the CISA KEV catalogue on 18 June 2026. Put Splunk Enterprise hosts in the emergency patch path.
• Management port: set acceptFrom in [httpServer] (default *) to admin and Splunk networks only. Do not set disableDefaultPort = true on components that need management; the documentation advises against it. Keep enableSplunkdSSL = true (the default).
• TLS: replace shipped certificates on all instances; turn on sslVerifyServerCert and sslVerifyServerName; restrict sslVersions to TLS 1.2 or later; on indexers receiving forwarder data, require client certificates so only enrolled forwarders can send.
• HEC: enable SSL, one token per source, single allowed index, no tokens with access to security result indexes.
• Forwarder identity: least-privileged service account, unique admin credential, no unused inputs or apps.
• Deployment server: mutual authentication on, client policies set to rejectAlways, changes through code review, host in the top administrative tier.
• Audit: forward _audit and deployment server integrity events to a protected destination.

Small versus large estates: a small estate with a few dozen forwarders can manage configuration with existing tools such as Ansible, Puppet or Salt (the documentation lists these as alternatives) and avoid running a deployment server at all. Large estates gain from the deployment server's scale but must invest in authentication, change control and monitoring to match.

Framework mapping

Common misconfigurations

• Deployment server reachable from everywhere, unauthenticated. Before 9.0 any host could download forwarder bundles (CVE-2022-32157, CVSS 7.5), and bundles often carried passwords and certificates.
• Default certificates and no verification. With sslVerifyServerCert = false, a forwarder will talk to anything presenting a certificate, so traffic interception or a rogue deployment server goes unnoticed.
• Forwarders running as root or Local System everywhere. A malicious scripted input then runs with full privileges on every host.
• One pass4SymmKey for every service and every environment. A key taken from a test instance opens production clustering and deployment.
• Secrets in deployment apps. API keys for scripted inputs copied to thousands of hosts, readable by anyone with local access.
• No alert when collection stops. Disabling a forwarder is quieter than deleting logs and often goes unnoticed for weeks. See the EDR Tampering and BYOVD Response Playbook for a parallel response pattern.

Practitioner checklist

• Inventory deployment servers, forwarders by version and run-as account, HEC tokens and pass4SymmKey values.
• Upgrade the deployment server and forwarders to versions that support deployment authentication, then enable it.
• Set serverRepositoryLocationPolicy and serverEndpointPolicy to rejectAlways on clients.
• Replace shipped certificates and turn on server certificate and name verification.
• Restrict port 8089 with acceptFrom and firewalls to admin and Splunk networks.
• Run Linux forwarders as splunkfwd with only CAP_DAC_READ_SEARCH.
• Issue one HEC token per source with a single allowed index, SSL on and an owner.
• Use distinct, random pass4SymmKey values per service type and environment, stored in a vault.
• Put deployment apps under version control and review, with file integrity alerts on the server.
• Alert on silent forwarders, token changes and deployment changes to a destination Splunk admins cannot silence.

Standards and references

• Splunk: Configure communication and bundle download authentication for agents and agent management
• Splunk: Secure Splunk Enterprise services with pass4SymmKey
• Splunk: Manage a Linux least-privileged user (Universal Forwarder Manual)
• Splunk: deploymentclient.conf specification
• Splunk: server.conf specification
• Splunk: Set up and use HTTP Event Collector
• Splunk: SVD-2022-0607 Deployment server forwarder bundle download (CVE-2022-32157)
• MITRE ATT&CK: Software Deployment Tools

Related NHI Mgmt Group resources: Splunk Enterprise Security Guide · SIEM Guide · Secrets Management Guide · Service Account Security Guide