Data Centre Network Fabric Security: Securing VXLAN, EVPN and Spine Leaf Designs

A VXLAN EVPN fabric will forward any well-formed packet that reaches a tunnel endpoint, because neither the encapsulation nor the control plane authenticates anything on its own. Spine leaf fabrics with VXLAN overlays and a BGP EVPN control plane are now the default design for new data centres and private clouds, and they replace the old perimeter of VLANs and a core firewall with thousands of tunnels, a routing protocol that carries every MAC address and a set of route targets that decide which tenant sees which segment. This guide covers the threat model of the underlay, the overlay and the control plane, how to harden each, how to segment tenants and enforce policy inside the fabric, and how to monitor it. It is written for network security engineers, data centre architects and the platform teams who run the fabric. Physical facility controls are covered separately in the Data Centre Security Guide.

Key takeaways

• VXLAN has no security of its own. RFC 7348 states that it relies on existing IP mechanisms and warns that rogue endpoints can inject MAC-over-UDP frames or join the multicast groups that carry broadcast traffic.
• The underlay is the trust boundary. Only known VTEP loopbacks should be able to send or receive UDP 4789 inside the fabric, and the underlay address space must never be reachable from tenant networks or the internet.
• BGP EVPN is the fabric's brain, so treat it as critical infrastructure. Protect sessions with neighbour ACLs on TCP 179, GTSM on directly connected peerings, TCP-AO or MD5, and prefix limits, following BCP 194 (RFC 7454).
• Route targets and VRFs are your segmentation policy. A wrong route target import leaks one tenant into another as surely as a firewall rule of "any any", so manage them as code and review them like firewall changes.
• The overlay provides separation, not confidentiality. RFC 9135 is explicit that the EVPN scheme gives no confidentiality, source authentication, integrity or replay protection; use MACsec or IPsec where traffic leaves trusted links.

How a VXLAN EVPN fabric works, and where it is weak

A spine leaf fabric is a two tier Clos network: every leaf connects to every spine, servers attach to leaves, and the IP underlay uses equal cost multipath routing. VXLAN builds Layer 2 segments on top of that routed underlay. The terms that matter for security are:

• VTEP (VXLAN Tunnel End Point): the device that encapsulates and decapsulates traffic. RFC 7348 notes that it can run in a hypervisor, on a physical switch or on a server, in software or hardware. Every VTEP is a point where an attacker could inject traffic into any segment it serves.
• VNI (VXLAN Network Identifier): a 24-bit segment identifier in an 8-byte header, giving about 16 million segments against the roughly 4,000 usable VLAN IDs. Traffic is carried in UDP with destination port 4789 (IANA assigned) and an outer source port hashed from the inner frame for ECMP entropy.
• Flood and learn: the original RFC 7348 model learns remote MAC addresses from the data plane and sends broadcast, unknown unicast and multicast (BUM) traffic to an IP multicast group per VNI. Anyone who can send to the right group and port can be learned as a legitimate endpoint.
• BGP EVPN control plane: RFC 8365 (Standards Track, 2018) specifies EVPN as the control plane for VXLAN. MAC and IP reachability is advertised in BGP instead of learned from floods. Route type 2 carries MAC/IP advertisements, type 3 (inclusive multicast Ethernet tag) sets up BUM replication, types 1 and 4 handle multihoming and designated forwarder election (RFC 7432), and type 5 carries IP prefixes for inter-subnet routing (RFC 9136, used by RFC 9135).
• VRFs and route targets: each tenant is a Layer 3 VRF with one or more Layer 2 VNIs. Route targets decide which leaves import which routes, so they are the effective access control list of the overlay.

The security consequence is that the fabric has three planes to attack: the underlay IP network, the overlay data plane on UDP 4789, and the BGP control plane. A weakness in any one breaks tenant separation.

Threat model for the fabric

Hardening the underlay

The underlay should be a small, closed routed network whose only job is to carry VTEP to VTEP traffic and the control plane. Most fabric compromises that matter start with the underlay being reachable from somewhere it should not be.

1. Use a dedicated address block for loopbacks and point to point links. Do not advertise it into tenant VRFs, the campus or the internet. RFC 7348 recommends designating a dedicated VLAN for VXLAN traffic; in a routed fabric the equivalent is a dedicated underlay VRF or the default table with no tenant routes in it.
2. Filter UDP 4789 by source and destination. On leaf uplinks and border leaves, permit VXLAN only between known VTEP loopback addresses. RFC 7348 notes that the UDP encapsulation lets physical switches apply 5-tuple ACLs, so use them. Drop UDP 4789 arriving on any tenant facing or external interface.
3. Apply anti-spoofing at the edge. RFC 7454 recommends blocking packets sourced from your own address space at all edges (BCP 38 and RFC 3704 style filtering), which also protects internal BGP.
4. Prefer ingress replication for BUM traffic, or lock down multicast. RFC 8365 allows ingress replication, PIM-SSM, PIM-SM and BIDIR-PIM tunnel types. Ingress replication removes the multicast group as an attack surface at the cost of replication load on the ingress leaf. If you use multicast, restrict which sources and groups the rendezvous points accept.
5. Size the MTU properly. RFC 7348 states that VTEPs must not fragment VXLAN packets. A fabric that silently drops large frames tempts operators to disable checks or open paths to work around it, and RFC 5082 notes that fragmentation also weakens GTSM classification.
6. Encrypt links that leave a trusted room. MACsec (IEEE 802.1AE) on spine leaf and interconnect links protects against taps; IPsec, which RFC 7348 names as the way to authenticate and optionally encrypt VXLAN, suits traffic crossing shared or third-party transport.

Securing the BGP EVPN control plane

RFC 8365's security section points back to the BGP security considerations in RFC 4271 and RFC 4272 and warns that changes to the information used to build encapsulation headers can misroute or misdeliver user packets. In practice that means the EVPN sessions between leaves and spine route reflectors deserve the same protection as internet edge peerings.

MAC mobility and duplicate detection: RFC 7432 adds a MAC Mobility extended community with a sequence number, so receivers pick the latest advertisement when a host moves, and a sticky or static flag marking addresses that must never move. Use the static flag for anycast gateway and other infrastructure MACs, and enable the platform's duplicate MAC and duplicate IP detection so that an address flapping between leaves is frozen and alerted rather than silently redirected. Check platform defaults for the move threshold and alert on every trigger.

ARP and neighbour suppression: most EVPN implementations answer ARP and IPv6 neighbour discovery from the control plane table at the leaf. This narrows the window for ARP cache poisoning (ATT&CK T1557.002) only if host learning is protected at the access port, for example with DHCP snooping and dynamic ARP inspection.

Tenant segmentation and policy inside the fabric

Segmentation in a fabric happens at several layers, and each answers a different question. Decide which layer enforces which rule before building, or every exception becomes a route leak.

• Symmetric versus asymmetric IRB: RFC 9135 explains that asymmetric IRB requires every ingress leaf to hold ARP entries and interfaces for all subnets in the VRF, while symmetric IRB keeps them local and routes between VRF instances using type 5 routes. Symmetric IRB limits what each leaf knows, which is a modest security benefit as well as a scaling one.
• Route target hygiene: RFC 8365 allows route targets to be auto-derived from the VNI only with 2-octet AS numbers; with 4-octet AS numbers they must be configured manually. Manual route targets are where tenant leaks are typed in, so generate them from a source of truth and test imports before deployment.
• Shared services: DNS, NTP, backup and monitoring often justify a shared services VRF. Leak only the specific prefixes needed, in one direction where possible, and log the flows through a firewall rather than relying on route leaking alone.
• Virtualised hosts: NIST SP 800-125B recommends overlay networking for large data centres (VM-NS-R4), with overlay traffic carried in a way that keeps segmentation intact, and SDN controllers using standard protocols (VM-NS-R5). It also recommends a separate virtual switch, NICs and encrypted traffic for the hypervisor management network (VM-NS-R2).

For the strategy of choosing zones, see the Network Segmentation Guide and the Microsegmentation Buyers Guide. Payment environments built on a fabric must also meet the scoping rules in the PCI DSS Scoping and Network Segmentation Guide.

Where identity fits: the management plane and its machine credentials

Modern fabrics are run by controllers and automation pipelines, not by engineers typing into switches. That moves the real risk to the identities those tools use.

• Controller and automation accounts: a fabric controller or an automation service account with write access can change route targets on every leaf at once. Give each tool its own account, scope it to the devices and operations it needs, and record every change against that identity. See the Non-Human Identity (NHI) Guide.
• Device credentials and keys: local admin passwords, SNMPv3 credentials, TCP-AO or MD5 keys, API tokens and certificates for gNMI or RESTCONF are all secrets. Keep them in a vault, rotate them, and never embed them in templates committed to a repository.
• Human access: use AAA with TACACS+ or RADIUS, MFA, and privileged access workflows for console and controller access, from an out-of-band management network. ATT&CK T1021 Remote Services is the usual path from a compromised workstation to network devices.
• Device identity: NIST SP 800-53 IA-3 (Device Identification and Authentication) applies to switches joining the fabric. Zero touch provisioning should verify the device and its image before it receives configuration and keys.

Implementation: a phased approach

1. Inventory and baseline (weeks 1 to 4): list every VTEP, its loopback, every BGP session and its authentication state, every VRF and its route targets, and every interface that accepts UDP 4789. Record which management identities can change the fabric.
2. Close the underlay: deploy UDP 4789 and TCP 179 ACLs first in log-only or counting mode on border and service leaves, review hits for two weeks, then enforce. Roll back by removing the deny entry, keeping the logging entry.
3. Protect BGP: enable GTSM and session authentication one peer group at a time in a maintenance window. Both ends must match, so pair changes and keep the previous configuration staged for immediate rollback. Add prefix limits in warning mode before teardown mode.
4. Harden host learning: enable duplicate MAC and IP detection, static flags for gateway MACs, ARP suppression and access port protections, first in a pilot pod.
5. Codify segmentation: move VRFs, VNIs and route targets into a source of truth and generate configuration from it. Add pre-deployment checks that fail if a route target is imported into an unexpected VRF.
6. Encrypt where needed: enable MACsec on interconnect and any link leaving a controlled space, starting with one link pair and verifying throughput and failover.
7. Monitor and test: stream telemetry and flow logs into the SOC, then test with an authorised exercise that tries to send VXLAN from a tenant host and to peer an unauthorised BGP speaker.

Small versus large estates: a single small fabric run by one team can rely on ingress replication, a handful of route reflectors and manual review of a short route target list. Large multi-site fabrics with data centre interconnect need a source of truth, automated intent verification, MACsec or IPsec between sites and a dedicated network detection capability.

Detection and measurement

NIST SP 800-125B recommends monitoring both incoming and outgoing VM traffic (VM-TM-R1), enabling promiscuous mode only on the required port group (VM-TM-R2) and using port mirroring to more capable physical tools (VM-TM-R3). At the fabric level, monitor:

• BGP events: session flaps, authentication failures, GTSM drops, prefix limit warnings and unexpected new peers.
• EVPN events: duplicate MAC or IP detections, MAC moves at unusual rates, and new type 5 prefixes in a tenant VRF.
• ACL counters: hits on the UDP 4789 and TCP 179 deny entries, which should be zero on a clean fabric.
• Configuration drift: differences between running configuration and the source of truth, especially route targets and VRF definitions.
• Flow telemetry: inter-VRF flows through firewalls, and east-west flows inside sensitive VNIs.

Useful metrics:

• Percentage of BGP sessions with authentication and GTSM (target 100% of underlay peerings).
• Number of interfaces accepting UDP 4789 from non-VTEP sources (target zero).
• Route target changes deployed without peer review (target zero).
• Mean time to investigate a duplicate MAC alert.
• Percentage of fabric devices whose running configuration matches the source of truth.
• Percentage of fabric management identities that are individual and MFA protected or vaulted.

Feed these into the SOC; see the NDR Guide.

Framework mapping

In CIS Controls v8.1 these sit under Control 12 (Network Infrastructure Management) and Control 13 (Network Monitoring and Defense). In ISO/IEC 27001:2022 Annex A the closest controls are 8.20 (Networks security), 8.21 (Security of network services) and 8.22 (Segregation of networks).

Common failure patterns

• Underlay reachable from tenants: a border leaf leaks loopback routes into a tenant VRF or the campus, and anything there can now send VXLAN to every VTEP.
• Treating the VNI as a security boundary on its own: the VNI is a number in a UDP payload. Without underlay filtering it separates honest hosts, not attackers.
• Copied route targets: a new VRF is cloned from an old template and keeps an import of the old route target, quietly merging two tenants.
• Authentication left off "internal" sessions: operators protect internet BGP and leave fabric EVPN sessions open on the assumption that the underlay is trusted.
• One shared automation credential: a single service account with full write access to the controller is used by several pipelines and stored in plain text in a CI variable.
• Duplicate MAC alerts ignored: detections fire constantly because of misconfigured multihoming, so the alert is muted and a real hijack goes unnoticed.

Practitioner checklist

• Confirm the underlay address space is not advertised into any tenant VRF, the campus or the internet.
• Permit UDP 4789 only between known VTEP loopbacks and drop it on all tenant and external interfaces.
• Protect every BGP session with a neighbour ACL, TCP-AO or MD5, and GTSM where peers are directly connected.
• Set per neighbour, per address family maximum prefix limits with alerts.
• Prefer ingress replication for BUM traffic, or filter multicast sources and groups.
• Enable duplicate MAC and IP detection, static flags for gateway MACs and ARP suppression.
• Manage VRFs, VNIs and route targets from a source of truth with pre-deployment import checks.
• Encrypt interconnect and untrusted links with MACsec or IPsec.
• Give every controller and automation tool its own scoped, vaulted credential and log its changes.
• Run an authorised test of VXLAN injection and rogue BGP peering at least annually.

Standards and references

• IETF: RFC 7348 Virtual eXtensible Local Area Network (VXLAN)
• IETF: RFC 8365 A Network Virtualization Overlay Solution Using Ethernet VPN (EVPN)
• IETF: RFC 7432 BGP MPLS-Based Ethernet VPN
• IETF: RFC 9135 Integrated Routing and Bridging in Ethernet VPN (EVPN)
• IETF: RFC 7454 BGP Operations and Security (BCP 194)
• IETF: RFC 5082 The Generalized TTL Security Mechanism (GTSM)
• NIST: SP 800-125B Secure Virtual Network Configuration for Virtual Machine (VM) Protection
• NIST: SP 800-53 Rev. 5

Related NHI Mgmt Group resources: Network Segmentation Guide · BGP and Routing Security Guide · Data Centre Security Guide · Network Security Engineer Career Guide