Virtual extensible Local Area Network (VXLAN) is a tunneling protocol that tunnels Ethernet (layer 2) traffic over an IP (layer 3) network. Traditional layer 2 in three-tier network layers consist of several main issues that demands to much time and energy to manage by admins, such as Spanning-tree, limited amount of VLANs that provides network segmentation and large MAC address tables. While Spanning-tree blocks any redundant links to avoid loops and create a loop-free topology, it also means we pay for links we can’t use.
In advance, VLAN ID includes 12-bit, which means only 4094 available VLANs can be an issue for data centers. Furthermore, Because of exponentially growth of server virtualization, the number of addresses in the MAC address tables of our switches has grown. There are too many running virtualized servers on data centers while each VM has a virtual NIC and a virtual MAC address. The switch has to learn many MAC addresses on a single switch port. These challenges require a new solution to ease things on the network.
For addressing mentioned challenges, VXLAN uses an overlay and underlay network. An overlay network is a virtual network that runs on top of a physical underlay network. With VXLAN, the overlay operates as an Ethernet layer 2 and the underlay network is a layer 3 IP network also known as transport network. The underlay’s task is to transfer data between devices.
In this architecture we can use layer 3 instead of layer 2 which provides flexible protocols like IGP, OSPF or EIGRP to traffic load balancing on redundant links. Independency is another pro of this method. The virtual overlay network requires an underlay network though, changes in the overlay network won’t affect the underlay network. You can add and remove links in the underlay network, and as long as your routing protocol can reach the destination, your overlay network will remain unchanged.
Also, The VXLAN Network Identifier (VNI) plays a same role as the VLAN ID for regular VLANs. We use 24 bits for the VNI, which provides a vast number of VLANs that we can use, compared to those 4094 VLANs with a 12-bit VLAN ID. We can make many of VXLANs which enables large service providers with even thousands of customers to use as many as VXLANs per customer as needed.
For encapsulating and de-encapsulating layer 2 traffic, The VXLAN tunnel endpoint (VTEP) is used. Also, NVE (Network Virtual Interface) is a logical interface where the encapsulation and de-encapsulation occur. This device is the connection between the overlay and the underlay network. The VTEP can be deployed in both Software (host-based) and Hardware (gateway) ways.
Some hypervisors use virtual switches, and some of them support VXLAN and can be implemented as a software VTEP. A hardware VTEP is a network device like router, switch and etc., which supports VXLAN. A hardware VTEP is also a VXLAN gateway as it combines a regular VLAN and VXLAN segment into a single layer 2 domain. Some switches have VXLAN support with ASICs, offering better VXLAN performance than a software VTEP.
Hardware-based VXLAN function is supported on the Cisco Nexus 9000 Series Switches that extends Layer 2 connectivity across the Layer 3 transport network and provides a high-performance gateway between VXLAN and non-VXLAN infrastructures. Cisco Nexus 9000 Series Switches can run in Cisco ACI mode or NX-OS mode and The VXLAN implementations on Cisco Nexus 9000 Series Switches differ between ACI mode and NX-OS mode. Each modes requires a specific type of license to support various features on these platforms. Customers can use Cisco DCN and NX-OS PLR license to enable all the features permanently on Nexus 9000 switches.