VXLAN Implementation.
Simplified.
Mohammed Mohsin
CCIE#35485
INTRA-VXLAN COMMUNICATION – HOW COMMUNICATION
HAPPENS BETWEEN TWO HOSTS IN SAME VXLAN ACROSS A LAYER 3
NETWORK
MOHAMMED MOHSIN
1
CCIE#35485
VXLAN demystified!
➡️ VXLAN is in simple terms switching over routing. VXLAN allows you to
create an L2 network on top of an L3 network. Using underlay and overlay
networks.
➡️ Similar to what MPLS VPNs do. PE routers know the endpoint
destinations. But the core P routers along the path have no knowledge
about the overlay networks. They only route based on external MPLS label.
➡️ The difference is MPLS VPNs provide logical layer 3 networks, whereas
VXLAN provides layer 2 overlay networks over existing layer 3 IP networks.
MOHAMMED MOHSIN
2
CCIE#35485
VXLAN demystified!
MOHAMMED MOHSIN
3
CCIE#35485
How VXLAN works?
So, to understand how VXLAN is implemented, we need to know how underlay
network is built. VXLAN utilizes this underlay network to build its overlay network.
MOHAMMED MOHSIN
4
CCIE#35485
The essential questions to ask are:
📌 How is the underlay network built?
📌 Once reachability is established between VTEPs using any IGP or BGP, how they
establish overlay VXLAN tunnels?
📌 How does a VTEP forward layer 2 frames from a locally connected source to a
remote destination mac address behind another VTEP?
📌 How unicast mac addresses are learnt over the VTEP cloud?
MOHAMMED MOHSIN
5
CCIE#35485
The essential questions to ask are:
📌 How broadcast, unknown unicast and multicast (BUM) traffic is sent across the
VTEP cloud?
📌 How to make sure there are no loops?
📌 How to best utilize the bandwidth of the underlay network using ECMP?
📌 How to adapt to dynamic changes to endpoint mac addresses? Think of VM
mobility.
MOHAMMED MOHSIN
6
CCIE#35485
Below is a primer for implementation steps
➡️ VXLAN adds 50+ bytes of overhead to the layer 2 frame. So be prepared to
increase the default MTU along the path.
➡️ A loopback interface to be used as Network Virtualization Endpoint (NVE)
interface on all VTEPs. This will be the source of VTEP through which other VTEPs
will be reachable.
MOHAMMED MOHSIN
7
CCIE#35485
Below is a primer for implementation steps
MOHAMMED MOHSIN
8
CCIE#35485
Below is a primer for implementation steps
➡️ An interior gateway protocol (IGP) such as OSPF or BGP can be used as the
underlay routing protocol for VTEP reachability through NVE.
➡️ Multicast group to VNI mapping on the VTEP external interfaces.
➡️ VNI to VLAN mapping on the VTEP internal interfaces.
MOHAMMED MOHSIN
9
CCIE#35485
Intra-VXLAN communication
MOHAMMED MOHSIN
10
CCIE#35485
Intra-VXLAN communication
➡️ Intra-VXLAN communication also referred to as L2 VNI communication is
packet transfer between two hosts in same VXLAN across the Layer 3
network between two VTEPs. In this demonstration implementation EVPN is
used to carry MAC addresses between VTEPs.
➡️ EVPN is an extension to BGP that allows the network to carry endpoint
reachability information such as Layer 2 MAC addresses (L2 VNI) and Layer 3
IP addresses (L3 VNI). This control plane technology uses MP-BGP for MAC
and IP address endpoint distribution, where MAC addresses are treated as
routes. Without EVPN, VXLAN overlay networks operate in the flood-and-
learn mode.
MOHAMMED MOHSIN
11
CCIE#35485
Underlay Network – Building the foundation. It is
also called transport network.
➡️ IP reachability between VTEPs can be achieved using any IGP like OSPF or even
BGP which will enable loopback interface on each VTEP to be reachable from other
VTEPs.
➡️ Multicast routing is required to process BUM (broadcast, unknown unicast and
multicast) traffic over the VTEP cloud. VTEPs join a particular multicast group for
each VNI. Each VNI is mapped to a multicast address so that only VTEPs that host a
particular VNI will receive BUM traffic and not flooded to all VTEPs. Think of
ethernet broadcast domains! Without multicast, there is unnecessary flooding.
MOHAMMED MOHSIN
12
CCIE#35485
Underlay Network
MOHAMMED MOHSIN
13
CCIE#35485
Overlay network – That is VXLAN.
Tunneling the frames!
➡️ Configure VLAN to VNI mappings to identify packets at the source VTEPs.
➡️ Configure NVE interface on each VTEP and assign the local loopback interface
as the source. Also configured under NVE interface is the mapping between
multicast group and VNI.
MOHAMMED MOHSIN
14
CCIE#35485
Overlay network – VXLAN.
Tunneling the frames!
➡️ Configure MP-BGP EVPN neighborships between VTEPs to advertise mac-
addresses reachable between VTEPs. It is like each VTEP telling other VTEPs what
all mac-addresses are reachable through it or residing behind it. Do remember we
are configuring this for known unicast mac-addresses.
➡️ MP-BGP uses route targets to import and export L2 VNI routes containing mac-
addresses similar to how MPLS VPNs work.
MOHAMMED MOHSIN
15
CCIE#35485
Overlay network – VXLAN.
Tunneling the frames!
MOHAMMED MOHSIN
16
CCIE#35485
Host A to Host B Communication
MOHAMMED MOHSIN
17
CCIE#35485
Host A to Host B Communication
For two hosts in same VXLAN to communicate across the VXLAN overlay network,
below steps happen.
1️⃣ Host A sends ARP request for Host B mac-address which is behind remote
VTEP.
2️⃣ Local VTEP will identify the correct VNI and send the ARP broadcast request to
the multicast group address for that VNI. It will learn Host A mac-address and VNI
mapping.
MOHAMMED MOHSIN
18
CCIE#35485
Host A to Host B Communication
3️⃣ Remote VTEP will receive the ARP broadcast request and forward it on its local
L2 interface towards the destination host. It will record the source VTEP, VNI and
Host A mac-address.
4️⃣ Once reply is received from the destination host, remote VTEP will learn the
mac-address of Host B and reply to source VTEP with IP to MAC binding in ARP
response.
5️⃣ Host A can communicate with Host B over the VXLAN overlay network as if
they are connected to same switch!
MOHAMMED MOHSIN
19
CCIE#35485
Let's understand the technologies in a simple way!
linkedin.com/in/mohsinccie
MOHAMMED MOHSIN
20
CCIE#35485
