Distributed VXLAN IPv4 gateway configuration example
Network requirements
As shown in Figure 19:
Configure VXLAN 10 and VXLAN 30 as unicast-mode VXLANs on Switch A, Switch B, and Switch C to provide connectivity for the VMs across the network sites.
Manually establish VXLAN tunnels and assign the tunnels to the VXLANs.
Configure distributed VXLAN IP gateways on Switch A and Switch C to forward traffic between the VXLANs.
Configure Switch B as a border gateway to forward traffic between the VXLANs and the WAN connected to Switch E.
Figure 19: Network diagram
Configuration procedure
On VM 1 and VM 3, specify 10.1.1.1 and 20.1.1.1 as the gateway address, respectively. (Details not shown.)
Configure IP addresses and unicast routing settings:
# Assign IP addresses to interfaces, as shown in Figure 19. (Details not shown.)
# Configure OSPF on all transport network switches (Switches A through D). (Details not shown.)
# Configure OSPF to advertise routes to networks 10.1.1.0/24, 20.1.1.0/24, and 25.1.1.0/24 on Switch B and Switch E. (Details not shown.)
Configure Switch A:
# Enable L2VPN.
<SwitchA> system-view [SwitchA] l2vpn enable
# Set the VXLAN hardware resource mode.
[SwitchA] hardware-resource vxlan l3gw16k
# Create VSI vpna and VXLAN 10.
[SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan-10] quit [SwitchA-vsi-vpna] quit
# Create VSI vpnc and VXLAN 30.
[SwitchA] vsi vpnc [SwitchA-vsi-vpnc] vxlan 30 [SwitchA-vsi-vpnc-vxlan-30] quit [SwitchA-vsi-vpnc] quit
# Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch B and Switch C.
[SwitchA] interface loopback 0 [SwitchA-Loopback0] ip address 1.1.1.1 255.255.255.255 [SwitchA-Loopback0] quit
# Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 1.
[SwitchA] interface tunnel 1 mode vxlan [SwitchA-Tunnel1] source 1.1.1.1 [SwitchA-Tunnel1] destination 2.2.2.2 [SwitchA-Tunnel1] quit
# Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 2.
[SwitchA] interface tunnel 2 mode vxlan [SwitchA-Tunnel2] source 1.1.1.1 [SwitchA-Tunnel2] destination 3.3.3.3 [SwitchA-Tunnel2] quit
# Assign Tunnel 1 and Tunnel 2 to VXLAN 10.
[SwitchA] vsi vpna [SwitchA-vsi-vpna] vxlan 10 [SwitchA-vsi-vpna-vxlan-10] tunnel 1 [SwitchA-vsi-vpna-vxlan-10] tunnel 2 [SwitchA-vsi-vpna-vxlan-10] quit [SwitchA-vsi-vpna] quit
# Assign Tunnel 1 and Tunnel 2 to VXLAN 30.
[SwitchA] vsi vpnc [SwitchA-vsi-vpnc] vxlan 30 [SwitchA-vsi-vpnc-vxlan-30] tunnel 1 [SwitchA-vsi-vpnc-vxlan-30] tunnel 2 [SwitchA-vsi-vpnc-vxlan-30] quit [SwitchA-vsi-vpnc] quit
# On Ten-GigabitEthernet 1/0/1, create Ethernet service instance 1000 to match VLAN 2.
[SwitchA] interface ten-gigabitethernet 1/0/1 [SwitchA-Ten-GigabitEthernet1/0/1] service-instance 1000 [SwitchA-Ten-GigabitEthernet1/0/1-srv1000] encapsulation s-vid 2
# Map Ethernet service instance 1000 to VSI vpna.
[SwitchA-Ten-GigabitEthernet1/0/1-srv1000] xconnect vsi vpna [SwitchA-Ten-GigabitEthernet1/0/1-srv1000] quit [SwitchA-Ten-GigabitEthernet1/0/1] quit
# Create VSI-interface 1 and assign the interface an IP address and a MAC address. The IP address will be used as the gateway address for VXLAN 10.
[SwitchA] interface vsi-interface 1 [SwitchA-Vsi-interface1] ip address 10.1.1.1 255.255.255.0 [SwitchA-Vsi-interface1] mac-address 1-1-1
# Specify VSI-interface 1 as a distributed gateway and enable local proxy ARP on the interface.
[SwitchA-Vsi-interface1] distributed-gateway local [SwitchA-Vsi-interface1] local-proxy-arp enable [SwitchA-Vsi-interface1] quit
# Create VSI-interface 2 and assign the interface an IP address and a MAC address. The IP address will be used as the gateway address for VXLAN 30.
[SwitchA] interface vsi-interface 2 [SwitchA-Vsi-interface2] ip address 20.1.1.1 255.255.255.0 [SwitchA-Vsi-interface2] mac-address 2-2-2
# Specify VSI-interface 2 as a distributed gateway and enable local proxy ARP on the interface.
[SwitchA-Vsi-interface2] distributed-gateway local [SwitchA-Vsi-interface2] local-proxy-arp enable [SwitchA-Vsi-interface2] quit
# Disable source MAC check on transport-facing interface Ten-GigabitEthernet 1/0/2.
[SwitchA] interface ten-gigabitethernet 1/0/2 [SwitchA-Ten-GigabitEthernet1/0/2] undo mac-address static source-check enable
# Enable dynamic ARP entry synchronization for distributed VXLAN IP gateways.
[SwitchA] arp distributed-gateway dynamic-entry synchronize
# Specify VSI-interface 1 as the gateway interface for VSI vpna.
[SwitchA] vsi vpna [SwitchA-vsi-vpna] gateway vsi-interface 1 [SwitchA-vsi-vpna] quit
# Specify VSI-interface 2 as the gateway interface for VSI vpnc.
[SwitchA] vsi vpnc [SwitchA-vsi-vpnc] gateway vsi-interface 2 [SwitchA-vsi-vpnc] quit
# Configure a PBR policy for VXLAN 10. Set the policy name to vxlan10, and set the next hop to 10.1.1.2 (VSI-interface 1 on Switch B).
[SwitchA] acl advanced 3000 [SwitchA-acl-ipv4-adv-3000] rule 0 permit ip [SwitchA-acl-ipv4-adv-3000] quit [SwitchA] policy-based-route vxlan10 permit node 5 [SwitchA-pbr-vxlan10-5] if-match acl 3000 [SwitchA-pbr-vxlan10-5] apply next-hop 10.1.1.2 [SwitchA-pbr-vxlan10-5] quit
# Configure a PBR policy for VXLAN 30. Set the policy name to vxlan30, and set the next hop to 20.1.1.2 (VSI-interface 2 on Switch B).
[SwitchA] policy-based-route vxlan30 permit node 5 [SwitchA-pbr-vxlan30-5] if-match acl 3000 [SwitchA-pbr-vxlan30-5] apply next-hop 20.1.1.2 [SwitchA-pbr-vxlan30-5] quit
# Apply policies vxlan10 and vxlan30 to VSI-interface 1 and VSI-interface 2, respectively.
[SwitchA] interface vsi-interface 1 [SwitchA-Vsi-interface1] ip policy-based-route vxlan10 [SwitchA-Vsi-interface1] quit [SwitchA] interface vsi-interface 2 [SwitchA-Vsi-interface2] ip policy-based-route vxlan30 [SwitchA-Vsi-interface2] quit
Configure Switch B:
# Enable L2VPN.
<SwitchB> system-view [SwitchB] l2vpn enable
# Set the VXLAN hardware resource mode.
[SwitchB] hardware-resource vxlan border24k
# Create VSI vpna and VXLAN 10.
[SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan-10] quit [SwitchB-vsi-vpna] quit
# Create VSI vpnc and VXLAN 30.
[SwitchB] vsi vpnc [SwitchB-vsi-vpnc] vxlan 30 [SwitchB-vsi-vpnc-vxlan-30] quit [SwitchB-vsi-vpnc] quit
# Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch C.
[SwitchB] interface loopback 0 [SwitchB-Loopback0] ip address 2.2.2.2 255.255.255.255 [SwitchB-Loopback0] quit
# Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 2.
[SwitchB] interface tunnel 2 mode vxlan [SwitchB-Tunnel2] source 2.2.2.2 [SwitchB-Tunnel2] destination 1.1.1.1 [SwitchB-Tunnel2] quit
# Create a VXLAN tunnel to Switch C. The tunnel interface name is Tunnel 3.
[SwitchB] interface tunnel 3 mode vxlan [SwitchB-Tunnel3] source 2.2.2.2 [SwitchB-Tunnel3] destination 3.3.3.3 [SwitchB-Tunnel3] quit
# Assign Tunnel 2 to VXLAN 10.
[SwitchB] vsi vpna [SwitchB-vsi-vpna] vxlan 10 [SwitchB-vsi-vpna-vxlan-10] tunnel 2 [SwitchB-vsi-vpna-vxlan-10] quit [SwitchB-vsi-vpna] quit
# Assign Tunnel 3 to VXLAN 30.
[SwitchB] vsi vpnc [SwitchB-vsi-vpnc] vxlan 30 [SwitchB-vsi-vpnc-vxlan-30] tunnel 3 [SwitchB-vsi-vpnc-vxlan-30] quit [SwitchB-vsi-vpnc] quit
# Create VSI-interface 1 and assign the interface an IP address.
[SwitchB] interface vsi-interface 1 [SwitchB-Vsi-interface1] ip address 10.1.1.2 255.255.255.0 [SwitchB-Vsi-interface1] quit
# Create VSI-interface 2 and assign the interface an IP address.
[SwitchB] interface vsi-interface 2 [SwitchB-Vsi-interface2] ip address 20.1.1.2 255.255.255.0 [SwitchB-Vsi-interface2] quit
# Specify VSI-interface 1 as the gateway interface for VSI vpna.
[SwitchB] vsi vpna [SwitchB-vsi-vpna] gateway vsi-interface 1 [SwitchB-vsi-vpna] quit
# Specify VSI-interface 2 as the gateway interface for VSI vpnc.
[SwitchB] vsi vpnc [SwitchB-vsi-vpnc] gateway vsi-interface 2 [SwitchB-vsi-vpnc] quit
Configure Switch C:
# Enable L2VPN.
<SwitchC> system-view [SwitchC] l2vpn enable
# Set the VXLAN hardware resource mode.
[SwitchC] hardware-resource vxlan l3gw16k
# Create VSI vpna and VXLAN 10.
[SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan-10] quit [SwitchC-vsi-vpna] quit
# Create VSI vpnb and VXLAN 30.
[SwitchC] vsi vpnb [SwitchC-vsi-vpnb] vxlan 30 [SwitchC-vsi-vpnb-vxlan-30] quit [SwitchC-vsi-vpnb] quit
# Assign an IP address to Loopback 0. The IP address will be used as the source IP address of the VXLAN tunnels to Switch A and Switch B.
[SwitchC] interface loopback 0 [SwitchC-Loopback0] ip address 3.3.3.3 255.255.255.255 [SwitchC-Loopback0] quit
# Create a VXLAN tunnel to Switch A. The tunnel interface name is Tunnel 1.
[SwitchC] interface tunnel 1 mode vxlan [SwitchC-Tunnel1] source 3.3.3.3 [SwitchC-Tunnel1] destination 1.1.1.1 [SwitchC-Tunnel1] quit
# Create a VXLAN tunnel to Switch B. The tunnel interface name is Tunnel 3.
[SwitchC] interface tunnel 3 mode vxlan [SwitchC-Tunnel3] source 3.3.3.3 [SwitchC-Tunnel3] destination 2.2.2.2 [SwitchC-Tunnel3] quit
# Assign Tunnel 1 and Tunnel 3 to VXLAN 10.
[SwitchC] vsi vpna [SwitchC-vsi-vpna] vxlan 10 [SwitchC-vsi-vpna-vxlan-10] tunnel 1 [SwitchC-vsi-vpna-vxlan-10] tunnel 3 [SwitchC-vsi-vpna-vxlan-10] quit [SwitchC-vsi-vpna] quit
# Assign Tunnel 1 and Tunnel 3 to VXLAN 30.
[SwitchC] vsi vpnb [SwitchC-vsi-vpnb] vxlan 30 [SwitchC-vsi-vpnb-vxlan-30] tunnel 1 [SwitchC-vsi-vpnb-vxlan-30] tunnel 3 [SwitchC-vsi-vpnb-vxlan-30] quit [SwitchC-vsi-vpnb] quit
# On , create Ethernet service instance 1000 to match VLAN 4.
[SwitchC] interface ten-gigabitethernet 1/0/1 [SwitchC-] service-instance 1000 [SwitchC-Ten-GigabitEthernet1/0/1-srv1000] encapsulation s-vid 4
# Map Ethernet service instance 1000 to VSI vpnb.
[SwitchC-Ten-GigabitEthernet1/0/1-srv1000] xconnect vsi vpnb [SwitchC-Ten-GigabitEthernet1/0/1-srv1000] quit [SwitchC-Ten-GigabitEthernet1/0/1] quit
# Create VSI-interface 1 and assign the interface an IP address and a MAC address. The IP address will be used as the gateway address for VXLAN 10.
[SwitchC] interface vsi-interface 1 [SwitchC-Vsi-interface1] ip address 10.1.1.1 255.255.255.0 [SwitchC-Vsi-interface1] mac-address 1-1-1
# Specify VSI-interface 1 as a distributed gateway and enable local proxy ARP on the interface.
[SwitchC-Vsi-interface1] distributed-gateway local [SwitchC-Vsi-interface1] local-proxy-arp enable [SwitchC-Vsi-interface1] quit
# Disable source MAC check on transport-facing interface Ten-GigabitEthernet 1/0/2.
[SwitchC] interface ten-gigabitethernet 1/0/2 [SwitchC-Ten-GigabitEthernet1/0/2] undo mac-address static source-check enable
# Enable dynamic ARP entry synchronization for distributed VXLAN IP gateways.
[SwitchC] arp distributed-gateway dynamic-entry synchronize
# Specify VSI-interface 1 as the gateway interface for VSI vpna.
[SwitchC] vsi vpna [SwitchC-vsi-vpna] gateway vsi-interface 1 [SwitchC-vsi-vpna] quit
# Create VSI-interface 2 and assign the interface an IP address and a MAC address. The IP address will be used as the gateway address for VXLAN 30.
[SwitchC] interface vsi-interface 2 [SwitchC-Vsi-interface2] ip address 20.1.1.1 255.255.255.0 [SwitchC-Vsi-interface2] mac-address 2-2-2
# Specify VSI-interface 2 as a distributed gateway and enable local proxy ARP on the interface.
[SwitchC-Vsi-interface2] distributed-gateway local [SwitchC-Vsi-interface2] local-proxy-arp enable [SwitchC-Vsi-interface2] quit
# Specify VSI-interface 2 as the gateway interface for VSI vpnb.
[SwitchC] vsi vpnb [SwitchC-vsi-vpnb] gateway vsi-interface 2 [SwitchC-vsi-vpnb] quit
# Configure a PBR policy for the VXLANs. Set the policy name to vxlan and set the next hop to 20.1.1.2 (VSI-interface 1 on Switch B).
[SwitchC] acl advanced 3000 [SwitchC-acl-ipv4-adv-3000] rule 0 permit ip [SwitchC-acl-ipv4-adv-3000] quit [SwitchC] policy-based-route vxlan permit node 5 [SwitchC-pbr-vxlan-5] if-match acl 3000 [SwitchC-pbr-vxlan-5] apply next-hop 20.1.1.2 [SwitchC-pbr-vxlan-5] quit
# Apply policy vxlan to VSI-interface 2.
[SwitchC] interface vsi-interface 2 [SwitchC-Vsi-interface2] ip policy-based-route vxlan [SwitchC-Vsi-interface2] quit
Verifying the configuration
Verify the VXLAN IP gateway settings on Switch A:
# Verify that the VXLAN tunnel interfaces are up on Switch A.
[SwitchA] display interface tunnel 2 Tunnel2 Current state: UP Line protocol state: UP Description: Tunnel2 Interface Bandwidth: 64 kbps Maximum transmission unit: 1464 Internet protocol processing: Disabled Last clearing of counters: Never Tunnel source 1.1.1.1, destination 3.3.3.3 Tunnel protocol/transport UDP_VXLAN/IP Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops Output: 0 packets, 0 bytes, 0 drops
# Verify that VSI-interface 1 is up.
[SwitchA] display interface vsi-interface 1 Vsi-interface1 Current state: UP Line protocol state: UP Description: Vsi-interface1 Interface Bandwidth: 1000000 kbps Maximum transmission unit: 1444 Internet address: 10.1.1.1/24 (primary) IP packet frame type: Ethernet II, hardware address: 0001-0001-0001 IPv6 packet frame type: Ethernet II, hardware address: 0001-0001-0001 Physical: Unknown, baudrate: 1000000 kbps Last clearing of counters: Never Input (total): 0 packets, 0 bytes Output (total): 0 packets, 0 bytes
# Verify that the VXLAN tunnels have been assigned to VXLAN 10, and VSI-interface 1 is the gateway interface for VSI vpna.
[SwitchA] display l2vpn vsi name vpna verbose VSI Name: vpna VSI Index : 0 VSI State : Up MTU : 1500 Bandwidth : Unlimited Broadcast Restrain : Unlimited Multicast Restrain : Unlimited Unknown Unicast Restrain: Unlimited MAC Learning : Enabled MAC Table Limit : - MAC Learning rate : - Drop Unknown : - Flooding : Enabled Statistics : Disabled Gateway Interface : VSI-interface 1 VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flood proxy Tunnel1 0x5000001 Up Manual Disabled Tunnel2 0x5000002 Up Manual Disabled ACs: AC Link ID State Type XGE1/0/1 srv1000 0 Up Manual# Verify that Switch A has created ARP entries for the VMs.
[SwitchA] display arp Type: S-Static D-Dynamic O-Openflow R-Rule M-Multiport I-Invalid IP address MAC address VLAN/VSI Interface/Link ID Aging Type 11.1.1.4 000c-29c1-5e46 11 Vlan11 19 D 10.1.1.2 0003-0000-0000 N/A Vsi1 20 D 10.1.1.11 0cda-41b5-cf09 N/A Vsi1 20 D 20.1.1.12 0000-fc00-0b01 N/A Vsi2 19 D
Verify the configuration on the border gateway Switch B:
# Verify that the VXLAN tunnel interfaces are up on Switch B.
[SwitchB] display interface tunnel 2 Tunnel2 Current state: UP Line protocol state: UP Description: Tunnel2 Interface Bandwidth: 64 kbps Maximum transmission unit: 1464 Internet protocol processing: Disabled Last clearing of counters: Never Tunnel source 2.2.2.2, destination 1.1.1.1 Tunnel protocol/transport UDP_VXLAN/IP Last 300 seconds input rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Last 300 seconds output rate: 0 bytes/sec, 0 bits/sec, 0 packets/sec Input: 0 packets, 0 bytes, 0 drops Output: 0 packets, 0 bytes, 0 drops
# Verify that VSI-interface 1 is up.
[SwitchB] display interface vsi-interface 1 Vsi-interface1 Current state: UP Line protocol state: UP Description: Vsi-interface1 Interface Bandwidth: 1000000 kbps Maximum transmission unit: 1444 Internet address: 10.1.1.2/24 (primary) IP packet frame type: Ethernet II, hardware address: 0011-2200-0102 IPv6 packet frame type: Ethernet II, hardware address: 0011-2200-0102 Physical: Unknown, baudrate: 1000000 kbps Last clearing of counters: Never Input (total): 0 packets, 0 bytes Output (total): 0 packets, 0 bytes
# Verify that the VXLAN tunnels have been assigned to VXLAN 10, and VSI-interface 1 is the gateway interface for VSI vpna.
[SwitchB] display l2vpn vsi name vpna verbose VSI Name: vpna VSI Index : 0 VSI State : Up MTU : 1500 Bandwidth : Unlimited Broadcast Restrain : Unlimited Multicast Restrain : Unlimited Unknown Unicast Restrain: Unlimited MAC Learning : Enabled MAC Table Limit : - MAC Learning rate : - Drop Unknown : - Flooding : Enabled Statistics : Disabled Gateway interface : VSI-interface 1 VXLAN ID : 10 Tunnels: Tunnel Name Link ID State Type Flood proxy Tunnel2 0x5000002 Up Manual Disabled# Verify that Switch B has created ARP entries for the VMs.
[SwitchB] display arp Type: S-Static D-Dynamic O-Openflow R-Rule M-Multiport I-Invalid IP address MAC address VLAN/VSI Interface/Link ID Aging Type 12.1.1.4 0000-fc00-00ab 12 Vlan12 14 D 25.1.1.5 4431-9234-24bb 20 Vlan20 17 D 10.1.1.1 0000-fc00-00ab N/A Vsi1 17 D 10.1.1.11 0000-fc00-00ab N/A Vsi1 20 D 20.1.1.1 0000-fc00-00aa N/A Vsi3 17 D 20.1.1.12 0000-fc00-00aa N/A Vsi3 20 D
# Verify that Switch B has created FIB entries for the VMs.
[SwitchB] display fib 10.1.1.11 Destination count: 1 FIB entry count: 1 Flag: U:Useable G:Gateway H:Host B:Blackhole D:Dynamic S:Static R:Relay F:FRR Destination/Mask Nexthop Flag OutInterface/Token Label 10.1.1.11/32 10.1.1.11 UH Vsi1 Null [SwitchB] display fib 20.1.1.12 Destination count: 1 FIB entry count: 1 Flag: U:Useable G:Gateway H:Host B:Blackhole D:Dynamic S:Static R:Relay F:FRR Destination/Mask Nexthop Flag OutInterface/Token Label 20.1.1.12/32 20.1.1.12 UH Vsi1 Null
Verify that the network connectivity for VMs meets the requirements:
# Verify that VM 1 and VM 3 can ping each other. (Details not shown.)
# Verify that VM 1 and VM 3 can ping VLAN-interface 20 (25.1.1.5) on Switch E for WAN access. (Details not shown.)