Configure BGP-EVPN based Layer-2 VPN service

11 minutes read

In the last post, we configured the BGP-EVPN based Multi-homing of host/CE using EVPN Ethernet Segment. In this post, we will provision BGP-EVPN based Layer-2 VPN service between the Leafs. The EVPN Layer-2 service will enable forwarding between host-1 and host-5 which are part of the same subnet.

Reference Topology:

In this setup, Host-1 and Host-5 belong to the same subnet. Host-1 is dual-homed to Leaf-1 and Leaf-2 while Host-5 is single homed to the Leaf-5. Packets sourced from Host-1 for destination Host-5 will arrive to Leaf-1 or Leaf-2 based on the LAG’s hash calculation. On Leaf the lookup will be performed for destination Host-5 MAC address. Host-5’s MAC address will be learnt on Leaf-1 and Leaf-2 via EVPN control-plane. After the lookup, the traffic will be forwarded to the Host-5 MAC address using EVPN service label and transport label to reach to Leaf-5.

Task 1: Configure Host-1 and Host-5 IP address

Host-1 and Host-5 will be part of the same subnet to communicate over layer-2 stretch. Host-1 is connected dual-homed to uplink Leafs via LACP link aggregation and Host-5 is connected single-homed to Leaf-5. Configure IP address on Host-1’s and Host-5 as follows.


    Host-1

    interface Bundle-Ether1
     description "Bundle to Leaf-1/2"
     ipv4 address 10.0.0.10 255.255.255.0
     !

    Host-5

    interface TenGigE0/0/2/0
     description "Link to Leaf-5"
     ipv4 address 10.0.0.50 255.255.255.0
     !

Task 2: Configure Layer-2 interfaces and Bridge Domain on Leafs

Configure layer-2 interfaces with dot1q encapsulation for VLAN 10 on Leaf-1 and Leaf-2. Use the following configuration for both Leaf-1, Leaf-2 and Leaf-5.


    Leaf-1 and Leaf-2

    interface Bundle-Ether 1.10 l2transport
     encapsulation dot1q 10
     rewrite ingress tag pop 1 symmetric
    !

    Leaf-5

    interface TenGigE0/0/0/47.10 l2transport
     encapsulation dot1q 10
     rewrite ingress tag pop 1 symmetric
    !

Configure Bridge domain for the VLAN and add the VLAN tagged interfaces to the bridge-domain. Configure the following on Leaf-1, Leaf-2 and Leaf-5.


    Leaf-1 and Leaf-2

    l2vpn
     bridge group bg-1
      bridge-domain bd-10
       interface Bundle-Ether 1.10
       !
    !


    Leaf-5

    l2vpn
     bridge group bg-1
      bridge-domain bd-10
       interface TenGigE0/0/0/47.10
       !
    ! 

Verify that the bridge-domain and the related attachment circuits are up. Following output shows that the bridge-domain bd-10’s state is ‘up’, its attachment circuit is ‘up’.


    Leaf-1

    RP/0/RP0/CPU0:Leaf-1#show l2vpn bridge-domain bd-name bd-10
    Legend: pp = Partially Programmed.
    Bridge group: bg-1, bridge-domain: bd-10, id: 0, state: up, ShgId: 0, MSTi: 0
      Aging: 300 s, MAC limit: 64000, Action: none, Notification: syslog
      Filter MAC addresses: 0
      ACs: 1 (1 up), VFIs: 0, PWs: 0 (0 up), PBBs: 0 (0 up), VNIs: 0 (0 up)
      List of ACs:
        BE1.10, state: up, Static MAC addresses: 0
      List of Access PWs:
      List of VFIs:
      List of Access VFIs:


    Leaf-5

    RP/0/RP0/CPU0:Leaf-5#show l2vpn bridge-domain bd-name bd-10
    Legend: pp = Partially Programmed.
    Bridge group: bg-1, bridge-domain: bd-10, id: 0, state: up, ShgId: 0, MSTi: 0
      Aging: 300 s, MAC limit: 64000, Action: none, Notification: syslog
      Filter MAC addresses: 0
      ACs: 1 (1 up), VFIs: 0, PWs: 0 (0 up), PBBs: 0 (0 up), VNIs: 0 (0 up)
      List of ACs:
        Te0/0/0/47.10, state: up, Static MAC addresses: 0
      List of Access PWs:
      List of VFIs:
      List of Access VFIs:
    RP/0/RP0/CPU0:Leaf-5#

So far, we have configured local bridging on the Leafs and connected them to the hosts for vlan 10 tagged data. We verified that the local bridging and attachment circuits are ‘up’. In order for Host-1 to communicate to Host-5 via layer-2, we need to configure layer-2 stretch/service between the Leafs to which Hosts are connected.

The layer-2 service/stretch across the Leafs is offered by configuring EVPN EVI (EVPN Instance). EVI allows the layer-2 to be stretched via MP-BGP EVPN control-plane across multiple participating Leafs/PEs. An EVI is configured on a per layer-2 bridge basis across Leafs/PEs. Each EVI has a unique route distinguisher and one or more route targets.

For Layer-2 VPN use case, we are stretching the layer-2 between Leaf-1, Leaf-2 and Leaf-5. Therefore, we will provision Layer-2 VPN service by configure EVI on all three leafs.

Task 3: Configure EVPN EVI on Leaf-1, Leaf-2 for VLAN 10

First we will configure the EVI on Leaf-1 and Leaf-2, then we will verify that the Ethernet Segment for vlan 10 tagged data is up.

Configure EVI in EVPN config on Leaf-1 and Leaf-2. Also assign the route-target values for the EVI related network to get advertised and received via BGP EVPN control-plane. Advertise-mac keyword is used to advertise the MAC addresses in EVI to other Leafs part of EVI via BGP EVPN.


    Leaf-1 and Leaf-2

    evpn
     evi 10
      bgp
       route-target import 1001:11
       route-target export 1001:11
      !
      advertise-mac
      !
     source interface loopback 0
     !

Associate the EVI to bridge-domain for VLAN 10, this is where the attachment-circuit/host is connected to.


    l2vpn
     bridge group bg-1
      bridge-domain bd-10
       evi 10
       !
      !

As we have now configured layer-2 service with EVI for Bridge-domain 10, lets verify the Ethernet Segment status to see that the multi-homing is operational for Bridge-domain 10 forwarding.

Observe in the below output that for Ethernet-segment bundle interface ‘BE1’, there are two next-hops. The next-hops represent each Leaf-1 and Leaf-2 forming Leaf pair for Ethernet Segment. Also in below output we can see that Ethernet-segment state is ‘Up’ and all-active multi-homing is operational. We have one forwarder which is VLAN 10 and Leaf-1 is the elected designated forwarded (DF) for it.


    Leaf-1 

    RP/0/RP0/CPU0:Leaf-1#show evpn ethernet-segment detail 
 
    Ethernet Segment Id      Interface                          Nexthops            
    ------------------------ ---------------------------------- --------------------
    0011.1111.1111.1111.1111 BE1                                1.1.1.1
                                                                2.2.2.2
      ES to BGP Gates   : Ready
      ES to L2FIB Gates : Ready
      Main port         :
         Interface name : Bundle-Ether1
         Interface MAC  : 00bc.601c.d0da
         IfHandle       : 0x08000044
         State          : Up
         Redundancy     : Not Defined
      ESI type          : 0
         Value          : 11.1111.1111.1111.1111
      ES Import RT      : 1111.1111.1111 (Local)
      Source MAC        : 0000.0000.0000 (N/A)
      Topology          :
         Operational    : MH, All-active
         Configured     : All-active (AApF) (default)
      Service Carving   : Auto-selection
      Peering Details   : 1.1.1.1[MOD:P:00] 2.2.2.2[MOD:P:00]
      Service Carving Results:
         Forwarders     : 1
         Permanent      : 0
         Elected        : 1
         Not Elected    : 0
      MAC Flushing mode : STP-TCN
      Peering timer     : 3 sec [not running]
      Recovery timer    : 30 sec [not running]
      Carving timer     : 0 sec [not running]
      Local SHG label   : 24061
      Remote SHG labels : 1
                  24043 : nexthop 2.2.2.2

    RP/0/RP0/CPU0:Leaf-1#

With the following CLI command we can verify that the MAC address of Host-1 is being learnt on Leaf-1 and Leaf-2. MAC address of Host-5 will be learnt on Leaf-1 and Leaf-2 after we configure EVI on Leaf-5 for VLAN 10 layer-2 stretch.


	Leaf-1 

    RP/0/RP0/CPU0:Leaf-1#show l2route evpn mac all 
    Topo ID  Mac Address    Producer    Next Hop(s)                             
    -------- -------------- ----------- ----------------------------------------
    0        6c9c.ed6d.1d8b LOCAL       Bundle-Ether1.10
    RP/0/RP0/CPU0:Leaf-1#


    Leaf-2

    RP/0/RP0/CPU0:Leaf-2#show l2route evpn mac all 
    Sat Sep  1 22:49:43.498 UTC
    Topo ID  Mac Address    Producer    Next Hop(s)                             
    -------- -------------- ----------- ----------------------------------------
    0        6c9c.ed6d.1d8b L2VPN       Bundle-Ether1.10                        
    RP/0/RP0/CPU0:Leaf-2#

Task 4: Configure EVPN EVI on Leaf-5 for VLAN 10


    On Leaf-5

    evpn
     evi 10
      bgp
       route-target import 1001:11
       route-target export 1001:11
      !
      advertise-mac
      !
     source interface loopback 0
     !

Associate the EVI to bridge-domain for VLAN 10, this is where the attachment-circuit/host is connected to.


    l2vpn
     bridge group bg-1
      bridge-domain bd-10
       evi 10
       !
      !

Task 5: Verify EVPN EVI and Layer-2 Stretch between the Leaf-1, Leaf-2 and Leaf-5

We have configured the Layer-2 stretch between Leaf-1, Leaf-2 and Leaf-5 using EVPN EVI. In the next steps lets verify the layer-2 connectivity is up and we can reach from one host to another via layer-2. “show evpn evi detail” cli command shows the configured EVI and its associated bridge-domain. It also shows the route-target import and export values as shown in the below output.


    RP/0/RP0/CPU0:Leaf-1#show evpn evi detail

    VPN-ID     Encap  Bridge Domain                Type               
    ---------- ------ ---------------------------- -------------------
    10         MPLS   bd-10                        EVPN               
       Stitching: Regular
       Unicast Label  : 24060
       Multicast Label: 24121
       Flow Label: N
       Control-Word: Enabled
       Forward-class: 0
       Advertise MACs: Yes
       Advertise BVI MACs: No
       Aliasing: Enabled
       UUF: Enabled
       Re-origination: Enabled
       Multicast source connected: No

       Statistics:
         Packets            Sent                 Received
           Total          : 0                    0                   
           Unicast        : 0                    0                   
           BUM            : 0                    0                   
         Bytes              Sent                 Received
           Total          : 0                    0                   
           Unicast        : 0                    0                   
           BUM            : 0                    0                   
       RD Config: none
       RD Auto  : (auto) 1.1.1.1:10
       RT Auto  : 65001:10
       Route Targets in Use           Type                 
       ------------------------------ ---------------------
       1001:11                        Import               
       1001:11                        Export               

    RP/0/RP0/CPU0:Leaf-1#

Ping from Host-1 to Host-5 and verify that the Hosts are reachable. We can see in the below output that that Host-1 can ping Host-5. Also, below output shows that the MAC address for Host-5 is learnt on Leaf-1 and Leaf-2. Similarly, we are learning the MAC address of Host-1 on Leaf-5.



   Host-1
   
    RP/0/RSP0/CPU0:Host-1#ping 10.0.0.50            
    Type escape sequence to abort.
    Sending 5, 100-byte ICMP Echos to 10.0.0.50, timeout is 2 seconds:
    !!!!!
    Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
    RP/0/RSP0/CPU0:Host-1#


   Leaf-1
    
    RP/0/RP0/CPU0:Leaf-1#show l2route evpn mac all 
    Sat Sep  1 22:53:57.880 UTC
    Topo ID  Mac Address    Producer    Next Hop(s)                             
    -------- -------------- ----------- ----------------------------------------
    0        6c9c.ed6d.1d8b LOCAL       Bundle-Ether1.10                        
    0        a03d.6f3d.5443 L2VPN       5.5.5.5/24002/ME                        
    RP/0/RP0/CPU0:Leaf-1#


   Leaf-2

    RP/0/RP0/CPU0:Leaf-2#show l2route evpn mac all 
    Sat Sep  1 23:00:03.487 UTC
    Topo ID  Mac Address    Producer    Next Hop(s)                             
    -------- -------------- ----------- ----------------------------------------
    0        6c9c.ed6d.1d8b L2VPN       Bundle-Ether1.10                        
    0        a03d.6f3d.5443 L2VPN       5.5.5.5/24002/ME                        
    RP/0/RP0/CPU0:Leaf-2#

    Leaf-5

    RP/0/RP0/CPU0:Leaf-5#show l2route evpn mac all 
    Sat Sep  1 23:00:03.785 UTC
    Topo ID  Mac Address    Producer    Next Hop(s)                             
    -------- -------------- ----------- ----------------------------------------
    0        6c9c.ed6d.1d8b L2VPN       24007/I/ME                              
    0        a03d.6f3d.5443 LOCAL       TenGigE0/0/0/47.10                      
    RP/0/RP0/CPU0:Leaf-5#

We can verify the BGP EVPN control-plane to verify the various routes and mac addresses are advertised and learnt.

In the below output from Leaf-1 we can see the MAC address of Host-1 and Host-5 are being learnt under their respective route distinguishers. MAC addresses are advertised using EVPN Route-Type-2.

Example of Host-1 MAC learnt ([2][0][48][6c9c.ed6d.1d8b][0]/104)

The route distinguisher value is comprised of router-id:EVI eg. 1.1.1.1:10, 2.2.2.2:10 which are highlighted below.


    Leaf-5

    RP/0/RP0/CPU0:Leaf-5#show bgp l2vpn evpn rd 1.1.1.1:10

    Status codes: s suppressed, d damped, h history, * valid, > best
                  i - internal, r RIB-failure, S stale, N Nexthop-discard
    Origin codes: i - IGP, e - EGP, ? - incomplete
       Network            Next Hop            Metric LocPrf Weight Path
    Route Distinguisher: 1.1.1.1:10
    *>i[1][0011.1111.1111.1111.1111][0]/120
                          1.1.1.1                       100      0 i
    * i                   1.1.1.1                       100      0 i
    *>i[2][0][48][6c9c.ed6d.1d8b][0]/104
                          1.1.1.1                       100      0 i
    * i                   1.1.1.1                       100      0 i
    *>i[3][0][32][1.1.1.1]/80
                          1.1.1.1                       100      0 i
    * i                   1.1.1.1                       100      0 i

    Processed 3 prefixes, 6 paths
    RP/0/RP0/CPU0:Leaf-5#



    RP/0/RP0/CPU0:Leaf-5#show bgp l2vpn evpn rd 2.2.2.2:10

    Status codes: s suppressed, d damped, h history, * valid, > best
                  i - internal, r RIB-failure, S stale, N Nexthop-discard
    Origin codes: i - IGP, e - EGP, ? - incomplete
       Network            Next Hop            Metric LocPrf Weight Path
    Route Distinguisher: 2.2.2.2:10
    *>i[1][0011.1111.1111.1111.1111][0]/120
                          2.2.2.2                       100      0 i
    * i                   2.2.2.2                       100      0 i
    *>i[2][0][48][6c9c.ed6d.1d8b][0]/104
                          2.2.2.2                       100      0 i
    * i                   2.2.2.2                       100      0 i
    *>i[3][0][32][2.2.2.2]/80
                          2.2.2.2                       100      0 i
    * i                   2.2.2.2                       100      0 i

    Processed 3 prefixes, 6 paths
    RP/0/RP0/CPU0:Leaf-5#

CLI command “show evpn evi vpn-id 10 mac” can be used to verify the MAC address and Host IP addresses being learnt related to the EVI. In the following output of EVI table from Leaf-5, we can see that we are learning MAC address of Host-1 via EVI 10 on Leaf-5. We can reach to Host-1 MAC address either via next-hop 1.1.1.1 of Leaf-1 or 2.2.2.2 which is Leaf-2. We can run the same command on Leaf-1 and Leaf-2 for verification.


    Leaf-5

    RP/0/RP0/CPU0:Leaf-5#show evpn evi vpn-id 10 mac
    Sat Sep  1 23:24:00.808 UTC

    VPN-ID     Encap  MAC address    IP address       Nexthop                                 Label   
    ---------- ------ -------------- ---------------------------------------- -----------------------------
    10         MPLS   6c9c.ed6d.1d8b ::               1.1.1.1                                24060   
    10         MPLS   6c9c.ed6d.1d8b ::               2.2.2.2                                24042   
    10         MPLS   a03d.6f3d.5443 ::               TenGigE0/0/0/47.10                      24002   
    RP/0/RP0/CPU0:Leaf-5#

We are only seeing MAC address and not IP address of the Host in the above output. This is because we configured only Layer-2 service between the Leafs. Once we configure EVPN IRB, we will start advertising MAC + IP of the host via EVPN Route-Type-2 and will be able to see IP address in the above show command as well as in Leaf’s routing table.

Since only MAC address is advertised, the advertisement will only have Bridge-Domain/EVI label and its respective route-target. In below output on Leaf-5 for route type 2 learnt from Leaf-1 (RD 1.1.1.1:10), we can see the highlighted route-target and Bridge-Domain/EVI label value.


     Leaf-5
     
      RP/0/RP0/CPU0:Leaf-5#sh bgp l2vpn evpn rd 1.1.1.1:10 [2][0][48][6c9c.ed6d.1d8b][0]/104 detail
      BGP routing table entry for [2][0][48][6c9c.ed6d.1d8b][0]/104, Route Distinguisher: 1.1.1.1:10
      Versions:
        Process           bRIB/RIB  SendTblVer
        Speaker                 44          44
          Flags: 0x00040001+0x00010000; 
      Last Modified: Jul 26 01:34:57.072 for 00:00:03
      Paths: (2 available, best #1)
        Not advertised to any peer
        Path #1: Received by speaker 0
        Flags: 0x4000000025060005, import: 0x1f, EVPN: 0x1
        Not advertised to any peer
        Local
          1.1.1.1 (metric 20) from 6.6.6.6 (1.1.1.1)
            Received Label 24060 
            Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, not-in-vrf
            Received Path ID 0, Local Path ID 1, version 44
            Extended community: Flags 0x10: SoO:1.1.1.1:10 RT:1001:11 
            Originator: 1.1.1.1, Cluster list: 6.6.6.6
            EVPN ESI: 0011.1111.1111.1111.1111
        Path #2: Received by speaker 0
        Flags: 0x4000000020020005, import: 0x20, EVPN: 0x1
        Not advertised to any peer
        Local
          1.1.1.1 (metric 20) from 7.7.7.7 (1.1.1.1)
            Received Label 24060 
            Origin IGP, localpref 100, valid, internal, not-in-vrf
            Received Path ID 0, Local Path ID 0, version 0
            Extended community: Flags 0x10: SoO:1.1.1.1:10 RT:1001:11 
            Originator: 1.1.1.1, Cluster list: 7.7.7.7
            EVPN ESI: 0011.1111.1111.1111.1111
      RP/0/RP0/CPU0:Leaf-5#

In the next post, we are covering EVPN Integrated Routing and Bridging (IRB) configuration in detail.

Leave a Comment