View Cart
0 Items | Total: US$0.00
Welcome,      Register

You are here

Spanning Tree Priority on Nexus vPC+ and Fabricpath

Rating: 
3
Average: 3 (1 vote)

Cisco FabricpathIf you are familiar with Nexus vPC configuration, you might have been setting different STP priority on the primary and secondary switches so the primary is always a STP root, and have that lined up with, for example, HSRP active node. With vPC+ (ie. running vPC on a pair of switches that participate in fabricpath), the two Nexus switches appear as a single logical switch to both fabricpath cloud and upstream/downstream vPC switches, so it is actually crucial to make sure the upstream/downstream vPC switches receive consistent STP root priority regardless of which path is active. In this article, we demonstrate the importance of setting identical STP priority on the vPC+ peer switches, how the switches react when a superior BPDU is received, and other implications using Cisco Nexus 5000.

Below are a diagram, switch vPC configuration, and show-command outputs when everything is configured properly. Here the downstream switch ACESS-SW1 is connecting via port-channel 1 to the Nexus switches. Vlan 2 and 10 are in mode fabricpath, while vlan 20 and 21 are Classical Ethernet. A few things to note are:

  • Both Nexus switches uses a virtual MAC address (c84c.75ec.8000) as their STP ID instead of the respective VLAN interface MAC address (547f.eeaf.78dc and 547f.eeaf.97ca) for VLAN that are in mode fabricpath
  • Nexus switches continue to use the actual VLAN interface MAC address for the Classical Ethernet VLAN.
  • Both Nexus switches claim to be STP root
  • There is no STP running over peer-link as they are configured as ‘switchport mode fabricpath’, while Po1 still runs STP
  • Both Nexus switches require identical fabricpath (virtual) switch ID, which is advertised to the fabricpath cloud along with their individual fabricpath switch ID

Nexus vPC+ fabricpath

 

************** Configuration **************
!--------- NEXUS-SW1 ---------
spanning-tree vlan 1-4093 priority 8192
!
vpc domain 1
  role priority 8192
  system-priority 8192
  peer-keepalive destination 192.168.0.1 source 192.168.0.2 vrf KEEPALIVE
  fabricpath switch-id 1
!
!--------- NEXUS-SW2 --------
spanning-tree vlan 1-4093 priority 8192
!
vpc domain 1
  role priority 16384
  system-priority 8192
  peer-keepalive destination 192.168.0.2 source 192.168.0.1 vrf KEEPALIVE
  fabricpath switch-id 1
!
************** Show Outputs **************
!--------- NEXUS-SW1 ---------
NEXUS-SW1# sh int vl 10
Vlan10 is up, line protocol is up
  Hardware is EtherSVI, address is  547f.eeaf.78dc
  Description: *** VLAN10 ***
  Internet Address is 172.16.10.2/24
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec

NEXUS-SW1# sh spann root

                                        Root  Hello Max Fwd
Vlan                   Root ID          Cost  Time  Age Dly  Root Port
---------------- -------------------- ------- ----- --- ---  ----------------
VLAN0002          8194 c84c.75ec.8000       0    2   20  15  This bridge is root <-- Virtual MAC
VLAN0010          8202 c84c.75ec.8000       0    2   20  15  This bridge is root
VLAN0020          8212 547f.eeaf.78dc       0    2   20  15  This bridge is root <-- Real MAC
!
NEXUS-SW1# sh spann vl 10

VLAN0010
  Spanning tree enabled protocol rstp
  Root ID    Priority    8202
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8202   (priority 8192 sys-id-ext 10)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1              Desg FWD 1         128.4096 (vPC) P2p    <-- No STP over Peer-Link
!
!
NEXUS-SW1# sh fabricpath switch-id
                        FABRICPATH SWITCH-ID TABLE
Legend: '*' - this system
=========================================================================
SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED
----------+----------------+------------+-----------+--------------------
1           547f.eeaf.97ca    Primary     Confirmed     No      Yes    <-- Virtual Switch ID
1           547f.eeaf.78dc    Primary     Confirmed     No      Yes    
<-- Virtual Switch ID
*2         547f.eeaf.78dc    Primary     Confirmed    Yes     No       <-- Individual Switch ID
3         547f.eeaf.97ca    Primary     Confirmed     Yes     No     

 

!--------- NEXUS-SW2 --------
NEXUS-SW2# sh int vl 10
Vlan10 is up, line protocol is up
  Hardware is EtherSVI, address is  547f.eeaf.97ca
  Description: *** VLAN10 ***
  Internet Address is 172.16.10.3/24
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
!
NEXUS-SW2# sh spann roo
                                        Root  Hello Max Fwd
Vlan                   Root ID          Cost  Time  Age Dly  Root Port
---------------- -------------------- ------- ----- --- ---  ----------------
VLAN0002          8194 c84c.75ec.8000       0    2   20  15  This bridge is root
<-- Virtual MAC
VLAN0010          8202 c84c.75ec.8000       0    2   20  15  This bridge is root
VLAN0021          8213 547f.eeaf.97ca       0    2   20  15  This bridge is root 
<-- Real MAC

NEXUS-SW2# sh spann vl 10

VLAN0010
  Spanning tree enabled protocol rstp
  Root ID    Priority    8202
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8202   (priority 8192 sys-id-ext 10)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1              Desg FWD 1         128.4096 (vPC) P2p   
<-- No STP over Peer-Link

Scenario 1: Better BPDU received from downstream vPC switch

When we configure ACCESS-SW1 with better STP priority on VLAN 10, the followings were observed.

  • Both Nexus switches becomes blocking with error ‘*L2GW_Inc’ on interface Po1 VLAN 10
  • After restoring the STP priority on ACCESS-SW1, the Nexus switches transition to ‘*LOOP_Inc
  • Po1 on the ACCESS-SW1 side needs to ‘shut’ and ‘no shut’ for VLAN 10 STP on the Nexus to return to forwarding state

This behavior is a result of the requirement that vPC+ peer switches need to be STP root on all VLAN that are in mode fabricpath.

************** Configuration **************
!-------- ACCESS-SW1 ------
spanning-tree vlan 10 priority 4096
!
************** Show Outputs **************
!--------- NEXUS-SW1 ---------
NEXUS-SW1#  sh spann vl 10

VLAN0010
  Spanning tree enabled protocol rstp
  Root ID    Priority    8202
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8202   (priority 8192 sys-id-ext 10)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1              Desg BKN*1         128.4096 (vPC) P2p *L2GW_Inc

NEXUS-SW1#  sh spann vl 10

VLAN0010
  Spanning tree enabled protocol rstp
  Root ID    Priority    8202
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8202   (priority 8192 sys-id-ext 10)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1              Desg BKN*1         128.4096 (vPC) P2p *LOOP_Inc

 

Scenario 2: Dual-homed downstream switch with no vPC

Here we have ACCESS-SW2 that is dual-connected to both Nexus but no running etherchannel. We then change the STP priority on NEXUS-SW2 to 16384. The followings were observed.

  • Ethernet1/2 on NEXUS-SW2 changes STP state to blocking with error ‘*L2GW_Inc

Because ACCESS-SW2 is not dual-connected with vPC, the BPDU it receives from NEXUS-SW1 was accepted on Te1/1/1 (as it has priority of 8192) and advertised out Te1/1/2 to NEXUS-SW2. Since NEXUS-SW2 is running vPC+, it would not accept any superior BPDU, hence putting the interface in STP blocking.

************** Before Changes **************
!--------- NEXUS-SW1 ---------
NEXUS-SW1# sh spann vl 2

VLAN0002
  Spanning tree enabled protocol rstp
  Root ID    Priority    8194
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8194   (priority 8192 sys-id-ext 2)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/2          Desg FWD 2         128.141  P2p

!--------- NEXUS-SW2 ---------
NEXUS-SW2# sh spann vl 2

VLAN0002
  Spanning tree enabled protocol rstp
  Root ID    Priority    8194
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    8194   (priority 8192 sys-id-ext 2)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/2          Desg FWD 2         128.141  P2p

!---------- ACCESS-SW2 ----------------
ACCESS-SW2#sh spann vlan 2

VLAN0002
  Spanning tree enabled protocol rstp
  Root ID    Priority    8194
             Address     c84c.75ec.8000
             Cost        2
             Port        53 (TenGigabitEthernet1/1/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32770  (priority 32768 sys-id-ext 2)
             Address     30f7.0d4e.56fa
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/1/1             Root FWD 2         128.53   P2p
Te1/1/2             Altn BLK 2         128.54   P2p

************** Configuration **************
!--------- NEXUS-SW2 ---------
spann vlan 2 prio 16384
!
************** After Changes **************
!--------- NEXUS-SW2 ---------
NEXUS-SW2# sh spann vl 2

VLAN0002
  Spanning tree enabled protocol rstp
  Root ID    Priority    16386
             Address     c84c.75ec.8000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    16386  (priority 16384 sys-id-ext 2)
             Address     c84c.75ec.8000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth1/2          Desg BKN*2         128.141  P2p *L2GW_Inc

 

Scenario 3: Controlling active link for dual-homed non-vpc switch

Now that the Nexus switches appear as a single switch, with everything in BPDU being equal, by default, ACCESS-SW2 would break the tie by STP port ID. If you want to have control over which link is forwarding, you can define STP port-priority on the NEXUS-SW1 or NEXUS-SW2 interface Ethernet1/2 and the one with lower priority will cause the corresponding interface on ACCESS-SW2 to forward.
Here, interface Te1/1/1 is forwarding by default. We will lower the port-priority on Ethernet 1/2 of NEXUS-SW2 down to 64 (default is 128). Te1/1/2 subsequently becomes forwarding and Te1/1/1 went into blocking.

************** Configuration **************
!--------- NEXUS-SW2 ---------
int e1/2
 spanning-tree port-priority 64
!
************** Show Outputs **************
!----------ACCESS-SW2 ----------------
ACCESS-SW2#sh spann vl 2

VLAN0002
  Spanning tree enabled protocol rstp
  Root ID    Priority    8194
             Address     c84c.75ec.8000
             Cost        2
             Port        54 (TenGigabitEthernet1/1/2)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32770  (priority 32768 sys-id-ext 2)
             Address     30f7.0d4e.56fa
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/1/1             Altn BLK 2         128.53   P2p
Te1/1/2             Root FWD 2         128.54   P2p
!

References: Cisco Nexus 5000 Series NX-OS FabricPath Configuration Guide, Release 5.1(3)N1(1)
 

About Author

Metha Chiewanichakorn, CCIE#23585 (Ent. Infra, Sec, SP), is a Cisco networking enthusiast with years of experience in the industry. He is currently working as a consulting engineer for a Cisco partner. As a founder of and an instructor at labminutes.com, Metha enjoys learning and challenges himself with new technologies.