Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

Problem in migration from PVST to MSTP

Dears

I'm facing a problem during spanning-tree migration on our network from PVST to MSTP . I get such below error . 

 

%SPANTREE-2-PVSTSIM_FAIL: Blocking root port Fa0/1: Inconsitent inferior PVST 
BPDU received on VLAN 2, claiming root 12290:0022.0dba.9d00
SW2#show spanning-tree
MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    8193
             Address     0022.0dba.9d00
             Cost        200000
             Port        3 (FastEthernet0/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
  Bridge ID  Priority    12288  (priority 12288 sys-id-ext 0)
             Address     0022.916d.5380
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Fa0/1               Root BKN*200000    128.3    P2p Bound(PVST) *PVST_Inc

Thanks in advance .

 

2 ACCEPTED SOLUTIONS

Accepted Solutions
Cisco Employee

Hi,You are facing a

Hi,

You are facing a relatively common problem related to so-called PVST+ Simulation. This is a feature on Cisco switches whose purpose is to allow for interoperation of MST and PVST+ regions. However, there are certain requirements for this interoperation to successfully take place. Going into exact details would probably be too extensive at this point; suffice it to say that when interworking PVST+ and MST, only two representants of PVST+ and MST really speak to each other: the PVST+ instance for VLAN1 on the behalf of the entire PVST+ region, and MSTI 0 on the behalf of the entire MST region. These two instances must arrive at a conclusion (determining port roles and states) that will be adopted both by all other MST instances (this is given by MST design) and by all PVST+ instances (this is the purpose of the PVST+ Simulation). This can only be accomplished if one of the following two conditions is met:

  • The root switch for all VLANs in the PVST+ region is located itself in the MST region. This can be accomplished by setting the priority of the root bridge for the MSTI 0 in the MST region to a lower value than the bridge priorities of all switches in the PVST+ region in all VLANs. This is the recommended way of running a mixed MST/PVST+ network. As an example of accomplishing this, perform the following steps:
    • On a PVST+ switch, issue the show spanning-tree root command. In the "Root ID" column, note the lowest shown priority among all VLANs (check the entire output and choose the lowest displayed priority). Afterwards, make sure that the MST root switch for MSTI 0 is configured with a priority lower than the lowest priority displayed in the show command described previously.
  • Root switches for all VLANs in the PVST+ region are located in the PVST+ region. This can be accomplished by setting the priority of the root switch for VLAN1 in the PVST+ region to a lower value than the priority of any MST switch in the MSTI 0 instance, and in addition, setting the priority of root switches for VLANs other than VLAN 1 in the PVST+ region to a value even lower than the priority of the VLAN1 root switch. As an example of accomplishing this, perform the following steps:
    • Whatever priorities you configure on MST switches in MSTI 0, keep them at 24576 or above. In the PVST+ region, configure the root bridge for VLAN 1 with a priority of 8192, and configure root bridges for VLANs 2, 3, ... with a priority of 4096.

The situation you are experiencing currently is that the VLAN1 root switch in the PVST+ region has a lower priority than any of your MST switches in MSTI 0 - according to your output, it is 8192, and thus becomes the root switch for MSTI 0 as well. However, the root switch for VLAN 2 has a priority of 12288, which is not even less than 8192, and this causes the PVST+ Simulation to fail and block the port. Depending on what is your network requirement, you can solve this in two ways: either you configure one of your MST switches in MSTI0 with a priority of 0 or 4096 to become the root of the entire PVST+ region for all VLANs (recommended), or you decrease the priority of root switches in VLANs 2, 3, 4... to 4096 or 0.

Feel welcome to ask further!

Best regards,
Peter

Cisco Employee

Hi Brian,I believe I know

Hi Brian,

I believe I know what is happening but the explanation is quite involved so please follow me very carefully.

You did not post a topology diagram but from the outputs you have posted, it appears that both GH.TEA.ROOM and GH.BOOM.GATE are directly connected via a Port-channel link to the HP switch that is the root switch for MSTI 0.

There are two very important observations you will need to keep in mind. First, the HP switch does not support the PVST Simulation mechanism. It speaks MST only, reverting to non-VLAN-aware RSTP or STP on boundary ports, and that's it. Even if it receives PVST+ BPDUs, it will not start any specific PVST+ interoperability mechanism, as opposed to Cisco Catalyst switches. An important consequence is that when the HP switch running MST is connected to a Cisco switch running PVST+, the MSTI 0 instance on the HP will communicate with the VLAN 1 PVST+ instance but other PVST+ instances will not see the HP switch at all. This is because the HP switch does not originate PVST+ BPDUs to be able to speak to individual PVST+ instances on the Cisco switch, and instead, on a boundary port, it reverts to sending and receiving plain (R)STP BPDUs which are always processed in VLAN 1 PVST+ instance on a Cisco switch. You can see this very nicely in the show spanning-tree root command on the GH.BOOM.GATE switch - for VLAN 1, the switch recognizes the HP as the root switch. For other VLANs, it considers itself to be the root switch (as there is no root port identified in the output).

Second, to the HP switch, all PVST+ BPDUs are just multicast frames. It does not process them as BPDUs, rather, it only floods them in their respective VLANs. For PVST+ switches interconnected by the HP switch, the HP simply "isn't there". Again, this can be seen nicely on the show spanning-tree root command on the GH.TEA.ROOM switch - notice that this switch recognizes the HP as the root switch in VLAN 1, but in other VLANs, it considers the GH.BOOM.GATE switch to be the root switch, simply ignoring the HP switch that sits between these two Cisco switches.

Now, when you configure the GH.BOOM.GATE to run MST while leaving GH.TEA.ROOM run PVST+, GH.BOOM.GATE will consider its Po1 to be the root port in MSTI 0 because, obviously, it receives superior BPDUs sourced from the HP switch. So, the Po1 is the root port on GH.BOOM.GATE. However, because GH.TEA.ROOM still runs PVST+, its PVST+ BPDUs are simply flooded across the HP switch and arrive at the Po1 port on the GH.BOOM.GATE. So this root port on GH.BOOM.GATE also becomes a boundary port.

For an MST boundary port to be a consistent root port on Cisco switches, the consistency rule states: If an MST boundary port becomes a root port for MSTI 0 (obviously because it receives superior BPDUs, either MST BPDUs or VLAN 1 PVST+ BPDUs), it must guaranteedly be a root port towards all PVST+ instances out there. This is verified by looking at PVST+ BPDUs for VLANs 2 and higher, and making sure they are identical or even superior to the MST/VLAN1 BPDUs that caused the boundary port to become the root port for MSTI 0 in the first place.

However, note that this rule can never be met in your network! Because the HP switch does not perform PVST Simulation to speak to PVST+ switches in individual per-VLAN PVST+ instances, GH.TEA.ROOM can never learn about the HP as the root switch in VLANs 2 and higher. Because GH.BOOM.GATE considers its Po1 port as the root boundary port, it is not going to replicate the MSTI 0 data into per-VLAN PVST+ BPDUs (this would be only done on designated boundary ports, not on root boundary ports), and instead, it is just going to check for all received PVST+ BPDUs to meet the consistency check described earlier. So GH.TEA.ROOM effectively stops hearing any PVST+ BPDUs whatsoever, and will soon start considering itself as the root switch for VLAN 2 and higher and start sending its own PVST+ BPDUs. Obviously, these are inferior to the BPDUs originated by the HP switch that made GH.BOOM.GATE choose its Po1 port as the root port, but GH.TEA.ROOM has no way of knowing that. As a result, GH.BOOM.GATE is truly receiving PVST+ BPDUs that are inferior to the MSTI 0 BPDU received from the HP switch, failing the consistency check and causing the Po1 become blocked.

Interestingly, in your show logging output, the inferior BPDU reported by GH.BOOM.GATE actually advertises GH.BOOM.GATE itself as the root switch in VLAN 4. I tend to believe that this BPDU was probably a Topology Change BPDU during the time when GH.TEA.ROOM still considered GH.BOOM.GATE a root switch for non-VLAN1 PVST+ instances (were you running Rapid PVST+?). In any case, within 20 seconds at most, GH.TEA.ROOM should have proceeded to treat itself as the root switch in non-VLAN1 PVST+ instances and this inferior BPDU should have been reported on GH.BOOM.GATE.

There is no simple solution to this problem. One way of solving this problem would be to set the HP MSTI 0 priority to 4096, and before connecting a switch running PVST+, leave its VLAN 1 priority at 32,768 while setting the priority for all other VLANs to 0. This will make the PVST Simulation on Cisco switches happy while keeping the HP as the root switch for MSTI 0.

Another solution would be to configure the HP switch to entirely block the PVST+ BPDUs. A loop-free topology will still be maintained in the network thanks to the interoperation of VLAN 1 IEEE STP and MSTI 0 while preventing the PVST+ BPDUs to wreak havoc with the PVST Simulation. I do not know, however, if the HP switch is capable of filtering frames based on their destination multicast MAC address. If it is, filtering the frames destined to 01:00:0c:cc:cc:cd would do the trick.

I hope this helps but please feel welcome to ask further!

Best regards,
Peter

24 REPLIES
New Member

Any help would be appreciated

Any help would be appreciated ...

Cisco Employee

Hi,You are facing a

Hi,

You are facing a relatively common problem related to so-called PVST+ Simulation. This is a feature on Cisco switches whose purpose is to allow for interoperation of MST and PVST+ regions. However, there are certain requirements for this interoperation to successfully take place. Going into exact details would probably be too extensive at this point; suffice it to say that when interworking PVST+ and MST, only two representants of PVST+ and MST really speak to each other: the PVST+ instance for VLAN1 on the behalf of the entire PVST+ region, and MSTI 0 on the behalf of the entire MST region. These two instances must arrive at a conclusion (determining port roles and states) that will be adopted both by all other MST instances (this is given by MST design) and by all PVST+ instances (this is the purpose of the PVST+ Simulation). This can only be accomplished if one of the following two conditions is met:

  • The root switch for all VLANs in the PVST+ region is located itself in the MST region. This can be accomplished by setting the priority of the root bridge for the MSTI 0 in the MST region to a lower value than the bridge priorities of all switches in the PVST+ region in all VLANs. This is the recommended way of running a mixed MST/PVST+ network. As an example of accomplishing this, perform the following steps:
    • On a PVST+ switch, issue the show spanning-tree root command. In the "Root ID" column, note the lowest shown priority among all VLANs (check the entire output and choose the lowest displayed priority). Afterwards, make sure that the MST root switch for MSTI 0 is configured with a priority lower than the lowest priority displayed in the show command described previously.
  • Root switches for all VLANs in the PVST+ region are located in the PVST+ region. This can be accomplished by setting the priority of the root switch for VLAN1 in the PVST+ region to a lower value than the priority of any MST switch in the MSTI 0 instance, and in addition, setting the priority of root switches for VLANs other than VLAN 1 in the PVST+ region to a value even lower than the priority of the VLAN1 root switch. As an example of accomplishing this, perform the following steps:
    • Whatever priorities you configure on MST switches in MSTI 0, keep them at 24576 or above. In the PVST+ region, configure the root bridge for VLAN 1 with a priority of 8192, and configure root bridges for VLANs 2, 3, ... with a priority of 4096.

The situation you are experiencing currently is that the VLAN1 root switch in the PVST+ region has a lower priority than any of your MST switches in MSTI 0 - according to your output, it is 8192, and thus becomes the root switch for MSTI 0 as well. However, the root switch for VLAN 2 has a priority of 12288, which is not even less than 8192, and this causes the PVST+ Simulation to fail and block the port. Depending on what is your network requirement, you can solve this in two ways: either you configure one of your MST switches in MSTI0 with a priority of 0 or 4096 to become the root of the entire PVST+ region for all VLANs (recommended), or you decrease the priority of root switches in VLANs 2, 3, 4... to 4096 or 0.

Feel welcome to ask further!

Best regards,
Peter

Cisco Employee

Wonderfull Explanation Peter!

Wonderfull Explanation Peter! I am the true follower of you believe me :-)

 

Majed,

As peter explain we have also documented same on this by one of my collegue which has clear explanation with example how to achieve this. Please find the link below:

http://www.cisco.com/c/en/us/support/docs/lan-switching/multiple-instance-stp-mistp-8021s/116464-configure-pvst-00.html

 

HTH

Regards

Inayath

 

Cisco Employee

Hi Inayath,I am the true

Hi Inayath,

I am the true follower of you believe me :-)

I do not really know what to say. I am honored beyond words. I am very sincere here.

Yes, Aninda Chatterjee authored the TAC document you have referenced. We have in fact been in contact regarding that document. It is a very good explanation indeed.

Best regards,
Peter

Helloexcellent post peter

Hello

excellent post peter!

enjoyed the read!

 

res

paul

Please don't forget to rate any posts that have been helpful. Thanks.
New Member

Hi Peter, I have a similar

Hi Peter, I have a similar but slightly different problem that I would like your thoughts on the matter. 

I am working on a network that is comprised mainly of HP Comware switches running MST instance 0 for all VLANs. 

I have approx 12 Cisco switches also running in MST spanning-tree instance 0 configuration for all VLANs. 

The Root bridge on the Cisco switches are the HP Core L3 switch, which is correct, as this has been configured as the Primary Root for all VLANs (Priority 0)

The problem I face, is that if a new Cisco switch configured with PVST connects to the network all my Cisco switches uplinks to the Core go offline. In the logs I see examples of the following:

SPANTREE-2-PVSTSIM_FAIL: Blocking root port Po1: Inconsistent Inferior PVST BPDU received on VLAN 4, claiming root 32772:mac address

If the MST tree already has a Root for instance 0; why would only the Cisco switches shutdown the interface when receiving an inferior PVST BPDU. Should it not ignore this packet and maintain its current Root?

Cisco Employee

Hi Brian,What you say

Hi Brian,

What you say definitely makes sense: Once the MST contains the root bridge for the entire CIST (meaning its MSTI 0 priority is lower than any PVST+ switch' priority in any VLAN) then logically, all received PVST+ BPDUs on boundary port must be inferior. There is not point in blocking the port.

What the message above appears to say is that on Po1,

  • a PVST+ BPDU for VLAN 1 was received that actually advertised a better root bridge for VLAN 1 than the current MSTI 0 root bridge
  • at the same time, another PVST+ BPDU for VLAN 4 was received according that was inferior to the VLAN 1's PVST+ BPDU.

In other words, it would suggest that the attached Cisco switch was configured with a low VLAN 1 priority but left with priorities in all other VLANs intact. Would that actually be the case? Can that be possible?

By the way, to which switch does the MAC address in the SPANTREE-2-PVSTSIM_FAIL message belong? Is it the newly added switch?

Can you perhaps post the output of show spanning-tree root from any currently working Cisco Catalyst switch in your network?

Looking forward to hearing from you?

Best regards,
Peter

New Member

Hi Peter, apologies for the

Hi Peter, apologies for the delay. Took me a while to get access to this. Here is the requested output.

 

Cisco Sw1

GH.TEA.ROOM#show spanning-tree root

                                        Root    Hello Max Fwd
Vlan                   Root ID          Cost    Time  Age Dly  Root Port
---------------- -------------------- --------- ----- --- ---  ------------
VLAN0001             0 3ce5.a6d7.07c0         3    2   20  15  Po1
VLAN0004         32772 4c00.82c1.6980         3    2   20  15  Po1
VLAN0008         32776 4c00.82c1.6980         3    2   20  15  Po1
VLAN0012         32780 4c00.82c1.6980         3    2   20  15  Po1
VLAN0016         32784 4c00.82c1.6980         3    2   20  15  Po1
VLAN0020         32788 4c00.82c1.6980         3    2   20  15  Po1
VLAN0024         32792 4c00.82c1.6980         3    2   20  15  Po1

Cisco Sw2

GH.BOOM.GATE#show spanning-tree root

                                        Root    Hello Max Fwd
Vlan                   Root ID          Cost    Time  Age Dly  Root Port
---------------- -------------------- --------- ----- --- ---  ------------
VLAN0001             0 3ce5.a6d7.07c0         3    2   20  15  Po1
VLAN0004         32772 4c00.82c1.6980         0    2   20  15
VLAN0008         32776 4c00.82c1.6980         0    2   20  15
VLAN0012         32780 4c00.82c1.6980         0    2   20  15
VLAN0016         32784 4c00.82c1.6980         0    2   20  15
VLAN0020         32788 4c00.82c1.6980         0    2   20  15
VLAN0024         32792 4c00.82c1.6980         0    2   20  15


We change cisco sw2 into mst. This means we have now have 1 Cisco switch in the network in PVST (cisco sw1). The output below shows what happens to Cisco SW2. 

*Mar  1 12:26:11: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
*Mar  1 12:26:11: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan24, changed state to down
*Mar  1 12:26:13: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to up
*Mar  1 12:26:13: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan24, changed state to up
*Mar  1 12:26:14: %SPANTREE-2-PVSTSIM_FAIL: Blocking root port Po1: Inconsitent inferior PVST BPDU received on VLAN 4, claiming root 32772:4c00.82c1.6980
*Mar  1 12:26:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to down
*Mar  1 12:26:14: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan24, changed state to down$ne protocol on Interface Vlan1, changed state to down
*Mar  1 12:26:11: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed^ state to down

GH.BOOM.GATE#show spanning-tree 

MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    0
             Address     3ce5.a6d7.07c0
             Cost        10000
             Port        64 (Port-channel1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     4c00.82c1.6980
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Po1                 Root BKN*10000     128.64   P2p Bound(RSTP) *PVST_Inc 

 

GH.BOOM.GATE#show span root 

                                        Root    Hello Max Fwd
MST Instance           Root ID          Cost    Time  Age Dly  Root Port
---------------- -------------------- --------- ----- --- ---  ------------
MST0                 0 3ce5.a6d7.07c0     10000    2   20  15  Po1         

Then go an change Cisco sw1 and put it in mst. (so now we dont have any PVST switches in the network) Below is output from Cisco sw2 as soon as SW1 is changed to MST (the same thing happens if SW1 is removed from the network.) 

terminal out of sw2

*Mar  1 12:29:45: %SPANTREE-2-PVSTSIM_OK: PVST Simulation inconsistency cleared on port Port-channel1.
*Mar  1 12:29:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan1, changed state to up
*Mar  1 12:29:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan24, changed state to up

terminal out for "show stp root"

GH.TEA.ROOM#show spanning-tree root

                                        Root    Hello Max Fwd
MST Instance           Root ID          Cost    Time  Age Dly  Root Port
---------------- -------------------- --------- ----- --- ---  ------------
MST0                 0 3ce5.a6d7.07c0     10000    2   20  15  Po1
GH.TEA.ROOM#

The mac address 3ce5.a6d7.07c0 is the HP Core which is correct. 
 

Cisco Employee

Hi Brian,I believe I know

Hi Brian,

I believe I know what is happening but the explanation is quite involved so please follow me very carefully.

You did not post a topology diagram but from the outputs you have posted, it appears that both GH.TEA.ROOM and GH.BOOM.GATE are directly connected via a Port-channel link to the HP switch that is the root switch for MSTI 0.

There are two very important observations you will need to keep in mind. First, the HP switch does not support the PVST Simulation mechanism. It speaks MST only, reverting to non-VLAN-aware RSTP or STP on boundary ports, and that's it. Even if it receives PVST+ BPDUs, it will not start any specific PVST+ interoperability mechanism, as opposed to Cisco Catalyst switches. An important consequence is that when the HP switch running MST is connected to a Cisco switch running PVST+, the MSTI 0 instance on the HP will communicate with the VLAN 1 PVST+ instance but other PVST+ instances will not see the HP switch at all. This is because the HP switch does not originate PVST+ BPDUs to be able to speak to individual PVST+ instances on the Cisco switch, and instead, on a boundary port, it reverts to sending and receiving plain (R)STP BPDUs which are always processed in VLAN 1 PVST+ instance on a Cisco switch. You can see this very nicely in the show spanning-tree root command on the GH.BOOM.GATE switch - for VLAN 1, the switch recognizes the HP as the root switch. For other VLANs, it considers itself to be the root switch (as there is no root port identified in the output).

Second, to the HP switch, all PVST+ BPDUs are just multicast frames. It does not process them as BPDUs, rather, it only floods them in their respective VLANs. For PVST+ switches interconnected by the HP switch, the HP simply "isn't there". Again, this can be seen nicely on the show spanning-tree root command on the GH.TEA.ROOM switch - notice that this switch recognizes the HP as the root switch in VLAN 1, but in other VLANs, it considers the GH.BOOM.GATE switch to be the root switch, simply ignoring the HP switch that sits between these two Cisco switches.

Now, when you configure the GH.BOOM.GATE to run MST while leaving GH.TEA.ROOM run PVST+, GH.BOOM.GATE will consider its Po1 to be the root port in MSTI 0 because, obviously, it receives superior BPDUs sourced from the HP switch. So, the Po1 is the root port on GH.BOOM.GATE. However, because GH.TEA.ROOM still runs PVST+, its PVST+ BPDUs are simply flooded across the HP switch and arrive at the Po1 port on the GH.BOOM.GATE. So this root port on GH.BOOM.GATE also becomes a boundary port.

For an MST boundary port to be a consistent root port on Cisco switches, the consistency rule states: If an MST boundary port becomes a root port for MSTI 0 (obviously because it receives superior BPDUs, either MST BPDUs or VLAN 1 PVST+ BPDUs), it must guaranteedly be a root port towards all PVST+ instances out there. This is verified by looking at PVST+ BPDUs for VLANs 2 and higher, and making sure they are identical or even superior to the MST/VLAN1 BPDUs that caused the boundary port to become the root port for MSTI 0 in the first place.

However, note that this rule can never be met in your network! Because the HP switch does not perform PVST Simulation to speak to PVST+ switches in individual per-VLAN PVST+ instances, GH.TEA.ROOM can never learn about the HP as the root switch in VLANs 2 and higher. Because GH.BOOM.GATE considers its Po1 port as the root boundary port, it is not going to replicate the MSTI 0 data into per-VLAN PVST+ BPDUs (this would be only done on designated boundary ports, not on root boundary ports), and instead, it is just going to check for all received PVST+ BPDUs to meet the consistency check described earlier. So GH.TEA.ROOM effectively stops hearing any PVST+ BPDUs whatsoever, and will soon start considering itself as the root switch for VLAN 2 and higher and start sending its own PVST+ BPDUs. Obviously, these are inferior to the BPDUs originated by the HP switch that made GH.BOOM.GATE choose its Po1 port as the root port, but GH.TEA.ROOM has no way of knowing that. As a result, GH.BOOM.GATE is truly receiving PVST+ BPDUs that are inferior to the MSTI 0 BPDU received from the HP switch, failing the consistency check and causing the Po1 become blocked.

Interestingly, in your show logging output, the inferior BPDU reported by GH.BOOM.GATE actually advertises GH.BOOM.GATE itself as the root switch in VLAN 4. I tend to believe that this BPDU was probably a Topology Change BPDU during the time when GH.TEA.ROOM still considered GH.BOOM.GATE a root switch for non-VLAN1 PVST+ instances (were you running Rapid PVST+?). In any case, within 20 seconds at most, GH.TEA.ROOM should have proceeded to treat itself as the root switch in non-VLAN1 PVST+ instances and this inferior BPDU should have been reported on GH.BOOM.GATE.

There is no simple solution to this problem. One way of solving this problem would be to set the HP MSTI 0 priority to 4096, and before connecting a switch running PVST+, leave its VLAN 1 priority at 32,768 while setting the priority for all other VLANs to 0. This will make the PVST Simulation on Cisco switches happy while keeping the HP as the root switch for MSTI 0.

Another solution would be to configure the HP switch to entirely block the PVST+ BPDUs. A loop-free topology will still be maintained in the network thanks to the interoperation of VLAN 1 IEEE STP and MSTI 0 while preventing the PVST+ BPDUs to wreak havoc with the PVST Simulation. I do not know, however, if the HP switch is capable of filtering frames based on their destination multicast MAC address. If it is, filtering the frames destined to 01:00:0c:cc:cc:cd would do the trick.

I hope this helps but please feel welcome to ask further!

Best regards,
Peter

New Member

Thanks Peter, your analysis

Thanks Peter, your analysis of this problem is spot on. I would like to thank you for looking into this and providing me your thoughts as it has had me baffled for sometime. 

Looking at your proposed suggestions to solve this; I can almost be certain that the HP switches would not be able to filter based on destination MAC; but I will look into it. 

Modifying any future switches that are being connected to this network to ensure this doesn't happen again - although manageable - is not a position I want to leave this network in. Essentially this creates a vulnerability whereby anyone with a default Cisco switch can power it up and connect it to the network and essentially bring down any Cisco MST configured switches. This is something that would keep me awake at night. 

The topology is simple, Hub-and-spoke. Each edge switch (HP or Cisco) are connected to the Core via port-channel. Probably the simplest topology one can think of. I need to look at something that will "permanently" solve this problem, to ensure it cannot happen again.

Is PVSTSIM something that can be turned off? I have looked but cant see anyway to do this - possibly a magical command Cisco TAC could provide me with.

I am even entertaining the idea of turning off STP on the uplink ports on the Cisco switches, just to avoid this from possibly happening. They are Port-Channel interfaces so theoretically the ports on the Core would have to have been provisioned accordingly, thus a loop on these ports should never exist. But the thought of doing this I suspect will also keep me awake all night. 

Do you have any other suggestions that I could look at or entertain that could potentially resolve this thing for the long term. If I could turn off/disable the PVSTSIM that would solve it permanently.  

Again I appreciate your assistance and value your feedback. 

 

Cisco Employee

Hi Brian,You are welcome

Hi Brian,

You are welcome.

Regarding deactivating the PVST Simulation, I am sorry to say that I do not know of any way of truly accomplishing that. On selected platforms, such as Catalyst 6500, it is actually possible to use the no spanning-tree mst simulate pvst global to stop the switch perform the PVST Simulation (sending and processing PVST+ BPDUs on boundary ports receiving PVST+ BPDUs from other switches), but if such a switch still receives PVST+ BPDUs, the port gets blocked automatically without any consistency checks.

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-2SX/configuration/guide/book/stp_enha.html#wp1054786

The disadvantage is that on common access-level Catalysts (2960, 3560, 3750, ...), this feature is not available. There is no way of deactivating the PVST Simulation on these switches. Even if there was one, however, you have to note that this solution is analogous to the suggestion of cleverly modifying PVST+ STP priorities - you will have to configure at least something on a newly attached switch to be safe and sure that the problems do not happen. This is not really helpful.

What exact type of HP core switch are you using? I can have a look into its documentation to see if there is any tool we could use.

Theoretically, you could configure your existing Catalyst switches running MST to ignore PVST+ BPDUs on the Port-channel ports toward the HP switch using MAC ACLs - although tedious and not exactly by-the-book, it could at least prevent them from blocking their ports if a new, unconfigured Catalyst switch is connected to your network. Please be cautious that blocking STP is always a very dangerous thing to do, and I am suggesting this only because I know that all your current switches are running MST and are thereby prevented against loops regardless of PVST+ being run, and that your network is constructed in a hub-and-spoke fashion.

I am not happy to say this but I have to: The PVST Simulation mechanism is a well-meant mechanism but it ultimately proves to be much more troublesome than helpful.

Best regards,
Peter

New Member

Yes, it certainly seems that

Yes, it certainly seems that this "helpful" feature on the Cisco's is causing a serious issue/vulnerability. 

The HP core comprises of 2x HP 7506 switches in an IRF "virtual stack" configuration running Comware OS. 

I am surprised that I have stumbled across this issue. One would think that a simple hub-and-spoke with multi-vendor interop would have been done a million times over and this issue would have raised its head before, and a suitable fix be available. We must be on the bleeding edge here :)

I am thinking it would probably just be simpler to enable MST on all the existing Cisco switches int he network and if any further switches that go into the network also need to be configured with MST. This should avoid any PVST BPDU's being on the network and wreaking havoc. 

Again, although this solves the issue. There still remains a vulnerability that anyone can connect a default Cisco switch which will shutdown the rest of them. 

 

Cisco Employee

Hi Brian,One would think that

Hi Brian,

One would think that a simple hub-and-spoke with multi-vendor interop would have been done a million times

I would believe it was indeed done a million times - but in all those cases, all switches in a switched network already ran MST so the PVST Simulation was never triggered. Mixing different STP versions in a single network is always discouraged even though at least the vanilla IEEE versions are backward compatible.

I am thinking it would probably just be simpler to enable MST on all the existing Cisco switches int he network and if any further switches that go into the network also need to be configured with MST.

This is the proper way it should be done, and I am 100% in agreement here. All your switches shall be properly configured with MST before being plugged into network.

Allow me to be more philosophical than technical at this point: Notice that you are trying to make your network resilient to errors such as plugging in an unconfigured Cisco switch that causes mayhem with its PVST+. While that is a good intention, and if it was possible I would go for it, there is another point worth noticing: Are you actually allowing devices to be plugged into your network without properly inspecting their configuration beforehand? You see, the PVST+ is by far not the only problem you may be facing if you allow a switch to be connected to your network without having very strict procedures on validating its configuration beforehand. To put it plainly, I would impose some very strict rules on how a new switch should be configured and who is responsible for signing off that configuration before the switch can be plugged into the network. Having a network able to cope graciously with newly attached switches is definitely an advantage but first and foremost, the switches should not be attached to it like that - with no appropriate configuration applied.

Oh, by the way, it seems that your HP 7500 does support filtering the traffic based on MAC addresses! It should be something along:

system-view
acl number 4000
rule deny dest-mac 0100.0ccc.cccd 0000.0000.0000
rule permit

interface ...
packet-filter 4000 inbound

 

I am just blind-shooting this configuration here based on the configuration to your switch series but definitely, it seems that this kind of configuration should actually be supported. Would you mind giving it a try?

Best regards,
Peter

New Member

Thanks Peter, and I

Thanks Peter, and I completely agree with you. No one should be able to simply plug a switch into the network without being correctly authorized and/or configured. Although not probable, this is however still possible. And 4 years down the track, someone may forget about all of this, and simply plug a Cisco switch in etc. This is the risk I am trying to mitigate here. 

I intend to be back onsite later in the week and will attempt your HP configuration to block the PVST+ BPDU's by destination MAC. I would prefer to be applying this restriction on the existing Cisco switches Portchannel interfaces rather.

You see, If I configure this ACL on the HP switch and it works, I would essentially need to apply this ACL on each and every interface to be entirely safe, rather than addressing the devices that are susceptible to this, the Cisco switches. Are you aware of Cisco IOS being able to filter and block based on destination MAC? i.e configure the same block filter and apply it to the ingress of the portchannel interfaces? This would enable the Cisco switches to drop the PVST packets before they are processed and prevent PVSTSIM from ever having to initiate. This would solve the problem. Then I could happily configure all of them to MST. 

Cisco Employee

Hi Brian,Yes, filtering the

Hi Brian,

Yes, filtering the ingress PVST+ frames on Cisco Catalyst switches is possible. You'd do it this way:

 

mac access-list extended NoPVST
 deny   any host 0100.0ccc.cccd
 permit any any
!
interface ...
 mac access-group NoPVST in

 

On my Catalyst 2960 and 3560 where I tested this, the mac access-group command was not supported on a Port-channel interface. Instead, it was to be placed on all physical ports that make up the single Port-channel. After doing this, I have been indeed able to filter out the incoming PVST+ BPDUs while leaving the IEEE BPDUs operational. This could be used on all your existing Catalysts to prevent them from triggering PVST Simulation even if they received a PVST+ BPDU.

Best regards,
Peter

New Member

Peter, just an update on this

Peter, just an update on this. I made the changes and applied it to the physical interfaces of the Cisco switches. It successfully blocked PVST packets and ultimately ensured that PVSTSIM did not activate. I moved them all into MST mode (one by one) and testing was successful.

I would like to thank you for your input on this - it truly was a great help. I will be sure to look you up the next time I am in Slovakia.

 

Cisco Employee

Brian,It was a thrill to have

Brian,

It was a thrill to have helped, and trust me, I've learned a lot here as well! Should you ever be around Slovakia please be sure to let me know - you are much welcome! Do you actually come to this world's corner from time to time?

Best regards,
Peter

New Member

No, I have never been to that

No, I have never been to that side of the globe, but it has always been on my radar. So on day when I find myself there I will certainly let you know and I will buy you a beer in appreciation for your assistance.

Until then.

Brian

New Member

Hi Peter,We have got similiar

Hi Peter,

We have got similiar issues which has annoyed us for 2 months. I was led to this great article while I searching the solution from Internet.

We deployed VCE Vblock 320 last year and all has been working properly until we tried to add a new VLAN in VBlock 2 months ago.

 

Vblock contains 2 Cisco Nexus 5548 which connected as vPV peer-link at port -channel 50. The port-channel 1 is formed by GE1/9, 1/10 ports on both N5K and connected to the trk7 on HP ProCurve 8212 core switch via 10G links. PO101 and PO102 connect to the FI of Cisco UCS.

HP core switch has been setup as MST as below

dm-hp8212a# sh spanning-tree          

 Multiple Spanning Tree (MST) Information

  STP Enabled   : Yes
  Force Version : MSTP-operation
  IST Mapped VLANs : 1-4094
  Switch MAC Address : c09134-afe200
  Switch Priority    : 0    
  Max Age  : 20
  Max Hops : 20   
  Forward Delay : 15

  Topology Change Count  : 4164        
  Time Since Last Change : 2 days      

  CST Root MAC Address : c09134-afe200
  CST Root Priority    : 0           
  CST Root Path Cost   : 0           
  CST Root Port        : This switch is root

  IST Regional Root MAC Address : c09134-afe200
  IST Regional Root Priority    : 0           
  IST Regional Root Path Cost   : 0           
  IST Remaining Hops            : 20          

  Root Guard Ports     :
  Loop Guard Ports     :
  TCN Guard Ports      :
  BPDU Protected Ports :                                         
  BPDU Filtered Ports  :                                         
  PVST Protected Ports :                                         
  PVST Filtered Ports  :                                         

                   |           Prio              | Designated    Hello         
  Port   Type      | Cost      rity State        | Bridge        Time PtP Edge
  ------ --------- + --------- ---- ------------ + ------------- ---- --- ----
  A1     100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes Yes
  A2     100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes Yes
  A3     100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes Yes
  A4     100/1000T | Auto      128  Disabled     |
  A5     100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes Yes
  A6     100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes Yes
  A7     100/1000T | Auto      128  Disabled     |
  A8     100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes Yes
  A9     100/1000T | Auto      128  Disabled     |
  A10    100/1000T | Auto      128  Disabled     |
  A11    100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes No  
  A12    100/1000T | 20000     128  Forwarding   | c09134-afe200 2    Yes No  
  A13    100/1000T | Auto      128  Disabled     |
  A14    100/1000T | Auto      128  Disabled     |
  A15    100/1000T | Auto      128  Disabled     |

And both N5K01 and N5K02 have been setup as MST as well to match the HP core switch.

We decided to add a new VLAN 46 to the Vblock and we did add to UCS, Nexus 1000v and then two N5K switches. The issue of disconnection between Vblock (N5K) and HP core occurred just after we finished the N5K01 and started to add VLAN 46 to PO50 at N5K02.  Here are the running-config spanning tree, commands we tried to run on both N5K switches and followed by spanning tree information BEFORE and AFTER changes:

Spanning-tree in running-config:

spanning-tree mode mst
spanning-tree port type edge bpduguard default
spanning-tree port type edge bpdufilter default
spanning-tree vlan 1-3967,4048-4093 priority 49152
spanning-tree mst configuration
  name c09134afe200

 

Commands we run:

conf t

vlan 46

name NumerisDMServer

interface port-channel50

switchport trunk allowed vlan add 46

interface port-channel101

switchport trunk allowed vlan add 46

interface port-channel102

switchport trunk allowed vlan add 46

 

interface port-channel1

switchport trunk allowed vlan add 46

 

Before Change:

dmvb-nx5k01# sh spanning-tree detail

 

 MST0000 is executing the mstp compatible Spanning Tree protocol

  Bridge Identifier has priority 32768, sysid 0, address 002a.6a26.4901

  Configured hello time 2, max age 20, forward delay 15

  Current root has priority 0, address c091.34af.e200

  Root port is 4096 (port-channel1), cost of root path is 200

  Topology change flag not set, detected flag not set

  Number of topology changes 31 last change occurred 132:50:39 ago

          from port-channel1

  Times:  hold 1, topology change 35, notification 2

          hello 2, max age 20, forward delay 15

  Timers: hello 0, topology change 0, notification 0

 

 Port 4096 (port-channel1, vPC) of MST0000 is root forwarding

   Port path cost 200, Port priority 128, Port Identifier 128.4096

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 0, address c091.34af.e200

   Designated port id is 64.296, designated path cost 0

   Timers: message age 4, forward delay 0, hold 0

   Number of transitions to forwarding state: 3

   Link type is point-to-point by default, Boundary RSTP

   PVST Simulation is enabled by default

   BPDU: sent 2, received 239141

 

 Port 4132 (port-channel37, vPC) of MST0000 is designated forwarding

   Port path cost 200, Port priority 128, Port Identifier 128.4132

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.4132, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 1

   Link type is point-to-point by default, Boundary PVST

   PVST Simulation is enabled by default (port is in PVST simulation mode)

   BPDU: sent 10039928, received 96

 

 Port 4133 (port-channel38, vPC) of MST0000 is designated forwarding

   Port path cost 200, Port priority 128, Port Identifier 128.4133

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.4133, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 1

   Link type is point-to-point by default, Boundary PVST

   PVST Simulation is enabled by default (port is in PVST simulation mode)

   BPDU: sent 10039929, received 64

 

 Port 4145 (port-channel50, vPC Peer-link) of MST0000 is designated forwarding

   Port path cost 1000, Port priority 128, Port Identifier 128.4145

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.4145, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 4

   The port type is network

   Link type is point-to-point by default, Internal

   PVST Simulation is enabled by default

   BPDU: sent 2008423, received 2008430

 

 Port 4196 (port-channel101, vPC) of MST0000 is designated forwarding

   Port path cost 200, Port priority 128, Port Identifier 128.4196

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.4196, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 6

   The port type is edge by port type edge trunk configuration

   Link type is point-to-point by default, Internal

   Bpdu guard is enabled by default

   Bpdu filter is enabled by default

   PVST Simulation is enabled by default

   BPDU: sent 11, received 0

 

 Port 4197 (port-channel102, vPC) of MST0000 is designated forwarding

   Port path cost 200, Port priority 128, Port Identifier 128.4197

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.4197, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 5

   The port type is edge by port type edge trunk configuration

   Link type is point-to-point by default, Internal

   Bpdu guard is enabled by default

   Bpdu filter is enabled by default

   PVST Simulation is enabled by default

   BPDU: sent 11, received 0

 

 Port 4297 (port-channel202, vPC) of MST0000 is designated forwarding

   Port path cost 200, Port priority 128, Port Identifier 128.4297

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.4297, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 3

   Link type is point-to-point by default, Internal

   Bpdu guard is enabled

   PVST Simulation is enabled by default

   BPDU: sent 2008002, received 0

 

 Port 151 (Ethernet1/23) of MST0000 is designated forwarding

   Port path cost 2000, Port priority 128, Port Identifier 128.151

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.151, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 3

   Link type is point-to-point by default, Internal

   PVST Simulation is enabled by default

   BPDU: sent 2008186, received 0

 

 Port 157 (Ethernet1/29) of MST0000 is designated forwarding

   Port path cost 20000, Port priority 128, Port Identifier 128.157

   Designated root has priority 0, address c091.34af.e200

   Designated bridge has priority 32768, address 002a.6a26.4901

   Designated port id is 128.157, designated path cost 200

   Timers: message age 0, forward delay 0, hold 0

   Number of transitions to forwarding state: 1

   The port type is edge

   Link type is point-to-point by default, Internal

   Bpdu guard is enabled by default

   Bpdu filter is enabled by default

   PVST Simulation is enabled by default

   BPDU: sent 11, received 0

 

 

 

 

 

dmvb-nx5k01# sh spanning-tree brief

 

MST0000

  Spanning tree enabled protocol mstp

  Root ID    Priority    0

             Address     c091.34af.e200

             Cost        200

             Port        4096 (port-channel1)

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

 

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)

             Address     002a.6a26.4901

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

 

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1              Root FWD 200       128.4096 (vPC) P2p Bound(RSTP)

Po37             Desg FWD 200       128.4132 (vPC) P2p Bound(PVST)

Po38             Desg FWD 200       128.4133 (vPC) P2p Bound(PVST)

Po50             Desg FWD 1000      128.4145 (vPC peer-link) Network P2p

Po101            Desg FWD 200       128.4196 (vPC) Edge P2p

Po102            Desg FWD 200       128.4197 (vPC) Edge P2p

Po202            Desg FWD 200       128.4297 (vPC) P2p

Eth1/23          Desg FWD 2000      128.151  P2p

Eth1/29          Desg FWD 20000     128.157  Edge P2p

 

dmvb-nx5k01#

 

 

dmvb-nx5k01# sh spanning-tree

 

MST0000

  Spanning tree enabled protocol mstp

  Root ID    Priority    0

             Address     c091.34af.e200

             Cost        200

             Port        4096 (port-channel1)

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

 

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)

             Address     002a.6a26.4901

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

 

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1              Root FWD 200       128.4096 (vPC) P2p Bound(RSTP)

Po37             Desg FWD 200       128.4132 (vPC) P2p Bound(PVST)

Po38             Desg FWD 200       128.4133 (vPC) P2p Bound(PVST)

Po50             Desg FWD 1000      128.4145 (vPC peer-link) Network P2p

Po101            Desg FWD 200       128.4196 (vPC) Edge P2p

Po102            Desg FWD 200       128.4197 (vPC) Edge P2p

Po202            Desg FWD 200       128.4297 (vPC) P2p

Eth1/23          Desg FWD 2000      128.151  P2p

Eth1/29          Desg FWD 20000     128.157  Edge P2p

 

dmvb-nx5k01#

 

 

 

 

dmvb-nx5k01# sh spanning-tree mst

 

##### MST0    vlans mapped:   1-4094

Bridge        address 002a.6a26.4901  priority      32768 (32768 sysid 0)

Root          address c091.34af.e200  priority      0     (0 sysid 0)

              port    Po1             path cost     200     

Regional Root this switch

Operational   hello time 2 , forward delay 15, max age 20, txholdcount 6

Configured    hello time 2 , forward delay 15, max age 20, max hops    20

 

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1              Root FWD 200       128.4096 (vPC) P2p Bound(RSTP)

Po37             Desg FWD 200       128.4132 (vPC) P2p Bound(PVST)

Po38             Desg FWD 200       128.4133 (vPC) P2p Bound(PVST)

Po50             Desg FWD 1000      128.4145 (vPC peer-link) Network P2p

Po101            Desg FWD 200       128.4196 (vPC) Edge P2p

Po102            Desg FWD 200       128.4197 (vPC) Edge P2p

Po202            Desg FWD 200       128.4297 (vPC) P2p

Eth1/23          Desg FWD 2000      128.151  P2p

Eth1/29          Desg FWD 20000     128.157  Edge P2p

 

dmvb-nx5k01# sh spanning-tree mst configuration

Name      [c09134afe200]

Revision  0     Instances configured 1

Instance  Vlans mapped

--------  ---------------------------------------------------------------------

0         1-4094

-------------------------------------------------------------------------------

dmvb-nx5k01#

 

 

 

 

dmvb-nx5k01# sh spanning-tree summary

Switch is in mst mode (IEEE Standard)

Root bridge for: none

Port Type Default                        is disable

Edge Port [PortFast] BPDU Guard Default  is enabled

Edge Port [PortFast] BPDU Filter Default is enabled

Bridge Assurance                         is enabled

Loopguard Default                        is disabled

Pathcost method used                     is long

PVST Simulation                          is enabled

STP-Lite                                 is enabled

 

Name                   Blocking Listening Learning Forwarding STP Active

---------------------- -------- --------- -------- ---------- ----------

MST0000                      0         0        0          9          9

---------------------- -------- --------- -------- ---------- ----------

1 mst                        0         0        0          9          9

dmvb-nx5k01#

 

 

 

dmvb-nx5k01# sh spanning-tree active

 

MST0000

  Spanning tree enabled protocol mstp

  Root ID    Priority    0

             Address     c091.34af.e200

             Cost        200

             Port        4096 (port-channel1)

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

 

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)

             Address     002a.6a26.4901

             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

 

Interface        Role Sts Cost      Prio.Nbr Type

---------------- ---- --- --------- -------- --------------------------------

Po1              Root FWD 200       128.4096 (vPC) P2p Bound(RSTP)

Po37             Desg FWD 200       128.4132 (vPC) P2p Bound(PVST)

Po38             Desg FWD 200       128.4133 (vPC) P2p Bound(PVST)

Po50             Desg FWD 1000      128.4145 (vPC peer-link) Network P2p

Po101            Desg FWD 200       128.4196 (vPC) Edge P2p

Po102            Desg FWD 200       128.4197 (vPC) Edge P2p

Po202            Desg FWD 200       128.4297 (vPC) P2p

Eth1/23          Desg FWD 2000      128.151  P2p

Eth1/29          Desg FWD 20000     128.157  Edge P2p

 

dmvb-nx5k01#

dmvb-nx5k01#

dmvb-nx5k01# show spanning-tree mst configuration digest

Name      [c09134afe200]

Revision  0     Instances configured 1

Digest          0xac36177f50283cd4b83821d8ab26de62

Pre-std Digest  0xbb3b6c15ef8d089bb55ed10d24df44de

dmvb-nx5k01#

 

 

dm-hp8212a#

dm-hp8212a# sh spanning-tree mst-config

 

 MST Configuration Identifier Information

 

  MST Configuration Name : c09134afe200                   

  MST Configuration Revision : 0   

  MST Configuration Digest : 0xAC36177F50283CD4B83821D8AB26DE62

 

  IST Mapped VLANs : 1-4094

 

  Instance ID Mapped VLANs                                            

  ----------- ---------------------------------------------------------

 

dm-hp8212a#

 

 

 

 

After Change

`show spanning-tree summary`
Switch is in mst mode (IEEE Standard)
Root bridge for: none
Port Type Default                        is disable
Edge Port [PortFast] BPDU Guard Default  is enabled
Edge Port [PortFast] BPDU Filter Default is enabled
Bridge Assurance                         is enabled
Loopguard Default                        is disabled
Pathcost method used                     is long
PVST Simulation                          is enabled
STP-Lite                                 is enabled

Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
MST0000                      1         0        0          8          9
---------------------- -------- --------- -------- ---------- ----------
1 mst                        1         0        0          8          9
`show spanning-tree active`

MST0000
  Spanning tree enabled protocol mstp
  Root ID    Priority    0
             Address     c091.34af.e200
             Cost        200
             Port        4096 (port-channel1)
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     002a.6a26.4901
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1              Root BKN*200       128.4096 (vPC) P2p Bound(PVST) *PVST_Inc
Po37             Desg FWD 200       128.4132 (vPC) P2p Bound(PVST)
Po38             Desg FWD 200       128.4133 (vPC) P2p Bound(PVST)
Po50             Desg FWD 1000      128.4145 (vPC peer-link) Network P2p
Po101            Desg FWD 200       128.4196 (vPC) Edge P2p
Po102            Desg FWD 200       128.4197 (vPC) Edge P2p
Po202            Desg FWD 200       128.4297 (vPC) P2p
Eth1/23          Desg FWD 2000      128.151  P2p
Eth1/29          Desg FWD 20000     128.157  Edge P2p

 

From above you can see the message like this:

Po1              Root BKN*200       128.4096 (vPC) P2p Bound(PVST) *PVST_Inc

I've touched base with HP and Cisco and I haven't got any action plan for this. We've also done the changes for 4 times and were walked through with VCE/Cisco people however, we still have't got it resolved.

 

the last email I received from Cisco and was told "the reason that the Nexus believes that the HP switch is configured as PVST, is because it is receiving PVST BPDU’ rather than MST BPDU’s."

 

Is that true?

 

Thanks for your time to read the post and I hope you can shed some lights to the solutions.

 

Thanks!

 

Wilson

 

 

 

 

 

 

 

 

 

Cisco Employee

Hi Wilson,I apologize for

Hi Wilson,

I apologize for responding so late. I still hope this response will be helpful.

You have quite an interesting problem at hand. If I understand your network setup properly, you do not have any switches in your network configured for PVST, they all run MST, and yet the Nexus switches claim they have PVST boundary ports and they even declare PVST Simulation inconsistency.

I have a hypothesis that could explain the issue you are experiencing but it is up to you to say whether it is plausible. What I believe has happened is that you were migrating your Nexus switches from PVST to MSTP gradually. When you migrated one of the Nexus switches to MSTP while the other Nexus switches still ran PVST, the migrated Nexus declared the ports receiving PVST BPDUs as PVST boundary ports and also started sending PVST BPDUs on them (as a part of the PVST Simulation). When you migrated the other switch to MSTP, it was receiving PVST BPDUs from the already-migrated Nexus that still ran PVST Simulation on its apparent boundary ports, so the other switch started the PVST Simulation as well. Effectively, all Nexus switches have been migrated to MSTP but they are holding each other in deadlock and cause each other to keep the PVST Simulation running.

There are multiple steps to prevent this from occurring:

  1. Your HP switch has a feature called PVST Filtering. Effectively, this feature causes your HP switch to drop PVST BPDUs. I strongly suggest activating it. That will prevent the HP switch from propagating PVST BPDUs between the attached Nexus switches, causing them to start the PVST Simulation. As the Nexus switches run MSTP, switching loops should not occur as MSTP BPDUs will continue to be processed normally and MSTP will itself take care of eliminating them. The PVST Filtering feature can be enabled on the HP switch using the spanning-tree port-list pvst-filter global configuration command. In the port-list argument, list all ports including EtherChannel ports if possible (called trunks on HP switches).
  2. If the Nexus switches indeed hold each other in a deadlock, try pushing the Nexus switches into re-detecting the neighbor's STP version. This can be accomplished using the clear spanning-tree detected-protocol command. This command may need to be used repeatedly on all Nexus switches. The renegotiation may cause transient connectivity outages as it may result into MSTP reestablishing the port roles and states.
  3. Try deactivating the PVST Simulation on the Nexus switches using the no spanning-tree mst simulate pvst global command in the global config mode. This command will stop PVST Simulation on the switch, preventing it from sending PVST BPDUs on boundary ports. However, even with the PVST Simulation deactivated, a Nexus switch will immediately put a port receiving PVST BPDUs into blocking state (exactly because the PVST Simulation is not running so the contents of the PVST BPDUs cannot be processed appropriately). Therefore, perform this particular step only if the previous steps solved the issue and the show spanning-tree command on the Nexus switches does not report any PVST boundary ports.

If the above steps, especially step 1 and 2 do not help, please try to provide the following information:

  • show spanning-tree bridge from a bridge that reports the PVST Simulation inconsistency
  • show spanning-tree root from the same bridge
  • The exact logging message that is generated by the switch when it detects the PVST Simulation inconsistency. That message contains the Bridge ID of the offending switch that appears to originate the inconsistent BPDUs.
  • Locate the offending switch based on the MAC address from the Bridge ID, and provide the show spanning-tree bridge and show spanning-tree root from that switch as well

Looking forward to hearing from you!

Best regards,
Peter

Bronze

Nice explanation Peter.Just

Nice explanation Peter.

Just to add to this. I have run in to the same thing in the past.  HP switches at the core of the network in a ring. Multiple Cisco Access switches plugged in to the HP core. All switches were running MST (we did the migration during a downtime so this particular behavior was not noticed or may have been ignored). So everything was working nicely everyone was happy.. until one day I decided to plug in a seemingly "innocent" Cisco 887 router (with multiple BVI/VLANs configured) in to the core HP switch. Yeah.. took down the whole network. Ended up disabling STP on the router to get the network back online.

New Member

Hello Paul,I've read you're

Hello Paul,

I've read you're explanation but still there are details that I don't understand. I just had the same issue and I was able to replicate it on the lab. The topology is very simple

 

SW1(MSTP priority 0 for MSTI0 and priority 4096 for MSTI1(vlan 300))

|

SW2(PVST default priority for all vlans)

|

SW3(MSTP default priority on both MSTI0 and MSTI1)

 

Although SW1 is the root for all VLANs the mismatch in priorities is causing the issue, SW3 put the uplink in BKN state, If I set the priority to a same value or join SW2 to MSTP the issue disappear. Is there any explanation for this behavior?

 

Cisco Employee

Hi Ruben,

Hi Ruben,

It would seem that in your lab scenario, the default setup on SW2 is causing the PVST Simulation on SW3 to fail. Specifically, the rules of PVST Simulation require that for a MST boundary port to become a Root port, the following conditions must all be met:

  1. Received VLAN1 PVST+ BPDUs on this port must advertise a root switch whose Bridge ID is lower than the Bridge ID of the receiving switch
  2. Received VLAN1 PVST+ BPDUs on this port must be superior to any other VLAN1 BPDUs received by any other port of the receiving switch
  3. Received PVST+ BPDUs for VLANs other than VLAN1 received on this port must be superior to VLAN1 BPDUs received on this port

Note that in your case, condition 3 is not met. SW2 with its default settings forwards two BPDUs to SW3 each two seconds:

VLAN1:
Root BID = 0:0:SW1-MAC
Root Path Cost = x
Sending BID = 32768:1:SW2-MAC
Sending PID = y

VLAN 300:
Root BID = 0:0:SW1-MAC
Root Path Cost = x
Sending BID = 32768:300:SW2-MAC
Sending PID = y

Note that because of the Extended System ID used by SW2, its own Sending BID in VLAN300 is higher, thus inferior, to its own Sending BID in VLAN1. Consequently, BPDUs in VLAN300 are inferior to BPDUs in VLAN1 while the rules for PVST Simulation require the relation of the BPDUs to be just the opposite.

If my theory is correct then simply lowering the SW2 priority in VLAN 300 (and all non-VLAN1 VLANs) below 32768 should allow SW3 to be happy and keep its Root port in the Forwarding state. Can you please confirm this?

Best regards,
Peter

New Member

You're right, I can confirm

You're right, I can confirm that lowering all VLANs priorities but 1 on SW2 fixed the issue.

Thank you so much for your prompt response

 

3570
Views
35
Helpful
24
Replies