Spanning Tree & incorrect root bridge assignment

Ronald Spencer · ‎08-18-2010

Hi NetPros,

I inherited a network that was poorly configured. It is really more like an onion. As I peel back each layer, it causes my eyes to water (if you know what I mean). My latest challenge is Spanning Tree. I have a largely non-redundant, multivendor, network. Spanning-tree, which is on by default, was not properly configured. As I have worked to correct that, I notice a couple issues.

Originally, I thought that it was because VLAN 1 was not being permitted across the upline to the Root (by way of a Extreme Networks Summit x450e-48p, because the transit path for those 3 switches is glass and the 3750 does not have any optical ports). The more I think about this the more I question that logic. Prior to changing the allowed statement on the trunk, other vlans on the switch acknowledged the Root.

With everything in its default configuration (or mostly) all switches were declaring the same priorities. I have updated this to reflect my desire that the root should be switch X (spanning-tree vlan 1-4094 root primary). This cleared up issues with the vlans on the Edge. it is declaring itself to be root for all vlans it knows about (the desired outcome). It also cleared up spanning tree for the 3560's with one exception. All 3 - 3560's believe they are the spanning-tree root for vlan 58. 2 of the switches are configured for PVST and the 3rd is configured for RPVST.

I was concerned about the versions of stp (from my other switches) but the stpd on my extreme gear is disabled and the stp mode on the dell switches (2) is rstp. (The Dells, by the way, also beleive they are spanning-tree root for their respective vlans).

Topology looks like this:

Edge

| | | |

| | | -------------------|

|---- | -----------| |

4506 Extreme Dell 1 Dell 2

| | |

| | -----------------|

| -----------| |

3560 3560 3560

On the dells, I configured the bridge priority to 61440. When debuging spanning-tree at the edge, I still received "Heard root on <trunk port to dell>. This message stopped when I configured the dell as RSTP (left the priority the same). Still the Dells insist they are the root of Spanning tree for the 3 vlans they know about.

I am beating my head against the wall and am wondering (before I have to do any drywall patching) what am I missing?

The output of the show spanning-tree summary command for each of the 3560's is listed below: Also included is the output for the edge (3750-24)

///// 3560-1 /////

LNBT-S-WHSE-001#sh span sum
Switch is in pvst mode
Root bridge for: VLAN0058
Extended system ID           is enabled
Portfast Default             is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is disabled
EtherChannel misconfig guard is enabled
UplinkFast                   is disabled
BackboneFast                 is disabled
Configured Pathcost method used is short

Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0001                     0         0        0          2          2
VLAN0009                     0         0        0          3          3
VLAN0010                     0         0        0          4          4
VLAN0020                     0         0        0          2          2
VLAN0050                     0         0        0          2          2
VLAN0055                     0         0        0          2          2
VLAN0058                     0         0        0          2          2
VLAN0059                     0         0        0          2          2
VLAN0199                     0         0        0          2          2

Name Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
---------------------- -------- --------- -------- ---------- ----------
9 vlans 0 0 0 21 21

///// 3560-2 /////

LNBT-S-NTSB-001#sh span sum
Switch is in pvst mode
Root bridge for: VLAN0058
Extended system ID           is enabled
Portfast Default             is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is disabled
EtherChannel misconfig guard is enabled
UplinkFast                   is disabled
BackboneFast                 is disabled
Configured Pathcost method used is short

Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0009                     0         0        0          9          9
VLAN0010                     0         0        0          1          1
VLAN0020                     0         0        0          1          1
VLAN0050                     0         0        0          1          1
VLAN0055                     0         0        0          1          1
VLAN0058                     0         0        0          1          1
VLAN0059                     0         0        0          2          2
---------------------- -------- --------- -------- ---------- ----------
7 vlans                      0         0        0         16         16

///// 3560-3 /////

LNBT-S-NTST-001#sh span sum
Switch is in rapid-pvst mode
Root bridge for: VLAN0058
Extended system ID           is enabled
Portfast Default             is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is enabled
EtherChannel misconfig guard is enabled
UplinkFast                   is disabled
BackboneFast                 is disabled
Configured Pathcost method used is short

Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0009                     0         0        0          9          9
VLAN0010                     0         0        0          6          6
VLAN0020                     0         0        0          1          1
VLAN0050                     0         0        0          1          1
VLAN0055                     0         0        0          1          1
VLAN0058                     0         0        0          1          1
VLAN0059                     0         0        0          2          2
---------------------- -------- --------- -------- ---------- ----------
7 vlans                      0         0        0         21         21

///// 3750 /////

LNBT-S3-EDGE-001#sh span sum
Switch is in pvst mode
Root bridge for: VLAN0001, VLAN0005, VLAN0009-VLAN0012, VLAN0015, VLAN0020
VLAN0040, VLAN0050, VLAN0055, VLAN0058-VLAN0059, VLAN0199
Extended system ID           is enabled
Portfast Default             is disabled
PortFast BPDU Guard Default is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is disabled
EtherChannel misconfig guard is enabled
UplinkFast                   is disabled
BackboneFast                 is disabled
Configured Pathcost method used is short

Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
VLAN0001                     0         0        0          4          4
VLAN0005                     0         0        0          3          3
VLAN0009                     0         0        0          4          4
VLAN0010                     0         0        0          5          5
VLAN0011                     0         0        0          4          4
VLAN0012                     0         0        0          4          4
VLAN0015                     0         0        0          4          4
VLAN0020                     0         0        0          4          4
VLAN0040                     0         0        0          3          3
VLAN0050                     0         0        0          5          5
VLAN0055                     0         0        0          4          4
VLAN0058                     0         0        0          4          4
VLAN0059                     0         0        0          4          4
VLAN0199                     0         0        0          4          4
---------------------- -------- --------- -------- ---------- ----------
14 vlans                     0         0        0         56         56

Giuseppe Larosa · ‎08-18-2010

Hello Ronald,

what you are doing will pay on the long term

>> was concerned about the versions of stp (from my other switches) but the stpd on my extreme gear is disabled and the stp mode on the dell switches (2) is rstp. (The Dells, by the way, also beleive they are spanning-tree root for their respective vlans).

My guess is that the Dell are running a single 802.1W Rapid STP instance for all Vlans, if so they do not listen to STP BPDUs on vlans different from vlan 1. This can explain why they consider themselves root bridge for the vlans defined on them.

For the C3560 that are connected downstream the extreme gear the case is really strange:

you have disabled STP on the extreme switch (this is something that should not be done) and this can be part of the problem.

In theory if the extreme switch runs a legacy 802.1D old STP PVST+ (but you are using Rapid PVST) moves to use proprietary BPDUs that are seen as multicast frames by the legacy switch.

So the PVST+ frames should be propagated to the switches downstream in all Vlans.

You have a single vlan 58 for which your switches downstream do not receive frames from the root bridge.

What is the native vlan on the links between the C3560 and the extreme networks?

Check both sides on each link.

What you can do also is to use a local SPAN session on C3560 to replicate traffic of uplink to a port and to connect a sniffer there to verify if these PVST+ BPDUs are actually missing

You should enable STP on the extreme network switch for your safety.

Hope to help

Giuseppe

Ronald Spencer · ‎08-18-2010

Hi Giuseppe,

With regards to your questions, I made the following observations:

Native VLAN: The 3750 (edge) and 3 - 3560 are set to native vlan 1. There is inconsistency in the vtp mode (the 3 - 3560's believe they are vtp server. The 3570 and the 4506 are set to transparent. We are not (intentionally using vtp, so I will set the 3 of them to transparent).

Interestingly enough, we are permitting vlan 58 on the trunks (from cisco's perspective, but are not tagging the ports for that vlan on the extreme). I will be correcting this tonight.

Thank you for your help. I will follow up after correcting the aforementioned inconsistencies.

Peter Paluch · ‎08-18-2010

Hello Ronald,

An onion? That's an understatement

Okay, let's be serious. I have a couple of mostly disorganized thoughts:

One of the most important things to verify at the beginning of this is whether all VLANs are created on all switches. Sometimes, administrators are tempted to create only those VLANs on a switch that are going to be used on access ports on that very switch. However, even a switch does not have any access ports in a particular VLAN, it needs to have it defined if it interconnects other switches by a trunk. Otherwise, that VLAN would not pass through the switch. Once again, it is very important for now that all VLANs are defined on all switches in your topology.
The second issue is to verify whether all these VLANs (and for now also including VLAN1) are allowed on all trunks.
As the next step, it is very important to use the same STP version on all switches in the topology. At this point, I strongly doubt that the Dell and Extreme switch understand the Cisco-proprietary PVST+ and RPVST+. What they probably understand is only the IEEE-formatted 802.1D BPDU sent in VLAN1, however, the PVST+ and RPVST+ use a slightly modified BPDU format and a different Ethernet encapsulation. Please note that Cisco switches cannot be configured to run a plain STP or RSTP. They will always run PVST+ or RPVST+ along STP or RSTP. In a multivendor network with VLANs, the best way to go is to use the Multiple STP (MSTP). Do all your switches support MSTP? If yes you should very seriously consider start using it. Otherwise, we may be trying to solve the unsolvable - we cannot expect a non-Cisco switch to be happy with Cisco-proprietary (R)STP ripoff (although some non-Cisco product may actually understand it). Definitely, it is impossible to debug your network with all the switches running different versions of STP. Once again, I strongly encourage you to migrate to MSTP if all switches support it.
Having the stpd deactivated on the Extreme may mean that the switch is basically blocking all received BPDUs. I don't know about Extreme switches but bear in mind that deactivating the STP support does not necessarily mean that the switch becomes transparent for BPDUs. It is very much an implementation-dependent issue. In any case, I do not recommend deactivating STP on any switch in your network.

Please verify the issues described in these items, and let us know of any results and findings. I am looking forward very much to reading your answer.

Best regards,

Peter

Ronald Spencer · ‎08-18-2010

?Hi Peter,

I can appreciate disorganized thoughts (my original post). With respects to the the items posed:

1) all vlans all switches - this would be true for distribution and core switching, I understand. Would it be true for Access switches (where vlans are traversing the switch to get other locations)? If we have no access ports in a vlan, and no downstream switches (from that switch) would we still want all vlans all switches?

2) allowed on trunks. We discovered that the native vlan on the extreme is tagged. This will be corrected tonight. Also, vlan 58 (the vlan the 3 - 3560's believe they are the root for) does not exist on the extreme switch. This, also, will be corrected tonight.

3) I can confirm that the extreme and dells are capable of speaking 802.1d (STP), 802.1w (RSTP) and 802.1s (MSTP). My Cisco devices speak MST, PVST and R-PVST (of course). The research I did on interoperability produced the following document:

http://www.dell.com/downloads/global/products/pwcnt/en/app_note_1.pdf

The significant text from that document was found on page 2 under proposed solution:

Note: Cisco Catalyst 3550s (and most other midrange Cisco switches) have Cisco’s proprietary Per-VLAN
Spanning Tree Protocol (PVST+) enabled by default. However, Cisco switches will revert to using
802.1D STP with a peer switch after receiving a BPDU.

4) Depending on the outcome of the above, I will plan how to move foward with the extreme.

Thank you for all your help. (CCNA seems so long ago).

-Ronnie

Peter Paluch · ‎08-18-2010

Hello Ronnie,

To your questions:

At least for a Cisco switch, if it does not have a particular VLAN defined then it also does not run a (R)PVST+ instance for that VLAN. This makes it increasingly hard to debug the proper behavior of (R)PVST+ in your network. As a starting point (and keeping things simple for the beginning), I suggest very much having all VLANs on all switches. We can start optimizing this after we get the STP going but better not before. As Donald E. Knuth once commented, "premature optimization is the root of all evil"
The configuration issues on the Extreme switch may have had profound impacts on the STP operation. The corrections you decided to made are very important.

One note: you said that "the native VLAN is tagged". That is a contradiction - a native VLAN is not tagged by its very virtue, and if it is tagged, we do not call it a native VLAN anymore. Did you mean the VLAN1 in particular? In any case, exactly as Giuseppe suggested, make sure that the native VLAN (i.e. the untagged VLAN) is identical on all trunks. In order to not complicate things further, make the VLAN 1 as the native VLAN on all trunks.

Also please note that it is necessary to run some kind of STP on the Extreme, as also Giuseppe stressed before.
It looks like all your switches do support MSTP which aligns with my recommendation for MSTP in a multi-vendor VLAN environment. The note on page 2 that you quoted here has to be correctly interpreted: Cisco switches always run both (R)STP and (R)PVST+ side by side, and they always send and receive both IEEE-compatible BPDUs and (R)PVST+ BPDUs. If they receive only standard 802.1D/802.1w BPDUs then for VLAN 1, the state of the port will be determined by that received BPDU. However, I am not aware of any other reverting. That would also be in contradiction to the inteded (R)PVST+ behavior which is to literally tunnel its BPDUs through the (R)PVST-incompatible network.

Are you familiar with the MSTP configuration and principle of operation? MSTP is beyond CCNA stuff and it requires more initial configuration than just turning it on.

Giuseppe pointed out a very, very important issue: the native VLAN must be same. I also cannot stress this too much From what you have indicated in your reply to Giuseppe's post, you indeed have had a native VLAN mismatch: if the VLAN 58 was untagged on the Extreme switch then VLAN 1 on Cisco and VLAN 58 on Extreme were leaking one into another. As I recommended in the item 2, make sure that the VLAN 1 is untagged and all other VLANs are tagged on all trunks in your topology, thereby making VLAN1 the native VLAN.

Best regards,

Peter

Ronald Spencer · ‎08-18-2010

Hi Peter,

Thank you for talking the time to write back. To address your responses:

1) I see the value in what you are suggesting. Will add that to my implementation plan. But I thought it was the Love of Money that was the root of all kinds of evil? ;-)

2) I think that I might have miss communicated this one. If you substitute my substitution (vlan 1 for native vlan) you end up with what I was trying to say. if you have never worked with an extreme, they are really a peculiar beast. Extremes are modular in their config. Also, whereby you might be able to call a Cisco switch "interface-centric", the Extreme box is VLAN-Centric. you configure a vlan name, then add a the vlanID (or tag), then add the ports to the valn (instead of the vlan to the ports). What I found on the extreme switch is that the trunk ports to my 3560's were added to vlan1 as tagged (instead of untagged. They should have been added as untagged.

regarding 58...What I meant to say is that these ports (the trunk ports for my 3560's) were not added to vlan 58 (tagged). The were not added to 58 at all because 58 is not configured on that switch. (this needs to be addressed).

3) My level of experience / familiarity wih MSTP is confined to SWITCH (as I have embarked on the daunting endeavor of NP certification), and what I have read online.

-Ronnie

.

Peter Paluch · ‎08-19-2010

Hi Ronnie,

Love of Money? Well, that can be seen also as a special case of premature optimization

Regarding the VLAN-centric Extreme switches, I have not worked with them but I believe that some Siemens and HP switches are similar - you define VLANs and then assign ports into them using the untagged and tagged keywords. Not a bad approach, either.

Regarding the MSTP, you do not need any complicated configuration for a start. I would begin with a totally simple region configuration that has a name, say, "OnionNetwork" (pun intended ), revision 1 and all VLANs mapped to the default instance 0.

Best regards,

Peter