STP Loop Inconsistency on 6509 CATOS

Unanswered Question
Feb 13th, 2007

Hello,

Seeing the following on a C6509 running CATOS:

HAVC6502> (enable) sh spantree act

VLAN 1

Spanning tree mode MST

VLAN 1 mapped to MST Instance: 0.

Port Vlan Port-State Cost Prio Portfast Channel_id

------------------------ ---- ------------- --------- ---- -------- ----------

3/1 1 forwarding 20000 32 disabled 0

3/2 1 forwarding 20000 32 disabled 0

3/3 1 forwarding 20000 32 disabled 0

3/4 1 forwarding 20000 32 disabled 0

3/5 1 loop-inconsis 20000 32 disabled 0

3/7 1 loop-inconsis 20000 32 disabled 0

4/3 1 forwarding 20000 32 disabled 0

4/4 1 loop-inconsis 20000 32 disabled 0

4/5 1 loop-inconsis 20000 32 disabled 0

4/6 1 loop-inconsis 20000 32 disabled 0

4/7 1 forwarding 20000 32 disabled 0

4/8 1 forwarding 20000 32 disabled 0

4/9 1 forwarding 20000 32 disabled 0

4/10 1 loop-inconsis 20000 32 disabled 0

4/11 1 forwarding 20000 32 disabled 0

4/12 1 loop-inconsis 20000 32 disabled 0

4/13,5/13 1 forwarding 10000 32 disabled 1768

#spantree global defaults

set spantree global-default loop-guard enable

I have set the above loop guard function to allow ports to automatically recover from inconsistent status once BPDUs are heard again on the port (given a loop has been seen) - however, these ports appear stuck in this state. Removing and readding global loop guard statement above has in the past reset the port status, but in this instance it does not seem to work.

Can anyone advise on how to reset these ports manually?

Code used is: bootflash:cat6000-sup2k8.8-5-2.bin

rgds

Phil

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
ssoberlik Mon, 02/19/2007 - 11:27

For Soft resetting the module on which this port occurs use the reset command and see if it recovers.For Hard resetting the module using the set module power up|down command and see if it recovers.

phil_carter Tue, 02/20/2007 - 01:27

Thanks for this but this is obviously service impacting and will effect all connections per module (not just per port).

Has anyone else seen this problem and had to rectify it using a more cautious approach? If so, what configuration changes were made?

Francois Tallet Tue, 02/20/2007 - 09:41

In theory, as you mentioned, as soon as some bpdus are received from the remote end, the inconsistency should be cleared. It would be nice to see what the other end thinks of those links btw, just to check that we're not in a real inconsistency scenario. This kind of inconsistency can be caused by a link failure but also by some software issue or a misconfiguration (the links where the feature is configured *have to* be point to point with regard to STP). I'm no CatOS guru, but I guess that disabling and enabling back the port should reset loopguard. Loopguard has a dynamic component: it is tracking a designated port. As soon as the port is brought down, the information about the designated port should be cleared along with the inconsistency.

Regards,

Francois

phil_carter Wed, 02/21/2007 - 01:48

Francois - thanks for the update.

HAVC6502> (enable) sh spantree act

VLAN 1

Spanning tree mode MST

VLAN 1 mapped to MST Instance: 0.

Port Vlan Port-State Cost Prio Portfast Channel_id

------------------------ ---- ------------- --------- ---- -------- ----------

3/1 1 forwarding 20000 32 disabled 0

3/2 1 forwarding 20000 32 disabled 0

3/3 1 forwarding 20000 32 disabled 0

3/4 1 forwarding 20000 32 disabled 0

3/5 1 loop-inconsis 20000 32 disabled 0

Code: cat6000-sup2k8.8-5-2.bin

For example, the other end of port 3/5 (which is a GBE3 blade enclosure switch):

Gi0/17 Root FWD 4 128.17 P2p

Gi0/21 Desg BKN*100 128.21 P2p Bound(RSTP) *LOOP_Inc

Code: cgesm-i6l2-mz.122-25.SE1.bin

Both ends of the link are up and are a direct fibre between the two. Both appear in loop inconsistent state.

All the inconsistent ports seen are between the 6509 and GBE3 switches (cisco 371098-001).

I will try bouncing a single port to see if this resolves the problem - however, it seems strange that removing and readding loopguard globally on the 6509 has previously resolved this, but no longer works.

Phil

Francois Tallet Wed, 02/21/2007 - 09:28

Hi Phil,

Yes, I would expect that removing the loopguard configuration should clear the inconsistency. I'll ask someone who is more familiar with CatOS.

We had in IOS some race condition that could lead both ends of a link to become loop inconsistent. The problem with this scenario is that you only exit the inconsistency by receiving a bpdu from your neighbor. However, when inconsistent, the port does not send bpdu (if it was sending bpdu, there would have some other possible deadlocks). Thus, both ends of a link inconsistent is a situation that cannot be recovered without user intervention. That's really bad. We've made our best to prevent the situation from occurring (see CSCeg90349), but there might be some race condition that we have missed.

I would recommend that you only configure loopguard on one side of the link. In fact, loopguard is only necessary on the side that is receiving bpdus (root port or alternate port). It's really painful to use a feature that is supposed increase the uptime of your network and that brings it down instead:-( I'm not a big fan of loopguard configured everywhere.

The latest MST versions include by default what we called "the dispute mechanism". This is a feature that appeared in the IEEE specification and that replaces loopguard and works much better. Its only drawback is that it only works between switches running RSTP or MST (which seems to be the case for you). I think it's available in 8.6 in CatOS, and is included in recent MST code in IOS (the "IEEE standard" version of the IOS MST, which supports up to 64 MST Instances).

Regards,

Francois

phil_carter Thu, 02/22/2007 - 01:35

Francois,

I'm running MST everywhere (single instance), so any web links to specific documentation on the newer version of MST incoporating the dispute mechanism would be great. I'd be very ineterested in reading up on this...

We have loopguard configured on all switches, and to be fair it has worked a treat since configured (barring this instance). Before when STP decided to have an off-day we were having to manually bounce links and ended up running around in circles as more and more (random) links err-disabled themselves. Do you recommend not configuring loopguard on the core then (or not specifically on the root as all ports will be DSG/FORWARD state)?

I had a quick look at the bug and it seems to refer specifically to RPVST and a change to the root bridge - this is not the case with what I am seeing (although the end result is the same).

thanks

Phil

Francois Tallet Thu, 02/22/2007 - 12:18

Hi Phil,

I'm recommending that you configure loopguard on only one side of your point-to-point links. Configure it on the port that is receiving BPDUs (port with a root or alternate role generally). Don't configure it on designated ports. Indeed, Root and Distribution switches mostly have designated port. Because several instance can have different role on the same port, configuring loopguard on both ends of a link might be necessary (between core switches generally).

I'm not good at finding things on cisco.com. I found a relatively implicit reference to this disupte feature at:

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/122sx/swcg/mst.htm#wp1142550

The flag it raises in the show spanning-tree command is mentioned here:

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/122sx/swcg/mst.htm#wp1142550

I can explain the dispute mechanism in more detail if necessary.

Regards,

Francois

Actions

This Discussion