I think I have a spanning problem

jfeo · ‎03-01-2009

The problem occurs every 24 hours network locks up after rebooting all switches seems to be fine.

One switch I see in flash

3560 12.2.37 SE1

irectory of flash:/

2 drwx 256 Feb 27 2009 20:25:57 -05:00 crashinfo_ext

4 drwx 256 Feb 28 1993 19:00:03 -05:00 crashinfo

5 -rwx 616 Feb 28 1993 19:00:28 -05:00 vlan.dat

465 drwx 192 Feb 28 1993 19:13:52 -05:00 c3560-ipservicesk9-mz.122-37.SE1

7 -rwx 10163 Mar 1 2009 22:02:31 -05:00 config.text

8 -rwx 1928 Mar 1 2009 22:02:31 -05:00 private-config.text

I also see this in the logs with different mac address.

%SW_MATM-4-MACFLAP_NOTIF:Host 00a0.f856.4a72 in vlan 1 is flapping between port Gi0/2 and port Gi0/1

%SW_MATM-4-MACFLAP_NOTIF: Host 0004.009c.d5e7 in vlan 1 is flapping between port Gi0/2 and port Gi0/1

%SW_MATM-4-MACFLAP_NOTIF: Host 00a0.f856.4f1c in vlan 1 is flapping between port Gi0/2 and port Gi0/1

This might sound weird but

this pops in and out of the config on same switch

switchport trunk pruning vlan 2-960,969-1001

Yudong Wu · ‎03-01-2009

There might be a loop in your network.

Do you know what devices are connected to port gi0/2 and gi0/1? You might need to check them.

jfeo · ‎03-01-2009

I have Rapid spanning tree on

I have 9 switches Daisie chained in one loop

I checked all switch and RSTP is running

Like I said this only happen every 12 to 24 hours

Yudong Wu · ‎03-02-2009

"I have 9 switches Daisie chained in one loop", it is not a good design to have nine switches in a loop. You might need to consider network topology in the future.

As for the issue, the following posts have pointed out some good direction.

Have this nework been working well?

If yes, you really need to trace down on the devices which are connected to g0/1 and g0/2. The other thing you can try is to enable loopguard or UDLD on all links between switches to see if it is caused by a unidirectional link.

Leo Laohoo · ‎03-01-2009

Is spanning-tree portfast enabled on both Gi0/1 & Gi0/2?

Can you also post the lastest crashinfo file?

jfeo · ‎03-01-2009

portfast is not configured on those ports only the ethernet ports.

how do I get the crash info file

jfeo · ‎03-01-2009

Is this what u R looking 4

Using flash:/crashinfo/crashinfo_4.

Cisco IOS Software, C3560 Software (C3560-IPSERVICESK9-M), Version 12.2(37)SE1, RELEASE SOFTWARE (fc1)

Compiled Thu 05-Jul-07 22:22 by antonino

Debug Exception (Could be NULL pointer dereference) Exception (0x2000)!

SRR0 = 0x00668420 SRR1 = 0x00029230 SRR2 = 0x00666C94 SRR3 = 0x00021000

ESR = 0x00000000 DEAR = 0x00000000 TSR = 0x8C000000 DBSR = 0x10000000

CPU Register Context:

Vector = 0x00002000 PC = 0x009C591C MSR = 0x00029230 CR = 0x20000004

LR = 0x009C5844 CTR = 0x00228B94 XER = 0x00000018

R0 = 0x00000000 R1 = 0x020CE418 R2 = 0x00000000 R3 = 0x00000000

R4 = 0xFFFFFFFE R5 = 0x00000000 R6 = 0x020CE3F0 R7 = 0x00000000

R8 = 0x00029230 R9 = 0x01BF0000 R10 = 0x01EAF658 R11 = 0x00000000

R12 = 0x0000001B R13 = 0x00110000 R14 = 0x0119F010 R15 = 0x00000000

R16 = 0x00000000 R17 = 0x00000000 R18 = 0x00000000 R19 = 0x00000000

R20 = 0x00000000 R21 = 0x00000000 R22 = 0x00000000 R23 = 0x00000000

R24 = 0x00000000 R25 = 0x00000000 R26 = 0x009890BC R27 = 0x01F72188

R28 = 0x01D0BE8C R29 = 0x01A59770 R30 = 0x00000000 R31 = 0x07B1EE44

Stack trace:

PC = 0x009C591C, SP = 0x020CE418

Frame 00: SP = 0x020CE428 PC = 0x009C581C

Frame 01: SP = 0x020CE448 PC = 0x0119BEF0

Frame 02: SP = 0x020CE470 PC = 0x0119CD74

Frame 03: SP = 0x020CE488 PC = 0x0119EEA4

Frame 04: SP = 0x020CE4A0 PC = 0x0119F0CC

Frame 05: SP = 0x020CE4A8 PC = 0x009C78FC

Frame 06: SP = 0x00000000 PC = 0x009BECC4

Terryn Barbarich · ‎03-02-2009

Have you tried running:

show spanning-tree vlan detail on the relevant VLAN to see when it the last topology change was.

Leo Laohoo · ‎03-02-2009

Yes. That's it. Can you post the crashinfo that was created on 28 February?

I agree with Glen. The loop is likely to be caused by some loopie plugging in a switch/hub into the network.

Since the alarm is coming from both interface (Gi0/1, Gi0/2), how about disconnecting one of them? Once you've done this and no one is seeing any outages, you need to find out where the loop is coming from.

glen.grant · ‎03-02-2009

What is on g0/1 and g0/2 ? This is basically telling you where the problem is . I would look for looped wiring between those ports or some user who thought it was a good idea to bring in his own switch or hub and plug it in . Also when this happens look to see if the switch can see itself via cdp , if so then you have a physically wired loop out there .

ilnaiduccna · ‎03-02-2009

Hi all,

Is there may be problem with Root Switch & priority?

Regards,

Naidu.

jfeo · ‎03-02-2009

Not that I can see

I have one switch configured as the root and another as the back up.

jfeo · ‎03-02-2009

Number of topology changes 5 last change occurred 10:12:48 ago uptime is 15 hours, 29 minutes

Number of topology changes 16 last change occurred 10:14:07 ago uptime-1 day, 11 hours, 37 minutes

Number of topology changes 14 last change occurred 10:15:17 ago uptime-1 day, 11 hours, 38 minutes

Number of topology changes 16 last change occurred 10:16:07 ago uptime-1 day, 11 hours, 41 minutes

Number of topology changes 2 last change occurred 1d11h ago uptime-1 day, 11 hours, 46 minutes

Number of topology changes 3 last change occurred 15:48:12 ago uptime-1 day, 11 hours, 48 minutes

Number of topology changes 4 last change occurred 10:37:51 ago uptime-15 hours, 50 minutes Root switch

Number of topology changes 3 last change occurred 12:58:49 ago uptime-1 day, 11 hours, 53 minutes

Number of topology changes 4 last change occurred 10:29:35 ago uptime-13 hours, 4 minutes

Leo Laohoo · ‎03-02-2009

an you post the crashinfo that was created on 28 February?

I agree with Glen. The loop is likely to be caused by some loopie plugging in a switch/hub into the network.

Since the alarm is coming from both interface (Gi0/1, Gi0/2), how about disconnecting one of them? Once you've done this and no one is seeing any outages, you need to find out where the loop is coming from.

ARUNPRABHU A · ‎03-03-2009

Hi,

You Can better disconnect any one of the Interface and check the MAC address FLAP is happening or not.

The Similar kind of issue i faced and that network Setup is like.

One switch connected to hub through two ports , and each port is in differen VLAN.

eg -

Switch fa 0/1 ( in VLAN 2 ) - Hub port 2

Switch fa 0/4 ( in VLAN 3 ) - Hub Port 4

since hub is an extended medium you can't have user in the hub of different VLAN's.

Finally after moving out the users of one VLAN from the HUB, the issue got resolved.

Rgds, ArunPrabhu A