Troubleshooting intermittent network outages on a Catalyst 3548XL.

Unanswered Question
Jun 9th, 2008

Hello,


I have a client with four Catalyst 3548XL switches (containing 4 VLANS) that are experiencing sporadic network outages. During the failure, users lose connection to the network drives on the servers, printing fails, and the internet cannot be accessed. The failures affect all vlans, and the network seems to restore itself after 10-15 minutes.


The failures could potentially be isolated to the switch containing the servers, as printing, drive shares, and DNS requests all go through that switch... however, I am not seeing anything in the event logs of the servers to indicate a problem here.


The switches are all running IOS 12.0(5)XU. The show diagnosticlog shows me that many (not all) of the ports are changing states from up to down and back again. The show diag link-flap shows me that some ports are experiencing a large amount of link-flap errors (relative to other ports, but i don't know if 497 is an excessive number), but again, I can't isolate to a root cause from here.


Can anyone give me some troubleshooting tips here? As you can probably tell, I am a cisco newbie. Are there othere diagnostic logs available to me on the switches that I am unaware of? I will attach one of the configs.


thanks,

scott



  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
cowetacoit Mon, 06/09/2008 - 10:26

Can you post a show span? I would turn off spanning-tree portfast default. Sounds like a spanning tree convergence problem or loop.

cowetacoit Mon, 06/09/2008 - 10:28

Why do you have this port configured for access and trunk?


interface FastEthernet0/2

duplex full

speed 100

switchport access vlan 999

switchport trunk pruning vlan 3-1001

switchport mode trunk

spanning-tree portfast

!



I would change to


interface FastEthernet0/2

duplex full

speed 100

switchport access vlan 999

switchport mode access

no switchport trunk pruning vlan 3-1001

no switchport mode trunk

no spanning-tree portfast

!

scottmilner Mon, 06/09/2008 - 10:35

Jun 9, 2008, 11:28am PST

Why do you have this port configured for access and trunk?


interface FastEthernet0/2

duplex full

speed 100

switchport access vlan 999

switchport trunk pruning vlan 3-1001

switchport mode trunk

spanning-tree portfast

!



I didn't build the account... I was handed it a few months ago, and am still getting a feel for how things are laid out. This is a recent problem, so until this time, the switch configs hadn't been looked at. I will do some reading, but can you tell me why you would make the change?

scottmilner Mon, 06/09/2008 - 10:32

sure. as I look, it looks like spanning-tree portfast is enabled on each port.


Can you help me out a bit? What is spanning-tree portfast? I read a bit on it from the Cisco website, and without understanding everything fully, it sounded like it is recommended by cisco as a security measure.



cowetacoit Mon, 06/09/2008 - 10:36

portfast should only be used on edge ports, connecting to devices where a network loop is impossible. So a printer or phone would be ok. I don't trust some people so i leave it off. Basically it will "trust" that port not to ever have a network loop. So if someone plugged a patch cable in to the switch and then back into the switch STP would never know to shut one of the ports down. "no spanning-tree portfast default". This may not be the problem but it's a start.


post a SHOW LOG too.

dominic.caron Mon, 06/09/2008 - 10:53

Hi,


STP will block the port even with portfast enable. Problem is traffic will have time to pass before the firtbpdu. I would still recommand bpduguard when using portfast to help preserve the topology.

dominic.caron Mon, 06/09/2008 - 10:37

The link flap output is the number of up-down event since the last reboot of the switch. Is it excessive...depends on your users and uptime.


The first bad thing here is your version. It's a limited deployment train full of bug. You need to use 12.0(5)WC.


For you issue, is it a inter-vlan issue or is it also intra-vlan. If its intra-vlan, could be a spanning tree issue.


Can you do a "sh spanning-tree vlan" for all you vlan and check the number of changes(line 6 or 7 of the output)

cowetacoit Mon, 06/09/2008 - 10:57

Please run....

show logging

and

show spanning-tree vlan


you did spanning-tree vlan 1. Do you have more than one vlan? What are you using for inter vlan routing?

scottmilner Mon, 06/09/2008 - 11:13

Sorry... here are the spanning-tree printouts for the other two vlans.


There is a Cisco 2621 which is the primary router for the school, which I think is also handling the inter vlan routing. I have attached its config also.



scottmilner Mon, 06/09/2008 - 11:25

cowetacoit



Jun 9, 2008, 11:57am PST

Please run....

show logging

and

show spanning-tree vlan


you did spanning-tree vlan 1. Do you have more than one vlan? What are you using for inter vlan routing?


sorry... thread got confusing. show logging is above and show spanning-tree for the other vlans are below.

dominic.caron Mon, 06/09/2008 - 11:00

What about the other vlan?

What's the uptime?


Find your root STP switch and change the priority to 100. Never leave it at the default value.


spanning-tree vlan 1 priority 100

spanning-tree vlan 2 priority 100

etc.


This will cause a topology change. 45 seconds downtime.

scottmilner Mon, 06/09/2008 - 11:22

I posted the other two show spanning-tree vlan configs above.


I know, I am emphasizing my ignorance here, but I would assume that I would want (or hope) that my root STP switch contains the DC's and fileservers for my network... is that a true statement? What exactly will setting its priority to 100 do? Do I just need to be sure its priority is higher than the other three switches?


And I can't thank you enough for the help you and cowetacoit have been giving... Thanks!

cowetacoit Mon, 06/09/2008 - 11:27

The root bridge just controls the STP on your network. Did you disable portfast default?

scottmilner Mon, 06/09/2008 - 11:30

I haven't made any changes yet...I think portfast is still enabled on all active ports. I am taking notes on everything you and dominic are suggesting.


School is still in session, so I will need to implement changes after hours. We have had enough issues in the last couple of weeks that even a minor outage will ruffle a lot of feathers.

Actions

This Discussion