FastEthernet drops, GigabitEthernet stays up

[iskocic]

Disclaimer: I am a sysadmin and do not have extensive knowledge of switching and routing - just enough for what I do - so please be easy on me  :-)

We have six 3560G switches (WS-C3560G-48TS) on our LAN, all devices on one VLAN.  For the past few months, every week or so, all our 100BASE-T devices lose connection for about a minute, then come back.  At the same time, all gigabit devices stay up, uninterrupted.  This is so puzzling to me that I don't even know where to start troubleshooting.  The devices that lose connection are all connected to different switches and they do not seem to have anything in common, other than 100 Mbit ports. I have checked the logs on the switches and other devices (where i could) and could not find anything.

Does anyone have any suggestionson what to check or look for?

Any suggestion is welcome. Thanks in advance!

Igor

[NickNac79]

Hi Igor,

That's mighty wierd!

The first thing I would do is set up a syslog server, and have all your 3560s sending their logs to that - hopefully you'll catch something happening that will explain the situation.

Obviously you need to have your syslog server on a gigabit connection! :-)

Nick

[iskocic]

Thanks for the suggestion, Nick.  I checked the logging buffer on all the switches and luckily it was large enough to hold the entried from the time of the incident but could not find anything. I did set up the syslog server anyways so it'll be easier to analyze when it happens the next time.

[danrya]

When you say "lose connection" do you mean the "link goes down" or that they just can't communicate with other devices?

If it's the latter, there might be a traffic/broadcast storm that is greater than 100MB, but less than 1GB.  That would cause the 100MB devices to drop packets and lose connectivity.  You'd need to determine the traffic pattern on the devices to see if you can figure out what's causing it.

You might be able to use MRTG (Multi-Router Traffic Grapher) to monitor the switches and see if there's a traffic spike at that time.

Just search for MRTG on the internet.  Let me know if you need help setting up the MRTG system.

Dan

[iskocic]

Hi, Dan.

It's hard to tell whether the links actually go down or the devices just "can't get through".  For some of them the log entries on the switches say that the link is down, but for some others there is no record of the link going down.  I have a system monitoring app that polls all devices every minute and sends me email notifications if something goes down - that's how I know what went down. That, and users complaining :-).  Otherwise, for some of them, just by looking at any available log you wouldn't even know they "went down".

I had the same suspicion as you and thought something was causing high traffic and even looked the backup logs (as the backup was running at the time) but could not find anything.

I appreciate your suggestion and I am downloading MRTG right now and will set it up.  I don't want this to happen again but at the same time I kind of do because I want to know what is going on.  I'll post what (if anything) I find out.

Thanks again.