05-27-2009 11:26 PM - edited 03-06-2019 05:58 AM
Hi guys,
I have a 3750 switch that repeated loses connection several times a day. It would be up for hours and then drop in and out for several minutes during which you totally lose connection to it. Even devices within the same subnet are unable to connect to it.
Any ideas why and what to check for? The uptime shows the router does not actually go down and there is nothing in the log.
See below the sh ver output and the logs in the syslog server.
switch1#sh ver
Cisco IOS Software, C3750 Software (C3750-IPSERVICES-M), Version 12.2(25)SEB4, RELEASE SOFTWARE (fc1
)
Copyright (c) 1986-2005 by Cisco Systems, Inc.
Compiled Tue 30-Aug-05 15:47 by yenanh
ROM: Bootstrap program is C3750 boot loader
BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.1(14r)EA1a, RELEASE SOFTWARE (fc1)
switch1 uptime is 1 year, 8 weeks, 5 days, 15 hours, 49 minutes
System returned to ROM by power-on
Syslog
Alert: switch1.hwph.nhs.uk is Down. 06:06
Reset: switch1.hwph.nhs.uk is Up 05:37
Alert: switch1.hwph.nhs.uk is Down. 05:31
Reset: switch1.hwph.nhs.uk is Up 05:17
Alert: switch1.hwph.nhs.uk is Down. 05:11
Reset: switch1.hwph.nhs.uk is Up 04:55
Alert: switch1.hwph.nhs.uk is Down. 04:37
Reset: switch1.hwph.nhs.uk is Up 04:24
...................
....................
...................
Any assistance is much appreciated.
Cheers,
Martha.
05-27-2009 11:53 PM
Martha,
Start looking at SPT flaps:
"sh spanning-tree vlan X detail | inc last" will show if STP is an issue.
also, make sure no debugging is enable annd take a look at CPU history if possible. If CPU is spiking for some reason it would affect vital process like advertising routing.
Another possibility is uplink utilisation.
HTH
Sam
05-28-2009 12:24 AM
Hi Sam,
Thanks for your prompt response.
I have checked and there is definitely no debugging on the switch.
See output of other commands below.
I could not remotely access the switch during the outage so I have not got the cpu utilization for that period.
switch1#sh spanning-tree vlan 200 detail | i last
Number of topology changes 1 last change occurred 5d16h ago
switch1#sh spanning-tree vlan 200 detail
VLAN0200 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 200, address 0018.1855.4500
Configured hello time 2, max age 20, forward delay 15
We are the root of the spanning tree
Topology change flag not set, detected flag not set
Number of topology changes 1 last change occurred 5d16h ago
from StackPort1
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0, aging 300
switch1#sh spanning-tree vlan 200 detail
Port 22 (GigabitEthernet1/0/22) of VLAN0200 is designated forwarding
Port path cost 4, Port priority 128, Port Identifier 128.22.
Designated root has priority 33221, address 0018.1855.4500
Designated bridge has priority 33221, address 0018.1855.4500
Designated port id is 128.22, designated path cost 0
Timers: message age 0, forward delay 0, hold 0
Number of transitions to forwarding state: 1
The port is in the portfast mode
Link type is point-to-point by default
BPDU: sent 560, received 0
Port 64 (GigabitEthernet2/0/12) of VLAN0200 is designated forwarding
Port path cost 19, Port priority 128, Port Identifier 128.64.
Designated root has priority 33221, address 0018.1855.4500
Designated bridge has priority 33221, address 0018.1855.4500
Designated port id is 128.64, designated path cost 0
Timers: message age 0, forward delay 0, hold 0
Number of transitions to forwarding state: 1
The port is in the portfast mode
Link type is shared by default
BPDU: sent 87762, received 0
Port 74 (GigabitEthernet2/0/22) of VLAN0200 is designated forwarding
Port path cost 19, Port priority 128, Port Identifier 128.74.
Designated root has priority 33221, address 0018.1855.4500
Designated bridge has priority 33221, address 0018.1855.4500
Designated port id is 128.74, designated path cost 0
Timers: message age 0, forward delay 0, hold 0
Number of transitions to forwarding state: 1
The port is in the portfast mode
Link type is point-to-point by default
BPDU: sent 72906, received 0
Cheers,
Martha.
05-28-2009 12:49 AM
SPT looks stable, for cpu can you do "sh proc cpu history"
and for uplink, please do "sh int" and check for out put drops and if you have a performance monitoring platform check for spikes on overutilistation.
San
05-28-2009 01:10 AM
Thanks Sam,
Maximum CPU% over the last 72 hours is 30% and since that happened within the last 4 hours, it doesnt appear to have been a cpu issue. I have also checked the uplink ports and there are no drops etc.
It all looks fine to me, am I missing something?
switch1#sh int g2/0/24
GigabitEthernet2/0/24 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0018.1855.4544 (bia 0018.1855.4544)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:37, output 00:00:11, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
4568650 packets input, 496145456 bytes, 0 no buffer
Received 614418 broadcasts (0 multicast)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 614417 multicast, 0 pause input
0 input packets with dribble condition detected
4359147 packets output, 542417294 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
switch1#sh int g1/0/1
GigabitEthernet1/0/1 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is 0018.1855.4542 (bia 0018.1855.4542)
Internet address is 192.168.50.106/30
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 56000 bits/sec, 3 packets/sec
5 minute output rate 62000 bits/sec, 1 packets/sec
86414711 packets input, 418767872 bytes, 0 no buffer
Received 8576495 broadcasts (7962218 IP multicast)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 8576433 multicast, 0 pause input
0 input packets with dribble condition detected
93023508 packets output, 2682384039 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
Cheers,
Martha.
05-28-2009 02:54 AM
It looks ok !
What switch model is it ?
When the issue start, have you done any recent changes ?
Sam
05-28-2009 04:03 AM
It looked okay to me too but I was hoping I had missed something, buggy software...anything.
It is a 3750 and there have been no recent changes, I'm not sure what to check next.
Cheers,
Martha.
05-28-2009 04:09 AM
If you can have console access to the switch it would be possible to troubleshoot while issue is taking place. Might not be this particular switch after all, it might for example be a duplicate IP address elsewhere in your LAN, it could be the upstream switch etc..
Please share the issue when you nail it !
Good luck !
Sam
PS: If you are using VRFs I recall seeing a BUG sometimes ago on IOS for 3750
05-31-2009 10:13 PM
Thanks Sam.
The issue is still happening and I am still none the wiser regarding a resolution.
The device is in a remote site so I have no access to it. We are not using VRF's.
I will keep on digging and I will post a fix if I ever get one.
Thanks,
Martha.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: