Switch Interfaces Flapping

marvin calceta · ‎10-16-2014

hi,

noticed this on sh loggs:

153255: Oct 16 22:10:00.020 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/37, changed state to down
153256: Oct 16 22:10:00.020 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/41, changed state to down
153257: Oct 16 22:10:00.624 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/10, changed state to down
153258: Oct 16 22:10:00.725 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/11, changed state to down
153259: Oct 16 22:10:00.834 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/12, changed state to down
153260: Oct 16 22:10:00.993 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/13, changed state to down
153261: Oct 16 22:10:01.086 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/14, changed state to down
153262: Oct 16 22:10:01.161 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/31, changed state to down
153263: Oct 16 22:10:01.262 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/1, changed state to down
153264: Oct 16 22:10:01.295 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/2, changed state to down
153265: Oct 16 22:10:01.321 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/3, changed state to down
153266: Oct 16 22:10:01.346 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/4, changed state to down
153267: Oct 16 22:10:01.371 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/5, changed state to down
153268: Oct 16 22:10:01.388 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/6, changed state to down
153269: Oct 16 22:10:01.413 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/7, changed state to down
153270: Oct 16 22:10:01.438 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/8, changed state to down
153271: Oct 16 22:10:01.564 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/25, changed state to down
153272: Oct 16 22:10:01.581 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/26, changed state to down
153273: Oct 16 22:10:01.606 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/27, changed state to down
153274: Oct 16 22:10:01.631 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/28, changed state to down
153275: Oct 16 22:10:01.656 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/29, changed state to down
153276: Oct 16 22:10:01.681 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/30, changed state to down
153277: Oct 16 22:10:01.706 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/9, changed state to down
153278: Oct 16 22:10:01.740 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/33, changed state to down
153279: Oct 16 22:10:02.067 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/34, changed state to down
153280: Oct 16 22:10:02.092 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/35, changed state to down
153281: Oct 16 22:10:02.126 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/36, changed state to down
153282: Oct 16 22:10:02.168 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/37, changed state to down
153283: Oct 16 22:10:02.227 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/38, changed state to down
153284: Oct 16 22:10:02.268 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/39, changed state to down
153285: Oct 16 22:10:02.319 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/40, changed state to down
153286: Oct 16 22:10:02.344 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/41, changed state to down
001669: Oct 16 22:10:02.922 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/1: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001670: Oct 16 22:10:02.939 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/4: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001671: Oct 16 22:10:02.955 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/5: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001672: Oct 16 22:10:03.618 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/27: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001673: Oct 16 22:10:04.063 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/30: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
153287: Oct 16 22:10:05.163 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/25, changed state to up
153288: Oct 16 22:10:05.213 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/2, changed state to up
153289: Oct 16 22:10:05.246 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/7, changed state to up
153290: Oct 16 22:10:06.018 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/3, changed state to up
153291: Oct 16 22:10:06.018 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/6, changed state to up
153292: Oct 16 22:10:06.018 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/28, changed state to up
153293: Oct 16 22:10:06.027 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/29, changed state to up
153294: Oct 16 22:10:06.027 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/9, changed state to up
153295: Oct 16 22:10:06.027 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/10, changed state to up
153296: Oct 16 22:10:06.027 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/33, changed state to up
153297: Oct 16 22:10:06.035 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/35, changed state to up
153298: Oct 16 22:10:06.043 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/39, changed state to up
153299: Oct 16 22:10:06.043 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/26, changed state to up
153300: Oct 16 22:10:06.052 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/36, changed state to up
153301: Oct 16 22:10:06.169 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/25, changed state to up
001674: Oct 16 22:10:06.126 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/31: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001675: Oct 16 22:10:06.168 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/1: Power granted (COGPHMNLCEBU12ACCSW1-3)
001676: Oct 16 22:10:06.185 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/4: Power granted (COGPHMNLCEBU12ACCSW1-3)
001677: Oct 16 22:10:06.202 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/5: Power granted (COGPHMNLCEBU12ACCSW1-3)
153302: Oct 16 22:10:06.220 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/2, changed state to up
153303: Oct 16 22:10:06.253 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/7, changed state to up
153304: Oct 16 22:10:06.295 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/40, changed state to up
001678: Oct 16 22:10:06.210 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/11: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001679: Oct 16 22:10:06.319 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/12: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001680: Oct 16 22:10:06.386 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/13: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001681: Oct 16 22:10:06.411 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/14: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001682: Oct 16 22:10:06.445 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/37: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001683: Oct 16 22:10:06.479 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/38: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
001684: Oct 16 22:10:06.495 MNL: %ILPOWER-7-DETECT: Interface Fa3/0/41: Power Device detected: IEEE PD (COGPHMNLCEBU12ACCSW1-3)
153305: Oct 16 22:10:06.899 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/5, changed state to up
153306: Oct 16 22:10:06.949 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/1, changed state to up
153307: Oct 16 22:10:06.983 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/4, changed state to up
153308: Oct 16 22:10:07.025 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/3, changed state to up
153309: Oct 16 22:10:07.025 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/6, changed state to up
153310: Oct 16 22:10:07.025 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/28, changed state to up
153311: Oct 16 22:10:07.025 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/29, changed state to up
153312: Oct 16 22:10:07.033 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/9, changed state to up
153313: Oct 16 22:10:07.033 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/10, changed state to up
153314: Oct 16 22:10:07.033 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/33, changed state to up
153315: Oct 16 22:10:07.042 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/35, changed state to up
153316: Oct 16 22:10:07.050 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/39, changed state to up
153317: Oct 16 22:10:07.050 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/26, changed state to up
153318: Oct 16 22:10:07.058 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/36, changed state to up
153319: Oct 16 22:10:07.302 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/40, changed state to up
001685: Oct 16 22:10:06.537 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/27: Power granted (COGPHMNLCEBU12ACCSW1-3)
001686: Oct 16 22:10:06.638 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/30: Power granted (COGPHMNLCEBU12ACCSW1-3)
001687: Oct 16 22:10:06.638 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/31: Power granted (COGPHMNLCEBU12ACCSW1-3)
001688: Oct 16 22:10:06.856 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/11: Power granted (COGPHMNLCEBU12ACCSW1-3)
001689: Oct 16 22:10:06.856 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/12: Power granted (COGPHMNLCEBU12ACCSW1-3)
001690: Oct 16 22:10:06.856 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/13: Power granted (COGPHMNLCEBU12ACCSW1-3)
001691: Oct 16 22:10:06.864 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/14: Power granted (COGPHMNLCEBU12ACCSW1-3)
153320: Oct 16 22:10:07.906 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/5, changed state to up
153321: Oct 16 22:10:07.956 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/1, changed state to up
153322: Oct 16 22:10:07.990 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/4, changed state to up
153323: Oct 16 22:10:08.258 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/27, changed state to up
001692: Oct 16 22:10:07.686 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/37: Power granted (COGPHMNLCEBU12ACCSW1-3)
001693: Oct 16 22:10:07.686 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/38: Power granted (COGPHMNLCEBU12ACCSW1-3)
001694: Oct 16 22:10:07.695 MNL: %ILPOWER-5-POWER_GRANTED: Interface Fa3/0/41: Power granted (COGPHMNLCEBU12ACCSW1-3)
153324: Oct 16 22:10:09.265 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/27, changed state to up
153325: Oct 16 22:10:10.062 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/30, changed state to up
153326: Oct 16 22:10:10.196 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/31, changed state to up
153327: Oct 16 22:10:10.355 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/11, changed state to up
153328: Oct 16 22:10:10.447 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/12, changed state to up
153329: Oct 16 22:10:10.481 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/13, changed state to up
153330: Oct 16 22:10:10.582 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/14, changed state to up
153331: Oct 16 22:10:10.657 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/38, changed state to up
153332: Oct 16 22:10:10.724 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/37, changed state to up
153333: Oct 16 22:10:10.817 MNL: %LINK-3-UPDOWN: Interface FastEthernet3/0/41, changed state to up
153334: Oct 16 22:10:11.068 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/30, changed state to up
153335: Oct 16 22:10:11.202 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/31, changed state to up
153336: Oct 16 22:10:11.362 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/11, changed state to up
153337: Oct 16 22:10:11.454 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/12, changed state to up
153338: Oct 16 22:10:11.488 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/13, changed state to up
153339: Oct 16 22:10:11.588 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/14, changed state to up
153340: Oct 16 22:10:11.664 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/38, changed state to up
153341: Oct 16 22:10:11.731 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/37, changed state to up
153342: Oct 16 22:10:11.823 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet3/0/41, changed state to up

switch ports and turning up and down..

POE inline details is good.

any suggestion on the cause?

attached here is the int config.

tnx

Leo Laohoo · ‎10-16-2014

What is the switch model? Is this a chassis or is this a stack of switches? Describe it all.

From the looks of the "sh logs", I am suspecting line card 3 or switch stack member 3 is about to fail.

One way of testing is move some (not all) of the affected ports to different line card slots or switch stack member and observe for 12- or 24 hours.

marvin calceta · ‎10-16-2014

hi mate.

cisco WS-C3750V2-48PS
and yes it is a stack of switches.

Switch Ports Model
------ ----- -----
1 52 WS-C3750V2-48PS
* 2 52 WS-C3750V2-48PS
3 52 WS-C3750V2-48PS
4 52 WS-C3750V2-48PS

and yeah. initially dats what i did.
transferred some affected ports and now im observing.

"From the looks of the "sh logs", I am suspecting line card 3 or switch stack member 3 is about to fail."
=> majority of the affected ports where from Stack 3 though theres a few from stack1 and 2.

see below:

153479: Oct 17 06:42:41.025 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet2/0/38, changed state to up
153480: Oct 17 06:45:02.929 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet2/0/15, changed state to down
153481: Oct 17 06:45:03.818 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet2/0/17, changed state to down
153482: Oct 17 06:45:04.816 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/24, changed state to down
153483: Oct 17 06:45:04.942 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet2/0/15, changed state to up
153484: Oct 17 06:45:05.823 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet2/0/17, changed state to up
153485: Oct 17 06:45:06.829 MNL: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet1/0/24, changed state to up

thanks,

Leo Laohoo · ‎10-16-2014

Ma-ayong buntag.

Ok, post the following commands:

1. sh Interface FastEthernet3/0/41;

2. sh controller e FastEthernet3/0/41;

3. sh version

What are these clients connected to the ports? Are these ordinary PCs?

marvin calceta · ‎10-17-2014

interface FastEthernet3/0/11
switchport access vlan 21
switchport mode access
switchport voice vlan 5
switchport port-security maximum 2
switchport port-security
switchport port-security aging time 2
switchport port-security violation restrict
mls qos trust cos
storm-control broadcast level 5.00
storm-control multicast level 5.00
spanning-tree portfast
end

sh controllers ethernet-controller FastEthernet3/0/11

Transmit FastEthernet3/0/11 Receive
3132104667 Bytes 1723673043 Bytes
228948493 Unicast frames 146177662 Unicast frames
84173051 Multicast frames 1891479 Multicast frames
60417763 Broadcast frames 1026859 Broadcast frames
0 Too old frames 1217342809 Unicast bytes
0 Deferred frames 177244452 Multicast bytes
0 MTU exceeded frames 99109831 Broadcast bytes
0 1 collision frames 0 Alignment errors
0 2 collision frames 0 FCS errors
0 3 collision frames 0 Oversize frames
0 4 collision frames 0 Undersize frames
0 5 collision frames 1 Collision fragments
0 6 collision frames
0 7 collision frames 15007856 Minimum size frames
0 8 collision frames 58472854 65 to 127 byte frames
0 9 collision frames 70483320 128 to 255 byte frames
0 10 collision frames 2704439 256 to 511 byte frames
0 11 collision frames 1166632 512 to 1023 byte frames
0 12 collision frames 1260906 1024 to 1518 byte frames
0 13 collision frames 0 Overrun frames
0 14 collision frames 0 Pause frames
0 15 collision frames
0 Excessive collisions 10 Symbol error frames
0 Late collisions 0 Invalid frames, too large
0 VLAN discard frames 2 Valid frames, too large
0 Excess defer frames 2 Invalid frames, too small
22368663 64 byte frames 0 Valid frames, too small
144811863 127 byte frames
103257477 255 byte frames 0 Too old frames
19742492 511 byte frames 0 Valid oversize frames
20474430 1023 byte frames 0 System FCS error frames
62884382 1518 byte frames 0 RxPortFifoFull drop frame
0 Too large frames
0 Good (1 coll) frames
0 Good (>1 coll) frames

sh int FastEthernet3/0/11
FastEthernet3/0/11 is up, line protocol is up (connected)
Hardware is Fast Ethernet, address is f029.29da.da8d (bia f029.29da.da8d)
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is 10/100BaseTX
input flow-control is off, output flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output 00:00:13, output hang never
Last clearing of "show interface" counters 1d03h
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 329000 bits/sec, 17 packets/sec
578830 packets input, 64544882 bytes, 0 no buffer
Received 66541 broadcasts (5721 multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 5721 multicast, 0 pause input
0 input packets with dribble condition detected
1732840 packets output, 1693511233 bytes, 0 underruns
0 output errors, 0 collisions, 3 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out

sh ver
Cisco IOS Software, C3750 Software (C3750-IPBASEK9-M), Version 12.2(55)SE5, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2012 by Cisco Systems, Inc.

ROM: Bootstrap program is C3750 boot loader
BOOTLDR: C3750 Boot Loader (C3750-HBOOT-M) Version 12.2(53r)SEY4, RELEASE SOFTWARE (fc1)

uptime is 1 year, 28 weeks, 7 hours, 30 minutes
System returned to ROM by power-on

Leo Laohoo · ‎10-17-2014

What are these clients?

Do these interfaces "flap" when the clients are logged off or do these clients also "flap" when they are logged on?