Network Slow

[rickyt00]

Our network has been bogged down lately. It seems there is different things wrong, first there is a ton of errors on an interface. I clear it and about 5 mintues later I copied it(pasted below). I have replaced cables, interfaces, and tried different ports on the switch it is plugged into, but nothing has helped. Also there has been a high cpu load, the arp input and ip input has been very high causes the cpu % to be around 99%. Finally when doing a debug yesterday I found all of these encapsulation failed packets, but have been able to do nothing about it either, below is pasted a copy of my debug and sh interface. Any helped is appreciated, I have hit a wall and I am stumped. Thanks in advance for your help.

MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,

reliability 255/255, txload 185/255, rxload 162/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:01, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:20:17

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 204

Queueing strategy: fifo

Output queue :0/40 (size/max)

5 minute input rate 6391000 bits/sec, 7186 packets/sec

5 minute output rate 7285000 bits/sec, 1473 packets/sec

8526951 packets input, 868033782 bytes, 0 no buffer

Received 321 broadcasts, 0 runts, 0 giants, 0 throttles

175752 input errors, 14 CRC, 14 frame, 0 overrun, 175738 ignored

0 input packets with dribble condition detected

1723888 packets output, 1089896459 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

TCP src=47574, dst=46691, seq=2121718686, ack=0, win8192 SYN

IP: s=a.b.c.d (Ethernet1/0), d=a.b.c.d (Ethernet1/1), g=141.195131.41, len 48 encapsulation failed

TCP src=3250, dst=12833, seq=2121718686, ack=0, win8192 SYN

IP: s=a.b.c.d (Ethernet1/0), d=a.b.c.d (Ethernet1/1), g=141.195131.41, len 48 encapsulation failed

TCP src=52652, dst=19864, seq=2121718686, ack=0, win8192 SYN

IP: s=a.b.c.d (Ethernet1/0), d=a.b.c.d (Ethernet1/1), g=141.195131.41, len 48 encapsulation failed

TCP src=3463, dst=49757, seq=2121718686, ack=0, win8192 SYN

IP: s=a.b.c.d (Ethernet1/0), d=a.b.c.d (Ethernet1/1), g=141.195131.41, len 48 encapsulation failed

TCP src=63161, dst=11542, seq=2121718686, ack=0, win8192 SYN

IP: s=a.b.c.d (Ethernet1/0), d=a.b.c.d (Ethernet1/1), g=141.195131.41, len 48 encapsulation failed

[chuck.price]

What type of equipment is this and how is it configured?

[rickyt00]

The router is a 3620, as far as your configured question. It used to be just one router but since the box was getting so overloaded I put another 3620 in place for the subnet that was killing it so it didn't affect the other subnet. So the bad subnet comes into a 3620 out to another 3620 and then to a 3660 finally to the Internet. I am not sure if that answered your question?

[skarundi]

Here is an explanation of what ignores are:

Number of received packets ignored by the interface because the interface hardware ran low on internal buffers. These buffers are different than the system buffers mentioned previously in the buffer description. Broadcast storms and bursts of noise can cause the ignored count to be increased.

You need to find out what is killing the box.

Sniff the port connected to the interface to find out what is causing the high CPU. I suspect that if you find the cause of the high CPU utilization you'll stop the interface from ignoring packets.

[chuck.price]

You're problem could be a machine that has a bad NIC that is spraying crap into your network. I have seen a computer with a bad NIC take down a subnet.

[rickyt00]

Thanks for your help. Is there a good way to find a bad NIC in the subnet. Thanks again

[jamey]

I assume the router port is connected to a switch.

Check out the switch ports for any weirdness (CRC errors etc.). You might be able to trace back to a possible source of "jabber" by looking at the switch ports for errors and working your way back. e.g. Find a port on a switch with increasing error counts, see what's connected to that port, and work back...

Were there any recent changes to the network?

Other suggestions:

-Replace the ethernet cable to the router.

-try swapping the ethernet cable with the "bad" subnet into the other 3620 that is servicing the "good" subnet. See if you start getting ignored packets and input errors on the known good 3620. If you do this eliminates the other 3620 as being the problem. If you don't see the same things happen on the "good" 3620, it is possible you have a bad ethernet interface in the "bad" 3620.

-Check the switch port connected maybe try locking it down to 10/full.

-Have you tried rebooting the router?

Check out this URL on ignored packets:

http://www.cisco.com/en/US/products/hw/routers/ps221/products_tech_note09186a008009483a.shtml#router_dropping_packets

-HTH

[rickyt00]

I have swapped all possible cables out, swapped interfaces out, changed flash, dram, and rebooted obviously when I was doing all of this but I can't lock it down was it wrong. I looked at the switch that is connected to my router, and one port has some InDiscards but it is not that many at all, and even when disabling that port the load is still 98-99%. The port on the my 6509 switch is locked down at 10/full, and the same on the router. I hooked a packetsniffer up but nothing seemed to out of the ordinary. Thanks though for the help everyone.

[windston]

Hi, try to find which statations on the switch have more ouput traffic in that vlan, them disable all this ports and look for your cpu utilization, If cpu goes down this statios maybe have some virus

Windston