i posted a while back on an issue we were having where our branches intermittently dropped throughout the day. we thought we resolved the issue by changing a WAN cable however our celebration was shortlived.
every so often our IP Phones drop and our telenet application Sitcks (for a little more than 1 SEC. what we noticed is that everytime that happened there was a throttle recorded.
We are currently doing a virus scan and working with TAC but im not reaching any closer to resolving the issue.
what we have done so far
1. Secure EIGRP
2. Change WAN Cables
3. Change Router
4. Enable storm control - broadcast 20.00
5. check all switching ports to see if there were any errors
6. monitor CPU - Nothing over 25%
7. Disable Multicast -PIM - for our paging application
8. increase hold time
9. increase input queue
10. adjusted buffers with TAC assistance
no matter what the issue is still there. The throttles may happen every two days or sometimes not at all for a week and then come back again. everytime it happens, network is lost for a few seconds.
im looking for some directions into troubleshooting this issue. Should i simple connect a sniffer to the network and see what happens?
again our configs are not at all complex.
HEAD OFFCE TO BRANCH A AND B
HO = 2851-----4 3750
BRANCH A = 2811 - 3750----3550
BRANCH B = 3750
Any pointers would be greatly appreciated.
the carrier also reports no issue and we are not in a position to disagree at this time since we require proof of what is at fault.
it looks like you have already done what is possible.
Have you increased the input hold-queue on the interface.
Do you see input drops on the interface ?
the Throttles definition is :
The output of the show interface atm command might display a high number of throttles along with input queue drops. Input queue drops occur when a packet is being process switched. The throttles counter increments when a system buffer is available, but the interface already has the maximum number of packets waiting to be processed in the input hold queue
>>The router temporarily disables the interface to give the interface time to catch up and process the already enqueued packets.
>> this is the problem you see
You can troubleshoot throttles by determining the root cause of why a high number of packets are being process switched.
For general interfaces a link for troubleshooting input drops
You note both WAN routers and L3 switches, and only a L3 switch in Branch B. Where do you see the throttles? Could you further explain the connectivity between devices? Any QoS/CoS active on any of the devices?
first off we raised the input queue to 120 but that didnt resolve the issue. we also are seeing drops, looks like for every throttle there are about 3 drops...not really consistent as another might be 5 drops etc. - ive read the throttle docs a few times and i understand whats going on but im stumpped as to how to resolve. I also noted that one document stated that Explorer Storms normally causes this type of issue
As for the second reply, the throttles are seen at one of the branches -BRANCH A only.
the baisc architecture is as follows
we have a fibre (ISP) connecting all locations. Head office is a 2851 which is the Hub. BRANCH A is a 2811 and BRANCH B is the L3 Switch.
QOS is enabled -AUTO QOS on all switches and routers. However, QOS on the routers were disabled temporarily while we troubleshoot the issue.
Links are 3MB to each location
Thanks for the responses
PS just logged into the router
Internet address is ***********
MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, media type is T
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters 3d09h
Input queue: 0/75/69/0 (size/max/drops/flushes); Total output drops:
Queueing strategy: fifo
Output queue: 0/40 (size/max)
30 second input rate 4000 bits/sec, 6 packets/sec
30 second output rate 11000 bits/sec, 7 packets/sec
18150098 packets input, 2353919361 bytes, 0 no buffer
Received 140549 broadcasts, 0 runts, 0 giants, 3 throttles
This is a replacement router that we have installed with the defaults so you would not see the input queue adjusted. Cisco asked us if we had another router so we placed this one in the network with the same configuration before the issue happened.
Hi everyone, I would like to thank you in advance for any help you can provide a newcomer like myself!
Im studying the 100-105 book by Odom and am currently on the topic of Port security. I purchased a used 2960 and I'm trying to follow a...
While deploying a number of 18xx/2802/3802 model access points (APs), which run AP-COS as their operating platform. It can be observed on some occasions that while many of their access points were able to join the fabric WLC withou...
I am going to design and build an LAN network under a tunnel underground with long distance between the switches.
I will have 2 Catalyst switches and 8 Industrial IE3000, and they will be connected with fiber.
For now I am planning on use Layer-2 s...