Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

Input queue drops

I have a 7206VXR [ NPE-400] running 12.0(16)S8. I am experiecing rather abnormally high input queue drops on a transit network connected to one of the fast ethernets

This is affecting network performance as am seeing so many packet drops.

What could be wrong ?

Here is a sample output captured after 7 minutes of doing clear counters on the interface

gw1#show int fa4/1

FastEthernet4/1 is up, line protocol is up

Hardware is i82543 (Livengood), address is 0008.e260.0871 (bia 0008.e260.0871)

Internet address is 195.xx.xx.xx/26

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, rely 255/255, load 5/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:07:12

Queueing strategy: fifo

Output queue 0/40, 0 drops; input queue 0/75, 21754 drops

5 minute input rate 2966000 bits/sec, 709 packets/sec

5 minute output rate 2192000 bits/sec, 1089 packets/sec

297622 packets input, 158935506 bytes

Received 0 broadcasts, 0 runts, 0 giants, 36473 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast

0 input packets with dribble condition detected

452773 packets output, 110146983 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

  • Other Network Infrastructure Subjects
5 REPLIES
New Member

Re: Input queue drops

You also have alot of throttles.

Looks like you may be hammering your CPU. Do a SHOW PROC CPU and determine your CPU utilization. If you see that IP INPUT is a large amount of CPU time, make sure that fast switching is enabled (IP ROUTE-CACHE) on your interfaces.

If IP INPUT looks ok, find the process with the highest CPU and double check its config.

You may also want to verify that you don't have a modem that is echoing on your AUX or console port as this can kill the CPU. Verify that debugging is turned off...

Mick.

New Member

Re: Input queue drops

The cpu usage is actually less 10% and I have CEF enabled

gw1#show proc cpu

CPU utilization for five seconds: 7%/6%; one minute: 5%; five minutes: 5%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

1 16 791121 0 0.00% 0.00% 0.00% 0 Load Meter

2 255352 7229493 35 0.00% 0.00% 0.00% 0 OSPF Hello

3 627852 402036 1561 0.00% 0.01% 0.00% 0 Check heaps

4 0 1 0 0.00% 0.00% 0.00% 0 Chunk Manager

5 160 3251 49 0.00% 0.00% 0.00% 0 Pool Manager

6 0 2 0 0.00% 0.00% 0.00% 0 Timers

7 14820 819980 18 0.00% 0.00% 0.00% 0 Serial Backgroun

8 41860 3955078 10 0.00% 0.00% 0.00% 0 EnvMon

9 1072 40 26800 0.00% 0.00% 0.00% 0 OIR Handler

10 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager

11 816 3955078 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim

12 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager

13 40092 406876 98 0.00% 0.00% 0.00% 0 ARP Input

14 604 207264 2 0.00% 0.00% 0.00% 0 HC Counter Timer

15 20 3114 6 0.00% 0.00% 0.00% 0 DDR Timers

16 4 7 571 0.00% 0.00% 0.00% 0 Entity MIB API

17 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A'detect

18 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd

19 1433668 68261692 21 0.08% 0.04% 0.02% 0 Net Background

20 588 232715 2 0.00% 0.00% 0.00% 0 Logger

21 2240 3955072 0 0.00% 0.00% 0.00% 0 TTY Background

22 15680 3955077 3 0.00% 0.00% 0.00% 0 Per-Second Jobs

23 0 1 0 0.00% 0.00% 0.00% 0 CSP Timer

24 0 2 0 0.00% 0.00% 0.00% 0 Hawkeye Backgrou

25 0 1 0 0.00% 0.00% 0.00% 0 CES Line Conditi

26 0 2 0 0.00% 0.00% 0.00% 0 Flash MIB Update

27 0 24 0 0.00% 0.00% 0.00% 0 TurboACL

28 1436928 20920000 68 0.08% 0.02% 0.00% 0 IP Input

29 552 1149 480 0.00% 0.00% 0.00% 2 Virtual Exec

30 808 7496 107 0.00% 0.00% 0.00% 0 PPP IP Add Route

31 0 1 0 0.00% 0.00% 0.00% 0 X.25 Encaps Mana

32 0 1 0 0.00% 0.00% 0.00% 0 frr_tunnel

33 0 2 0 0.00% 0.00% 0.00% 0 SSCOP Input

34 0 2 0 0.00% 0.00% 0.00% 0 SSCOP Output

35 0 65922 0 0.00% 0.00% 0.00% 0 SSCOP Timer

36 204000 456880 446 0.00% 0.00% 0.00% 0 IP Background

37 0 2 0 0.00% 0.00% 0.00% 0 ILMI Input

38 0 1 0 0.00% 0.00% 0.00% 0 SNMP Timers

39 0 2 0 0.00% 0.00% 0.00% 0 ILMI Request

40 0 2 0 0.00% 0.00% 0.00% 0 ILMI Response

41 0 1 0 0.00% 0.00% 0.00% 0 ILMI Timer Proce

42 0 2 0 0.00% 0.00% 0.00% 0 ATM PVC Discover

43 0 2 0 0.00% 0.00% 0.00% 0 ATMSIG DRIVERAPI

44 5188 66179 78 0.00% 0.00% 0.00% 0 Adj Manager

45 44 43365 1 0.00% 0.00% 0.00% 0 TCP Timer

46 216 1903 113 0.00% 0.00% 0.00% 0 TCP Protocols

47 0 1 0 0.00% 0.00% 0.00% 0 Probe Input

48 0 29 0 0.00% 0.00% 0.00% 0 RARP Input

49 19516 241844 80 0.00% 0.00% 0.00% 0 BOOTP Server

50 202956 65920 3078 0.00% 0.00% 0.00% 0 IP Cache Ager

51 0 1 0 0.00% 0.00% 0.00% 0 PAD InCall

52 0 2 0 0.00% 0.00% 0.00% 0 X.25 Background

53 0 65921 0 0.00% 0.00% 0.00% 0 TCP Intercept Ti

54 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers

55 0 2 0 0.00% 0.00% 0.00% 0 TC-ATM Proc

56 0 2 0 0.00% 0.00% 0.00% 0 Tag Input

57 872 1319460 0 0.00% 0.00% 0.00% 0 LSP Tunnel Head

58 0 1 0 0.00% 0.00% 0.00% 0 ISDN Timer

59 5976 13572 440 0.24% 0.03% 0.00% 0 IP SNMP

60 0 1 0 0.00% 0.00% 0.00% 0 Encrypt Proc

61 0 5 0 0.00% 0.00% 0.00% 0 Key Proc

62 0 1 0 0.00% 0.00% 0.00% 0 Crypto Support

63 0 3 0 0.00% 0.00% 0.00% 0 Crypto CA

64 0 1 0 0.00% 0.00% 0.00% 0 SYSMGT Events

65 0 1 0 0.00% 0.00% 0.00% 0 CallMIB Backgrou

66 0 1 0 0.00% 0.00% 0.00% 0 ISDNMIB Backgrou

67 0 1 0 0.00% 0.00% 0.00% 0 SNMP ConfCopyPro

68 0 1 0 0.00% 0.00% 0.00% 0 Syslog Traps

69 491568 45042885 10 0.00% 0.01% 0.00% 0 Net Input

70 628 791123 0 0.00% 0.00% 0.00% 0 Compute load avg

71 523520 65920 7941 0.00% 0.00% 0.00% 0 Per-minute Jobs

72 352 22085 15 0.00% 0.00% 0.00% 0 CEF process

73 0 2 0 0.00% 0.00% 0.00% 0 CCP manager

74 236600 8803670 26 0.00% 0.00% 0.00% 0 PPP manager

75 20 3955557 0 0.00% 0.00% 0.00% 0 Multilink PPP

76 0 2 0 0.00% 0.00% 0.00% 0 Multilink PPP ou

77 0 2 0 0.00% 0.00% 0.00% 0 Multilink event

78 0 2 0 0.00% 0.00% 0.00% 0 IP Flow Backgrou

79 0 459 0 0.00% 0.00% 0.00% 0 CEF Scanner

80 91256 65921 1384 0.00% 0.00% 0.00% 0 OSPF Scanner

81 488 3990072 0 0.00% 0.00% 0.00% 0 NTP

82 0 1 0 0.00% 0.00% 0.00% 0 xcpa-driver

83 312 7219 43 0.00% 0.00% 0.00% 0 PPP auth

84 186088 7093554 26 0.00% 0.00% 0.00% 0 OSPF Router

85 0 18915 0 0.00% 0.00% 0.00% 0 TACACS+

86 0 1 0 0.00% 0.00% 0.00% 0 SNMP Traps

87 8 38 210 0.00% 0.00% 0.00% 3 Virtual Exec

88 56 829 67 0.16% 0.02% 0.00% 4 Virtual Exec

gw1#show proc cpu | in ^PID|Input

13 40092 406881 98 0.00% 0.00% 0.00% 0 ARP Input

28 1436936 20920185 68 0.08% 0.03% 0.00% 0 IP Input

33 0 2 0 0.00% 0.00% 0.00% 0 SSCOP Input

37 0 2 0 0.00% 0.00% 0.00% 0 ILMI Input

47 0 1 0 0.00% 0.00% 0.00% 0 Probe Input

48 0 29 0 0.00% 0.00% 0.00% 0 RARP Input

56 0 2 0 0.00% 0.00% 0.00% 0 Tag Input

69 491568 45043136 10 0.00% 0.00% 0.00% 0 Net Input

gw1#show debugging

gw1#show int fa4/1

FastEthernet4/1 is up, line protocol is up

Hardware is i82543 (Livengood), address is 0008.e260.0871 (bia 0008.e260.0871)

Internet address is 195.xx.xx.xx/26

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, rely 255/255, load 6/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:30:01

Queueing strategy: fifo

Output queue 0/40, 0 drops; input queue 0/75, 96461 drops

5 minute input rate 3038000 bits/sec, 761 packets/sec

5 minute output rate 2384000 bits/sec, 1132 packets/sec

1339150 packets input, 674184391 bytes

Received 0 broadcasts, 0 runts, 0 giants, 158314 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast

0 input packets with dribble condition detected

2071255 packets output, 542445301 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

gw1#show ip interface fastEthernet 4/1

FastEthernet4/1 is up, line protocol is up

Internet address is 195.xx.xx.xx/26

Broadcast address is 255.255.255.255

Address determined by setup command

MTU is 1500 bytes

Helper address is not set

Directed broadcast forwarding is disabled

Multicast reserved groups joined: 224.0.0.5 224.0.0.6

Outgoing access list is not set

Inbound access list is not set

Proxy ARP is enabled

Security level is default

Split horizon is enabled

ICMP redirects are always sent

ICMP unreachables are always sent

ICMP mask replies are never sent

IP fast switching is enabled

IP fast switching on the same interface is disabled

IP Flow switching is enabled

IP Flow switching turbo vector

IP Null turbo vector

IP multicast fast switching is enabled

IP multicast distributed fast switching is disabled

IP route-cache flags are Fast, Flow, No CEF

Router Discovery is disabled

IP output packet accounting is disabled

IP access violation accounting is disabled

TCP/IP header compression is disabled

RTP/IP header compression is disabled

Probe proxy name replies are disabled

Policy routing is disabled

Network address translation is disabled

WCCP Redirect outbound is disabled

WCCP Redirect inbound is disabled

WCCP Redirect exclude is disabled

BGP Policy Mapping is disabled

New Member

Re: Input queue drops

You're right... that all looks good. Check your buffers too to make sure you're not getting buffer failures.

You can try increasing the size of your input queue. For bursty traffic, this should work. Increase it to 200 and see what happens (hold-queue 200 in). If your input queue drops continue to rise at the same rate, change your input queue back to 75 as increasing this number will hurt you when constant traffic exceeds the router's ability to move it.

Have you verified with a packet sniffer, that the traffic on that segment is "good"? You may have a rogue process flooding your network.

Here are some other ideas:

http://www.cisco.com/warp/public/63/queue_drops.html

Mick.

New Member

Re: Input queue drops

increasing of the input queue depth doesnt help, the drops and throttles are still are occuring

gw1#sh int fa4/1

FastEthernet4/1 is up, line protocol is up

Hardware is i82543 (Livengood), address is 0008.e260.0871 (bia 0008.e260.0871)

Internet address is 195.xx.xx.xx/26

MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, rely 255/255, load 7/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 100Mb/s, 100BaseTX/FX

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:11:18

Queueing strategy: fifo

Output queue 0/40, 0 drops; input queue 0/75, 40293 drops

5 minute input rate 2989000 bits/sec, 828 packets/sec

5 minute output rate 3137000 bits/sec, 1321 packets/sec

566372 packets input, 256486626 bytes

Received 0 broadcasts, 0 runts, 0 giants, 63332 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

0 watchdog, 0 multicast

0 input packets with dribble condition detected

892559 packets output, 265881776 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

gw1#

New Member

Re: Input queue drops

You may also want to verify that you don't have a modem that is echoing on your AUX or console port as this can kill the CPU.

Can you please explain your above comment a little more. I think I may have seen this before.

Thanks in advance

Kevin

297
Views
0
Helpful
5
Replies