2811 ISR High CPU, HQF Shaper Backg is suspect

mmedwid · ‎09-20-2010

I recently put in a Cisco 2811 for acting as a SIP gateway and it provides Zone firewall. All is working. I am however seeing fairly constant high CPU. The highest proc % appears to be HQF Shaper Backg. What is this and how can I lower the CPU impact?

Also - does anyone understand why the individual proc percents do not add up to the total CPU utilization figure? Do tell!

c2811-1#sho proc cpu
CPU utilization for five seconds: 53%/50%; one minute: 54%; five minutes: 52%

180      121888      51382361      2      1.35%      1.43%      1.34%      0      HQF Shaper Backg
115      321248      307382      1045      1.19%      0.72%      0.85%      0      IP Input
41      21564      207381      103      0.15%      0.15%      0.15%      0      Per-Second Jobs
2      8120      41499      195      0.07%      0.19%      0.16%      0      Load Meter
342      156972      118980      1319      0.07%      0.12%      0.14%      0      SCCP Application
90      267688      207205      1291      0.07%      0.12%      0.13%      0      tCOUNTER
190      14908      56070      265      0.07%      0.10%      0.12%      0      FW DP Inspect pr
327      9512      404794      23      0.07%      0.08%      0.07%      0      IP NAT Ager
181      14652      2069324      7      0.07%      0.07%      0.07%      0      RBSCP Background
126      6276      254228      24      0.07%      0.03%      0.02%      0      Spanning Tree
64      5752      828956      6      0.07%      0.02%      0.00%      0      Netclock Backgro
50      2116      41496      50      0.07%      0.01%      0.00%      0      Compute load avg
153      2596      38679      67      0.07%      0.01%      0.00%      0      TCP Timer
109      2048      28821      71      0.07%      0.00%      0.00%      0      CDP Protocol
188      2492      404687      6      0.07%      0.00%      0.00%      0      Inspect process
332      1792      25084      71      0.07%      0.00%      0.00%      0      IP SLA Mon Sync
230      1236      83      14891      0.00%      0.31%      0.08%      514      SSH Process
5      387280      28658      13513      0.00%      0.16%      0.17%      0      Check heaps
51      105612      3472      30418      0.00%      0.07%      0.05%      0      Per-minute Jobs
317      398956      2439      163573      0.00%      0.05%      0.06%      0      CCSIP_SPI_CONTRO
349      9776      829031      11      0.00%      0.04%      0.03%      0      MLD
339      16228      64851      250      0.00%      0.03%      0.05%      0      NAT MIB Helper
258      7220      1036363      6      0.00%      0.03%      0.02%      0      Atheros LED Ctro
19      43352      124032      349      0.00%      0.01%      0.00%      0      ARP Input
122      904      239      3782      0.00%      0.01%      0.00%      0      AFW_application_
147      3716      318911      11      0.00%      0.01%      0.00%      0      CEF: IPv4 proces
1      392      428      915      0.00%      0.00%      0.00%      0      Chunk Manager
3      0      5      0      0.00%      0.00%      0.00%      0      Collection proce
4      0      1      0      0.00%      0.00%      0.00%      0      EDDRI_MAIN
6      40      22      1818      0.00%      0.00%      0.00%      0      Pool Manager

Tags:

David Aicher · ‎09-20-2010

The output of show process cpu is a little cryptic. For the five second field it is split into two total/interrupt the difference is what we see used for processes. In this case it is 53/50 so the processes should account for about 3%(not super accurate but close)

http://www.cisco.com/en/US/products/sw/iosswrel/ps1828/products_tech_note09186a00800a65d0.shtml#showproccpu

As for interrupts this is typically packet forwarding. There are other things that might fall under interrupt like alignment errors. You can perform show align to see if there are any incrementing. For more information see the following link.

http://www.cisco.com/en/US/products/hw/routers/ps133/products_tech_note09186a00800a70f2.shtml

HQF shaper backg is QOS shaping. It is probably not high enough to worry about given the 1% it is using.

Hopefully that explains it. If you feel this is higher than it should be please check out the performance information in the following document from Miercom.

http://www.miercom.com/dl.html?fid=20061201&type=report

Regards

David Aicher

mmedwid · ‎10-18-2010

If I remove all of the 0.0 utilization PIDs - note that the only ones left are near 1%. Yes they don't

nearly add up to 24% total. Any idea how that math works out? It would seem to me that here in

this snapshot that HQF shaper 1.67 really means a higher percent of CPU than 1.67 percent.

sfo-c2811-1#sho proc cpu | exc 0.0

CPU utilization for five seconds: 24%/19%; one minute: 24%; five minutes: 23%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

5 10660 653 16324 1.03% 0.24% 0.18% 0 Check heaps

115 65640 29008 2262 0.23% 1.39% 0.86% 0 IP Input

180 19504 1058750 18 1.67% 1.57% 1.53% 0 HQF Shaper Backg

343 5504 3578 1538 0.23% 0.21% 0.16% 0 SCCP Application

vragotha · ‎10-18-2010


CPU utilization for five seconds: 24%/19%; one minute: 24%; five minutes: 23%

24%/19% means 24% is overall CPU utilization, of which 19% is due to interrupts, meaning that 5% is utilized by processes

mmedwid · ‎10-19-2010

Thank you. That clarifies a lot!

mmedwid · ‎10-19-2010

So that raises the question what would cause the interrupt portion of CPU utilization to go so high? Here it looks like

processes are only taking up 4% yet the router is at 58% utilization. Thank you.

sfo-c2811-1#sho proc cpu

CPU utilization for five seconds: 58%/54%; one minute: 47%; five minutes: 46%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

1 332 259 1281 0.00% 0.00% 0.00% 0 Chunk Manager

2 12416 14027 885 0.07% 0.08% 0.14% 0 Load Meter

3 4652 1054 4413 0.00% 0.02% 0.00% 0 AFW_application_

4 0 1 0 0.00% 0.00% 0.00% 0 EDDRI_MAIN

5 156908 10102 15532 0.00% 0.21% 0.18% 0 Check heaps

6 64 25 2560 0.00% 0.00% 0.00% 0 Pool Manager

7 0 2 0 0.00% 0.00% 0.00% 0 Timers

8 0 1 0 0.00% 0.00% 0.00% 0 License Client N

David Aicher · ‎10-19-2010

Interrupt level CPU is typically packet forwarding. As long as the CPU is not sustained near 100% it shouldn't cause an issue. If you think it is higher than you expect it to be or you are planning on adding capicity then you may want to check on performance specs.

Here is a link to some third party testing.

http://www.miercom.com/dl.html?fid=20061201&type=report

Regards

Dave Aicher