CPU doubles going from 12.4(4)T1 to 12.4(11)T1

hank · ‎02-26-2007

We have a 7206VXR NPE-G1 which was running 12.4(4)T1 happily at around 25% CPU utilization. With the latest Cisco security patch alerts - we upgraded the IOS to 12.4(11)T1 and CPU immediately doubled. No config change.

It runs with C7200-ADVIPSERVICESK9-M. "sho proc cpu sorted" shows nothing higher than 1.5%.

It has a few GigaE interfaces and some FEs and pkt rates are as follows:

30 second input rate 92974000 bits/sec, 17418 packets/sec

30 second output rate 88059000 bits/sec, 17063 packets/sec

30 second input rate 102690000 bits/sec, 18859 packets/sec

30 second output rate 95586000 bits/sec, 18797 packets/sec

30 second input rate 1548000 bits/sec, 663 packets/sec

30 second output rate 6152000 bits/sec, 739 packets/sec

30 second input rate 2575000 bits/sec, 1025 packets/sec

30 second output rate 9037000 bits/sec, 1243 packets/sec

Many of the interfaces run NBAR (protocol discovery) as well as to rate-limit P2P. But all that ran fine pre-upgrade. Now CPU runs at 75%.

Ideas?

Thanks!

Danilo Dy · ‎02-26-2007

Can you execute "show proc cpu" and check which process/service is getting too much attention from CPU?

hank · ‎02-26-2007

None:

CPU utilization for five seconds: 67%/63%; one minute: 72%; five minutes: 71%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

71 501000 1496866 334 1.31% 1.29% 1.32% 0 IP Input

230 25716 1439400 17 0.98% 0.98% 0.98% 0 traffic_shape

231 139288 409570 340 0.32% 0.32% 0.32% 0 SDP Process

54 6212 3933515 1 0.32% 0.34% 0.32% 0 T3E3 EC IPC POLL

44 52228 20075 2601 0.24% 0.24% 0.24% 0 Per-Second Jobs

229 2196 37896 57 0.08% 0.03% 0.02% 0 PIM Process

94 16 566 28 0.08% 0.00% 0.00% 0 TCP Timer

227 748 20051 37 0.08% 0.01% 0.00% 0 IGMP Input

232 2536 618642 4 0.08% 0.08% 0.08% 0 PPP manager

10 0 331 0 0.00% 0.00% 0.00% 0 IPC Dynamic Cach

9 0 2 0 0.00% 0.00% 0.00% 0 ATM VC Auto Crea

12 44 19823 2 0.00% 0.00% 0.00% 0 IPC Periodic Tim

11 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager

13 24 19823 1 0.00% 0.00% 0.00% 0 IPC Deferred Por

15 0 1 0 0.00% 0.00% 0.00% 0 IPC BackPressure

16 304 19823 15 0.00% 0.00% 0.00% 0 EnvMon

17 0 1 0 0.00% 0.00% 0.00% 0 OIR Handler

14 0 1 0 0.00% 0.00% 0.00% 0 IPC Seat Manager

19 1296 5841 221 0.00% 0.00% 0.00% 0 ARP Input

20 800 20690 38 0.00% 0.00% 0.00% 0 ARP Background

Danilo Dy · ‎02-26-2007

Use the troubleshooting recommendation in this link and let us know the outcome http://www.cisco.com/warp/public/63/highcpu.html#symptons

hank · ‎03-01-2007

That link was of little use. The CPU doesn't indicate any problem. No individual process over 2%.

What is interesting is "show inter switching", but Output Intepreter didn't yield any insights. See attachment.