High CPU consuming

Unanswered Question
Jun 16th, 2011

Dear All

I have 7200 router, it was runing 124-4.XD3 IOS, the CPU was working normal avarage around 15%, its use for L2TP users, it was have avarage 2000 concurrent L2TP connection, there was no CPU consuming.

Then for business plan we change the IOS to 12.2(33)SR and configured Broadband aggregation, after that the CPU become higher day after day even the PPPoE and L2TP session not more than 900 concurrent. the CPU reach 60%, even in non-peak hours it reach 50%

-------

sh ver
Cisco IOS Software, 7200 Software (C7200P-ADVIPSERVICESK9-M), Version 12.2(33)SR
C6, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2010 by Cisco Systems, Inc.
Compiled Mon 08-Mar-10 08:03 by prod_rel_team

ROM: System Bootstrap, Version 12.4(12.2r)T, RELEASE SOFTWARE (fc1)
BOOTLDR: Cisco IOS Software, 7200 Software (C7200P-KBOOT-M), Version 12.2(33)SRC
6, RELEASE SOFTWARE (fc1)

System returned to ROM by reload at 14:58:14 UTC Thu Jun 2 2011
System image file is "disk2:c7200p-advipservicesk9-mz.122-33.SRC6.bin"
Last reload reason: Reload command

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

Cisco 7206VXR (NPE-G2) processor (revision A) with 917504K/65536K bytes of memor
y.
Processor board ID 36743260
MPC7448 CPU at 1666Mhz, Implementation 0, Rev 2.2
6 slot VXR midplane, Version 2.11

Last reset from power-on

PCI bus mb1 (Slots 1, 3 and 5) has a capacity of 600 bandwidth points.
Current configuration on bus mb1 has a total of 0 bandwidth points.
This configuration is within the PCI bus capacity and is supported.

PCI bus mb2 (Slots 2, 4 and 6) has a capacity of 600 bandwidth points.
Current configuration on bus mb2 has a total of 0 bandwidth points.
This configuration is within the PCI bus capacity and is supported.

Please refer to the following document "Cisco 7200 Series Port Adaptor
Hardware Configuration Guidelines" on Cisco.com <http://www.cisco.com>
for c7200 bandwidth points oversubscription and usage guidelines.


1 FastEthernet interface
3 Gigabit Ethernet interfaces
2045K bytes of NVRAM.

250880K bytes of ATA PCMCIA card at slot 2 (Sector size 512 bytes).
65536K bytes of Flash internal SIMM (Sector size 512K).
Configuration register is 0x2102

---------------------------------

sh processes cpu

CPU utilization for five seconds: 73%/17%; one minute: 67%; five minutes: 66%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

   1         620     11401         54  0.00%  0.00%  0.00%   0 Chunk Manager   

   2        1400    238221          5  0.00%  0.00%  0.00%   0 Load Meter      

   3           0         1          0  0.00%  0.00%  0.00%   0 L2X Socket proce

   4     4594808    312806      14689  7.27%  0.99%  0.69%   0 Check heaps     

   5       36380    195739        185  0.07%  0.00%  0.00%   0 Pool Manager    

   6           0         2          0  0.00%  0.00%  0.00%   0 Timers          

   7     1789408   3141124        569  0.15%  0.16%  0.16%   0 ARP Input       

   8         408   1241861          0  0.00%  0.00%  0.00%   0 ARP Background  

   9           0         2          0  0.00%  0.00%  0.00%   0 ATM Idle Timer  

  10           0         1          0  0.00%  0.00%  0.00%   0 ATM ASYNC PROC  

  11           0        85          0  0.00%  0.00%  0.00%   0 AAA_SERVER_DEADT

  12           0         1          0  0.00%  0.00%  0.00%   0 Policy Manager  

  13           0         4          0  0.00%  0.00%  0.00%   0 Entity MIB API  

  14       87356   1933514         45  0.00%  0.00%  0.00%   0 EEM ED Syslog   

  15           8     19853          0  0.00%  0.00%  0.00%   0 IPC Dynamic Cach

  16           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager

  17        1276    237895          5  0.00%  0.00%  0.00%   0 IPC Service NonC

  18         896   1189354          0  0.00%  0.00%  0.00%   0 IPC Periodic Tim

  19           0         1          0  0.00%  0.00%  0.00%   0 IPC Managed Time

  20         644   1189353          0  0.00%  0.00%  0.00%   0 IPC Deferred Por

  21           0         1          0  0.00%  0.00%  0.00%   0 IPC Seat Manager

  22           0         1          0  0.00%  0.00%  0.00%   0 IPC Seat Control

  23          88    238222          0  0.00%  0.00%  0.00%   0 IPC Loadometer  

  24           0         2          0  0.00%  0.00%  0.00%   0 Serial Backgroun

  25           0         1          0  0.00%  0.00%  0.00%   0 RO Notify Timers

  26           0         1          0  0.00%  0.00%  0.00%   0 RMI RM Notify Wa

  27         472    151083          3  0.00%  0.00%  0.00%   0 DDR Timers      

  28           0         2          0  0.00%  0.00%  0.00%   0 ATM AutoVC Perio

  29           0         2          0  0.00%  0.00%  0.00%   0 ATM VC Auto Crea

  30          16    119115          0  0.00%  0.00%  0.00%   0 Compute SRP rate

  31     2400080  21414450        112  0.00%  0.01%  0.08%   0 EnvMon          

  32           0         1          0  0.00%  0.00%  0.00%   0 OIR Handler     

  33           0         1          0  0.00%  0.00%  0.00%   0 Crash writer    

  34           0         1          0  0.00%  0.00%  0.00%   0 Exception contro

  35        2200    237093          9  0.00%  0.00%  0.00%   0 ALARM_TRIGGER_SC

  36           0         1          0  0.00%  0.00%  0.00%   0 CEF MIB API     

  37         948   1189356          0  0.00%  0.00%  0.00%   0 GraphIt         

  38           0         1          0  0.00%  0.00%  0.00%   0 SERIAL A'detect 

  39           0         2          0  0.00%  0.00%  0.00%   0 Dialer event    

  40           0         2          0  0.00%  0.00%  0.00%   0 SMART           

  41           0         1          0  0.00%  0.00%  0.00%   0 RFS server proce

  42           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd  

  43      112568    578621        194  0.00%  0.04%  0.05%   0 Net Background  

  44           0         2          0  0.00%  0.00%  0.00%   0 IDB Work        

  45       23576    664891         35  0.00%  0.00%  0.00%   0 Logger          

  46        1564   1189352          1  0.00%  0.00%  0.00%   0 TTY Background  

  47       38788   1189374         32  0.00%  0.03%  0.02%   0 Per-Second Jobs 

  48       80568     23708       3398  0.00%  0.00%  0.00%   0 Per-minute Jobs 

  49         776         5     155200  0.00%  0.00%  0.00%   0 PrstVbl         

  50        5160    154631         33  0.00%  0.00%  0.00%   0 IF-MGR control p

  51           0        15          0  0.00%  0.00%  0.00%   0 IF-MGR event pro

  52           0         1          0  0.00%  0.00%  0.00%   0 Inode Table Dest

  53           4     13677          0  0.00%  0.00%  0.00%   0 Transport Port A

  54           0         2          0  0.00%  0.00%  0.00%   0 INTR MGR PROCESS

  55     1553620    253665       6124  0.00%  0.13%  0.12%   0 HC Counter Timer

  56           0         2          0  0.00%  0.00%  0.00%   0 CES Timer       

  57           0         1          0  0.00%  0.00%  0.00%   0 POS APS Event Pr

  58           0         1          0  0.00%  0.00%  0.00%   0 SONET alarm time

  59           0         2          0  0.00%  0.00%  0.00%   0 Hawkeye Backgrou

  60           0         1          0  0.00%  0.00%  0.00%   0 CSP Timer       

  61           0         2          0  0.00%  0.00%  0.00%   0 FPD Management P

  62           0         1          0  0.00%  0.00%  0.00%   0 FPD Action Proce

  63        1108     74393         14  0.00%  0.00%  0.00%   0 Collection proce

  64           0         1          0  0.00%  0.00%  0.00%   0 CES Line Conditi

  65           0         2          0  0.00%  0.00%  0.00%   0 Flash MIB Update

  66           4         6        666  0.00%  0.00%  0.00%   0 Flash Card Oir  

  67           0         1          0  0.00%  0.00%  0.00%   0 Async write proc

  68           0         1          0  0.00%  0.00%  0.00%   0 Async write proc

  69           0         1          0  0.00%  0.00%  0.00%   0 Async write proc

  70           0         1          0  0.00%  0.00%  0.00%   0 Async write proc

  71           8     19853          0  0.00%  0.00%  0.00%   0 DFS flush period

  72     1190120  12956856         91  0.23%  0.23%  0.22%   0 AAA Server      

  73       37948    166779        227  0.00%  0.00%  0.00%   0 AAA ACCT Proc   

  74        3796  36930272          0  0.00%  0.00%  0.00%   0 ACCT Periodic Pr

  75           0         1          0  0.00%  0.00%  0.00%   0 AAA System Acct 

  76           0         1          0  0.00%  0.00%  0.00%   0 tHUB            

  77           0         2          0  0.00%  0.00%  0.00%   0 AAA Dictionary R

  78           0         2          0  0.00%  0.00%  0.00%   0 CEF switching ba

  79           0         2          0  0.00%  0.00%  0.00%   0 IPAM/ODAP Events

  80        4792  36930277          0  0.00%  0.01%  0.00%   0 IPAM Manager    

  81           0         2          0  0.00%  0.00%  0.00%   0 IPAM Events     

  82           0        15          0  0.00%  0.00%  0.00%   0 IP ARP Adjacency

  83     4277228  78191448         54  0.95%  0.95%  0.92%   0 IP Input        

  84           0         1          0  0.00%  0.00%  0.00%   0 ICMP event handl

  85           0         5          0  0.00%  0.00%  0.00%   0 TurboACL        

  86           0         2          0  0.00%  0.00%  0.00%   0 TurboACL chunk  

  87         168      3204         52  0.00%  0.00%  0.00%   0 ADJ resolve proc

  88           0         1          0  0.00%  0.00%  0.00%   0 IPv6 Echo event 

  89           0         1          0  0.00%  0.00%  0.00%   0 LSP Tunnel FRR  

  90           0         1          0  0.00%  0.00%  0.00%   0 MPLS Auto-Tunnel

  91           0         1          0  0.00%  0.00%  0.00%   0 O-UNI Client Msg

  92           0         1          0  0.00%  0.00%  0.00%   0 AAA EPD HANDLER 

  93           0         1          0  0.00%  0.00%  0.00%   0 PM EPD API      

  94           0         5          0  0.00%  0.00%  0.00%   0 PPP SIP         

  95       14144    101572        139  0.00%  0.00%  0.00%   0 PPP Bind        

  96       33920    303835        111  0.00%  0.00%  0.00%   0 PPP IP Route    

  98     1568632  37728948         41  0.15%  0.13%  0.11%   0 SSM connection m

  99           0         2          0  0.00%  0.00%  0.00%   0 Spanning Tree   

100           0         2          0  0.00%  0.00%  0.00%   0 ATM OAM Input   

101           0         2          0  0.00%  0.00%  0.00%   0 ATM OAM TIMER   

102          28      1987         14  0.00%  0.00%  0.00%   0 MOP Protocols   

103     6304800  22987724        274  1.03%  0.88%  0.81%   0 SSS Manager     

104           0         1          0  0.00%  0.00%  0.00%   0 SSS Policy Manag

105       22984    468723         49  0.00%  0.00%  0.00%   0 SSS Feature Mana

106         860   4652544          0  0.00%  0.00%  0.00%   0 SSS Feature Time

107           0         1          0  0.00%  0.00%  0.00%   0 SG DPM          

108           0         1          0  0.00%  0.00%  0.00%   0 Timer handler fo

109           0         1          0  0.00%  0.00%  0.00%   0 Prepaid response

110           0         1          0  0.00%  0.00%  0.00%   0 Timed Policy act

111           0         1          0  0.00%  0.00%  0.00%   0 AAA response han

112           0         2          0  0.00%  0.00%  0.00%   0 KRB5 AAA        

113           0         1          0  0.00%  0.00%  0.00%   0 delayed evt hand

114           0         1          0  0.00%  0.00%  0.00%   0 AC Switch       

115           0         1          0  0.00%  0.00%  0.00%   0 SG CMD HANDLER  

116           0         2          0  0.00%  0.00%  0.00%   0 IP PORTBUNDLE   

117          52    290816          0  0.00%  0.00%  0.00%   0 IP SIP Process  

118        4044  36930299          0  0.00%  0.00%  0.00%   0 RADIUS Proxy    

119           0         1          0  0.00%  0.00%  0.00%   0 TC SIP          

120          72     23792          3  0.00%  0.00%  0.00%   0 CEF background p

121           0         1          0  0.00%  0.00%  0.00%   0 IP IRDP         

122           0         2          0  0.00%  0.00%  0.00%   0 SNMP Timers     

123       42784    919143         46  0.00%  0.00%  0.00%   0 IP Background   

124      124252    846556        146  0.00%  0.00%  0.00%   0 IP RIB Update   

125          32       998         32  0.00%  0.00%  0.00%   2 Virtual Exec    

126        4900    178197         27  0.00%  0.00%  0.00%   0 VPDN failure man

127           0         1          0  0.00%  0.00%  0.00%   0 IP Traceroute   

128           0         1          0  0.00%  0.00%  0.00%   0 Socket Timers   

129        1644     27024         60  0.00%  0.00%  0.00%   0 DHCPD Receive   

130          52      2068         25  0.00%  0.00%  0.00%   0 static          

131         240     49569          4  0.00%  0.00%  0.00%   0 TCP Timer       

132        1696      7889        214  0.00%  0.00%  0.00%   0 TCP Protocols   

133           0      3971          0  0.00%  0.00%  0.00%   0 HTTP CORE       

134       10896 293774766          0  0.07%  0.07%  0.07%   0 HQF Shaper Backg

135        6972 293774795          0  0.07%  0.06%  0.07%   0 HQF Input Shaper

136           0         2          0  0.00%  0.00%  0.00%   0 LFDp Input Proc 

137           0         2          0  0.00%  0.00%  0.00%   0 SCTP Main Proces

138           0         2          0  0.00%  0.00%  0.00%   0 PPP Compress Inp

139           0         2          0  0.00%  0.00%  0.00%   0 PPP Compress Res

140           0         1          0  0.00%  0.00%  0.00%   0 COPS            

141           0         2          0  0.00%  0.00%  0.00%   0 Dialer Forwarder

142           0         3          0  0.00%  0.00%  0.00%   0 Flow Exporter Ti

143           0         1          0  0.00%  0.00%  0.00%   0 RARP Input      

144           0         2          0  0.00%  0.00%  0.00%   0 PPP NBF         

145           0         1          0  0.00%  0.00%  0.00%   0 PAD InCall      

146           0         2          0  0.00%  0.00%  0.00%   0 X.25 Background 

147           0         1          0  0.00%  0.00%  0.00%   0 X.25 Encaps Mana

148        2452  11862393          0  0.00%  0.00%  0.00%   0 RBSCP Background

149           0         1          0  0.00%  0.00%  0.00%   0 VPDN call manage

150      748224   1825397        409  0.23%  0.16%  0.16%   0 CEF: IPv4 proces

151        3408     77652         43  0.00%  0.00%  0.00%   0 ADJ background  

152           0         1          0  0.00%  0.00%  0.00%   0 ac_atm_state_eve

153           0         2          0  0.00%  0.00%  0.00%   0 IP Inband Sessio

154           0         1          0  0.00%  0.00%  0.00%   0 EM Background Pr

155           0         1          0  0.00%  0.00%  0.00%   0 IDMGR CORE      

156         240      4710         50  0.00%  0.00%  0.00%   0 LOCAL AAA       

157           0         5          0  0.00%  0.00%  0.00%   0 MPLS Auto Mesh P

158         360       219       1643  0.00%  0.20%  0.07%   3 Virtual Exec    

159           0         2          0  0.00%  0.00%  0.00%   0 ENABLE AAA      

160           0         2          0  0.00%  0.00%  0.00%   0 LINE AAA        

161           0         2          0  0.00%  0.00%  0.00%   0 TPLUS           

162           0         2          0  0.00%  0.00%  0.00%   0 Crypto Support  

163           0        12          0  0.00%  0.00%  0.00%   0 crypto engine pr

164           0         1          0  0.00%  0.00%  0.00%   0 Key Proc        

165           0         6          0  0.00%  0.00%  0.00%   0 Crypto ACL      

166           0         1          0  0.00%  0.00%  0.00%   0 Crypto INT      

167           0         4          0  0.00%  0.00%  0.00%   0 Crypto IKMP     

168          16     59560          0  0.00%  0.00%  0.00%   0 IPSEC key engine

169           0         1          0  0.00%  0.00%  0.00%   0 IPSEC manual key

170           0         1          0  0.00%  0.00%  0.00%   0 Crypto PAS Proc 

171           0         4          0  0.00%  0.00%  0.00%   0 Crypto CA       

172           0         1          0  0.00%  0.00%  0.00%   0 Crypto PKI-CRL  

173           0         1          0  0.00%  0.00%  0.00%   0 Crypto SSL      

174           0         1          0  0.00%  0.00%  0.00%   0 encrypt proc    

175           0         1          0  0.00%  0.00%  0.00%   0 GDOI GM Process 

176      362624   4767399         76  0.00%  0.02%  0.00%   0 AAA SEND STOP EV

177           0         2          0  0.00%  0.00%  0.00%   0 EEM ED Resource 

178           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Track    

179           0         1          0  0.00%  0.00%  0.00%   0 Syslog Traps    

180          56    119107          0  0.00%  0.00%  0.00%   0 RMON Recycle Pro

181           0         2          0  0.00%  0.00%  0.00%   0 RMON Deferred Se

182          24       138        173  0.00%  0.01%  0.00%   4 Virtual Exec    

183           0         1          0  0.00%  0.00%  0.00%   0 MV64 TDR Process

184           0        37          0  0.00%  0.00%  0.00%   0 EEM Server      

185           0         2          0  0.00%  0.00%  0.00%   0 EEM Policy Direc

186           0         3          0  0.00%  0.00%  0.00%   0 EEM ED CLI      

187           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Config   

188           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Counter  

189           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Env      

190           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Interface

191           0         3          0  0.00%  0.00%  0.00%   0 EEM ED IOSWD    

192           0         3          0  0.00%  0.00%  0.00%   0 EEM ED None     

193           0         3          0  0.00%  0.00%  0.00%   0 EEM ED OIR      

194           0         2          0  0.00%  0.00%  0.00%   0 tENM            

195           0         3          0  0.00%  0.00%  0.00%   0 EEM ED SNMP     

196           0         3          0  0.00%  0.00%  0.00%   0 EEM ED Test     

197           4     29860          0  0.00%  0.00%  0.00%   0 EEM ED Timer    

198          12     56583          0  0.00%  0.00%  0.00%   0 IP SLA MPLSLM Pr

199          20       305         65  0.00%  0.00%  0.00%   0 Syslog          

200           0         1          0  0.00%  0.00%  0.00%   0 RMON Packets    

201           0         1          0  0.00%  0.00%  0.00%   0 IP SLAs Ethernet

202           0         1          0  0.00%  0.00%  0.00%   0 IP SLA AutoEther

203         264     22608         11  0.00%  0.00%  0.00%   0 Net Input       

204     5244200    388598      13495  0.47%  0.44%  0.43%   0 Compute load avg

205           8         5       1600  0.00%  0.00%  0.00%   0 crypto sw pk pro

207     1398812 547611831          2  5.27%  5.27%  5.24%   0 L2X Data Daemon 

208      447680    774468        578  0.07%  0.05%  0.04%   0 L2TP mgmt daemon

209      213684    638194        334  0.07%  0.02%  0.00%   0 L2TUN Applicatio

210           0         1          0  0.00%  0.00%  0.00%   0 VPDN Test       

211     1440332  27764880         51  0.07%  0.13%  0.11%   0 PPPoE Background

212     2845100  35973988         79  0.31%  0.29%  0.28%   0 PPPoE Discovery 

213       91560  46276259          1  0.07%  0.04%  0.05%   0 PPP manager     

214     3801560  70682638         53  0.63%  0.61%  0.58%   0 PPP Events      

215         116   1190837          0  0.00%  0.00%  0.00%   0 Multilink PPP   

216           0         2          0  0.00%  0.00%  0.00%   0 Multilink event 

218       75752   1395317         54  0.00%  0.00%  0.00%   0 IP SNMP         

219       21072    698269         30  0.00%  0.00%  0.00%   0 PDU DISPATCHER  

220       86136    697192        123  0.00%  0.00%  0.00%   0 SNMP ENGINE     

221           0         2          0  0.00%  0.00%  0.00%   0 IP SNMPV6       

222           0         1          0  0.00%  0.00%  0.00%   0 SNMP ConfCopyPro

223           0         1          0  0.00%  0.00%  0.00%   0 SNMP Traps      

224     1520096  73207009         20  0.31%  0.29%  0.26%   0 RADIUS          

225           8     19906          0  0.00%  0.00%  0.00%   0 Time Range Proce

226      695960    234558       2967  0.00%  0.11%  0.11%   0 VTEMPLATE Backgr

227           4      9927          0  0.00%  0.00%  0.00%   0 DHCPD Timer     

228           4     19853          0  0.00%  0.00%  0.00%   0 DHCPD Database  

229        2120    545479          3  0.00%  0.00%  0.00%   0 EIGRP-IPv4: Rout

230      844824  22549554         37  0.15%  0.07%  0.07%   0 EIGRP-IPv4: PDM 

231     3497712 279146321         12 39.35% 39.22% 39.04%   0 EIGRP-IPv4: HELL

-----------------------

please note that EIGRP process have high CPU

cpu1.jpg

also note that i have 2 7200 router have the same confiugration but runing 12.4(4)XD10 and same number of session the CPU working normal 6% and EIGRP normail

cpu1.jpg

Thank you

Qaes

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 0 (0 ratings)
Shelley Bhalla Tue, 06/21/2011 - 07:09

It is hard to identify root cause by the explanation given. It could be a memory leak, route flapping or process switching issue. I believe you need to open a TAC case to troubleshoot this further.

Shelley.

faraidalu Mon, 08/15/2011 - 00:33

Hi There,

Just want to share a tip on ow to filter your CPU utilisation on your Cisco router. I find alias commands handy and use them all the time. From the config mode, you can type as below.

R(config)#alias exec cpu sho process cpu sorted | exc 0.00%__0.00%__0.00%

This will create an alias called cpu, from priviledge mode, you can then type cpu and it will filter all process' that are not consuming the CPU, this is done with the help of exc [exclude]. This excludes all process' with 0% utilisation.

Below is my example on one of my routers.

Thanks,

Faraimd

Actions

Login or Register to take actions

This Discussion

Posted June 16, 2011 at 12:24 AM
Stats:
Replies:3 Avg. Rating:
Views:3031 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard