High CPU Utilization in Catalyst 6509 VSS

[jackson.ku]

Hi,

The CPU utilization of my Catalyst 6509 VSS switch ( up to 20% or higher ) was always higher than other Catalyst 6509 non VSS switch. The show proc cpu sort are A/F. The IOS version is 12.2(33)SXI4a. Is there any suggestion for the VSS high CPU utilization?

Best Regards,

6509-VSS#sh proc cpu sort

CPU utilization for five seconds: 20%/10%; one minute: 17%; five minutes: 18%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

293   180677272 576311141        313  2.79%  2.58%  2.63%   0 Earl NDE Task   

  23   1238395721092969687        113  2.55%  1.22%  1.20%   0 IPC Seat Manager

537   137203248 710311815        193  0.95%  1.44%  1.46%   0 Port manager per

308     8875732  38978030        227  0.55%  0.48%  0.47%   0 CDP Protocol    

  11    22352240  40970620        545  0.47%  0.20%  0.20%   0 ARP Input       

314    38472656 287310225        133  0.39%  0.43%  0.40%   0 IP Input        

504     4765340  73348487         64  0.15%  0.10%  0.10%   0 RPC online_diag_

133          36       207        173  0.15%  0.03%  0.00%   1 Virtual Exec    

403     4744372  13696576        346  0.15%  0.16%  0.15%   0 CEF: IPv4 proces

604      621880 551345900          1  0.15%  0.08%  0.08%   0 HSRP Common     

341      5993161121847645          0  0.15%  0.13%  0.15%   0 Ethernet Msec Ti

316    11340988  98860214        114  0.15%  0.07%  0.08%   0 ADJ resolve proc

  50     1338496   9198605        145  0.15%  0.11%  0.13%   0 Per-Second Jobs 

383      432600    228133       1896  0.07%  0.00%  0.00%   0 XDR mcast       

  59    10383804 361448183         28  0.07%  0.10%  0.10%   0 VSIBC process   

120     3982420   4153256        958  0.07%  0.06%  0.07%   0 IPC LC Message H

536      151488 124223423          1  0.07%  0.02%  0.02%   0 PM Callback     

424     1738732   4471794        388  0.07%  0.03%  0.04%   0 HIDDEN VLAN Proc

416       84492  10509171          8  0.07%  0.00%  0.00%   0 FM core         

606     1899852  37997617         49  0.07%  0.05%  0.05%   0 OSPF-1 Hello    

135       85228  89567322          0  0.07%  0.01%  0.00%   0 Mwheel Process  

322      416756   9138950         45  0.07%  0.00%  0.00%   0 Spanning Tree   

445      193604 277199712          0  0.07%  0.03%  0.02%   0 RADIUS          

282      199000 277469935          0  0.07%  0.03%  0.02%   0 ACE Tunnel Task 

  24           0         1          0  0.00%  0.00%  0.00%   0 IPC Session Serv

   6           0         1          0  0.00%  0.00%  0.00%   0 PF Redun ICC Req

  22           0         1          0  0.00%  0.00%  0.00%   0 IPC Process leve

  21       12848   8937247          1  0.00%  0.00%  0.00%   0 IPC Deferred Por

  25           0         1          0  0.00%  0.00%  0.00%   0 IPC Stdby Update

  26         756    894349          0  0.00%  0.00%  0.00%   0 Compute SRP rate

  31           0         1          0  0.00%  0.00%  0.00%   0 Crash writer    

  27           0         2          0  0.00%  0.00%  0.00%   0 DDR Timers      

  33          20        60        333  0.00%  0.00%  0.00%   0 OBFL Cfg Dispatc

  28           0         2          0  0.00%  0.00%  0.00%   0 Dialer event    

  20      552748   8938985         61  0.00%  0.06%  0.07%   0 IPC Periodic Tim

  36          12        41        292  0.00%  0.00%  0.00%   0 rf proxy message

  37          12         6       2000  0.00%  0.00%  0.00%   0 client_entity_se

  19           0       166          0  0.00%  0.00%  0.00%   0 PF_Split Sync Pr

  29         140        37       3783  0.00%  0.00%  0.00%   0 ifIndex Receive 

  30           0         2          0  0.00%  0.00%  0.00%   0 Serial Backgroun

  32     2330020  11145258        209  0.00%  0.03%  0.00%   0 EnvMon          

  42           0        64          0  0.00%  0.00%  0.00%   0 Cat6k SNMP Trap 

  43           0         1          0  0.00%  0.00%  0.00%   0 ARP Snoop       

  44       18108   8937239          2  0.00%  0.00%  0.00%   0 Dynamic ARP Ins

[wang tao · Accepted solution]

yes,I have meet this case in my network...

So,see your output,the first process is "NDE",I think it will be:

1.Do you enable the NETFLOW?

2.Do you enable the Netflow export,the Netflow Export just is NDE

The Netflow will collect the traffic info,like source-ip destination-ip ports and bytes by each flow.

And the Netflow Export will put these traffic info to a server, the server will log these info and  analysis them.

Each Netflow Export packet encapsulated by CPU,so if you enable Netflow and enable NF-Export,your CPU will more higher.

[jackson.ku]

Hi,

Yes, I have enable netflow & NDE export. Is it possible to modify netflow & NDE export configuration ( but still enable it ) to decrease the CPU utilization?

Best Regards,

[wang tao]

Maybe like:

1.decrease the interface enabled netflow

2.I don't care the source/destination port,only lookup the IP

http://www.cisco.com/en/US/docs/ios/netflow/configuration/guide/12_2sr/nf_12_2sr_book.html

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6555/ps6601/prod_white_paper0900aecd80406232.html

[Colby Beam]

Just curious,  are you considering 20% high cpu? This would be pretty expected, and actually quite low vs some production networks I've seen.

6509-VSS#sh proc cpu sort

CPU utilization for five seconds: 20%/10%; one minute: 17%; five minutes: 18% <<<<

[jackson.ku]

Hi,

The cpu utilization for other non VSS Catalyst 6509 switch ( also enable netflow & NDE export ) is below 10%, but this VSS switch is 20% and higher, and we will install more distirbute and access switch under the VSS core switch, so we are worry the CPU utillization and switch performance...

Best Regards,