High CPU Utilization in L3 4507 Switch?

Unanswered Question
May 25th, 2010

Dear All,

             We are having Cisco L3 4507 Switch. Suddenly the CPU utilization reached more than 90%..Then i gave " no ip icmp redirects"..

After that CPU Utilization was somewhat reduced in to 60%.. Now again it is coming into 90 %..I dont know what to do?

When i am seeing the log"

It will be

May 26 10:57:10.387 IST: %C4K_EBM-4-HOSTFLAPPING: Host 00:18:FE:2D:A0:F6 in vlan 45 is flapping between port Po3 and port Po4
May 26 10:57:48.075 IST: %C4K_EBM-4-HOSTFLAPPING: Host 00:18:FE:2D:A0:F6 in vlan 45 is flapping between port Po3 and port Po4

Sh ver

cisco WS-C4507R (XPC8245) processor (revision 7) with 524288K bytes of memory.

L3Switch#sh memory summary
                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor    189A668   511072660    98829564   412243096   408491036   409139632

what i have to do in this case to reduce the CPU Utilization.?//

Thanks in advance

Senthi

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Ganesh Hariharan Tue, 05/25/2010 - 23:13

Dear All,

            We are having Cisco L3 4507 Switch. Suddenly the CPU utilization reached more than 90%..Then i gave " no ip icmp redirects"..

After that CPU Utilization was somewhat reduced in to 60%.. Now again it is coming into 90 %..I dont know what to do?

When i am seeing the log"

It will be

May 26 10:57:10.387 IST: %C4K_EBM-4-HOSTFLAPPING: Host 00:18:FE:2D:A0:F6 in vlan 45 is flapping between port Po3 and port Po4
May 26 10:57:48.075 IST: %C4K_EBM-4-HOSTFLAPPING: Host 00:18:FE:2D:A0:F6 in vlan 45 is flapping between port Po3 and port Po4

Sh ver

cisco WS-C4507R (XPC8245) processor (revision 7) with 524288K bytes of memory.

L3Switch#sh memory summary
                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor    189A668   511072660    98829564   412243096   408491036   409139632

what i have to do in this case to reduce the CPU Utilization.?//

Thanks in advance

Senthi

Senthil,

Is your host are getting flapped frequently in you switch logs,if yes then CPU will increase as it will result in STP convergence at frequent interval.For immediate measur just shutdown one of the port at switch end for that particular host and then check the CPU stats.

Also paste the show cpu process output

Hope to Help !!

Ganesh.H

Senthil Rajendra Wed, 05/26/2010 - 00:02

Dear Ganesh,

Actullay that switch in Production..So now i cant to shut any port channel.Right now i will post the CPU utilization..

L3Switch#sh proc cpu
CPU utilization for five seconds: 87%/6%; one minute: 88%; five minutes: 89%
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
   1           0        34          0  0.00%  0.00%  0.00%   0 Chunk Manager
   2       14656   2313150          6  0.00%  0.00%  0.00%   0 Load Meter
   3     8304164 133262730         62  0.00%  0.05%  0.05%   0 OSPF Hello
   4           0         1          0  0.00%  0.00%  0.00%   0 Deferred Events
   5    13003436   1631292       7971  0.00%  0.10%  0.11%   0 Check heaps
   6        1428      1857        768  0.00%  0.00%  0.00%   0 Pool Manager
   7           0         2          0  0.00%  0.00%  0.00%   0 Timers
   8           0         2          0  0.00%  0.00%  0.00%   0 Serial Backgroun
   9           0         1          0  0.00%  0.00%  0.00%   0 IPC Zone Manager
  10       79612  11539757          6  0.00%  0.00%  0.00%   0 IPC Periodic Tim
  11           0         1          0  0.00%  0.00%  0.00%   0 IPC BackPressure
  12       53572  11539757          4  0.08%  0.00%  0.00%   0 IPC Deferred Por
  13       87124   1399203         62  0.00%  0.00%  0.00%   0 IPC Seat Manager
  14           0         1          0  0.00%  0.00%  0.00%   0 IFS Agent Manage
  15   141721872 433776182        326  0.65%  0.71%  0.62%   0 ARP Input
  16       98196   2759811         35  0.00%  0.00%  0.00%   0 HC Counter Timer
  17         144         7      20571  0.00%  0.00%  0.00%   0 Entity MIB API
  18           0         1          0  0.00%  0.00%  0.00%   0 SERIAL A'detect
  19      108764  11539756          9  0.00%  0.00%  0.00%   0 Dynamic ARP Insp
  20           0         1          0  0.00%  0.00%  0.00%   0 Critical Bkgnd
  21      604248  32804671         18  0.00%  0.00%  0.00%   0 Net Background
  22      211576   1805450        117  0.00%  0.00%  0.00%   0 Logger
  23     3120880  11539729        270  0.08%  0.04%  0.01%   0 TTY Background
  24      115960  11539756         10  0.00%  0.00%  0.00%   0 Per-Second Jobs
  25  1505497232 721558337       2086  2.53%  3.32%  3.55%   0 Cat4k Mgmt HiPri
  26  2871370348 278244043      10319 71.48% 71.90% 72.85%   0 Cat4k Mgmt LoPri
  27       83372   8897588          9  0.00%  0.00%  0.00%   0 Galios Reschedul
  28           0         1          0  0.00%  0.00%  0.00%   0 IOS ACL Helper
  29      107428   2570351         41  0.00%  0.00%  0.00%   0 rf task
  30           4        27        148  0.00%  0.00%  0.00%   0 GalRedMsgProc
  31      860000  26900706         31  0.00%  0.00%  0.00%   0 Net Input
  32     2575220   4723227        545  0.00%  0.00%  0.00%   0 Compute load avg
  33     2956904    192736      15341  0.00%  0.02%  0.00%   0 Per-minute Jobs
  34       53848  11556198          4  0.00%  0.00%  0.00%   0 GalRedTimerProc
  35   259921672 347052896        748  1.72%  2.10%  2.26%   0 Spanning Tree
  36     1422188  11308230        125  0.08%  0.01%  0.00%   0 DTP Protocol
  37       10152   1156443          8  0.00%  0.00%  0.00%   0 Ethchnl
  38       32740    517486         63  0.00%  0.00%  0.00%   0 VLAN Manager
  39      414732  11570451         35  0.00%  0.00%  0.00%   0 UDLD
  40        1616    110971         14  0.00%  0.00%  0.00%   0 DHCP Snooping
  41         692    192792          3  0.00%  0.00%  0.00%   0 Port-Security
PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  42   131124936 437244250        299  1.80%  1.75%  1.72%   0 IP Input
  43     2108204  14406471        146  0.08%  0.01%  0.00%   0 CDP Protocol
  44     2246688    773472       2904  0.00%  0.01%  0.00%   0 IP Background
  45     2776820  10113337        274  0.00%  0.00%  0.00%   0 IP RIB Update
  46     3803200   7048666        539  0.00%  0.01%  0.00%   0 Adj Manager
  47     7202256  18310021        393  0.08%  0.14%  0.14%   0 CEF process
  48        2972     32178         92  0.00%  0.00%  0.00%   0 L2MM
  49           0        17          0  0.00%  0.00%  0.00%   0 MRD
  50       64872    525226        123  0.00%  0.00%  0.00%   0 IGMPSN
  51           0         5          0  0.00%  0.00%  0.00%   0 Socket Timers
  52           0         2          0  0.00%  0.00%  0.00%   0 L2TRACE SERVER
  53       19076    676653         28  0.00%  0.00%  0.00%   0 TCP Timer
  54        4180      2205       1895  0.00%  0.00%  0.00%   0 TCP Protocols
  55           0         1          0  0.00%  0.00%  0.00%   0 Probe Input
  56           0        12          0  0.00%  0.00%  0.00%   0 RARP Input
  57           0         1          0  0.00%  0.00%  0.00%   0 HTTP Timer
  58     1506236   8224363        183  0.00%  0.00%  0.00%   0 DHCPD Receive
  59        1976    192751         10  0.00%  0.00%  0.00%   0 IP Cache Ager
  60         976      3412        286  0.00%  0.00%  0.00%   1 Virtual Exec
  61           0        78          0  0.00%  0.00%  0.00%   0 AggMgr Process
  62     4254856 253113380         16  0.00%  0.02%  0.01%   0 PM Callback
  63         364      1458        249  0.00%  0.00%  0.00%   2 Virtual Exec
  64           0         1          0  0.00%  0.00%  0.00%   0 Syslog Traps
  65           0         2          0  0.00%  0.00%  0.00%   0 VTPMIB EDIT BUFF
  66          12        60        200  0.00%  0.00%  0.00%   0 SPAN switch
  67           0         6          0  0.00%  0.00%  0.00%   0 SNMP Timers
  68     1893324   4303730        439  0.08%  0.03%  0.00%   0 IP SNMP
  69      318280   2057706        154  0.00%  0.00%  0.00%   0 PDU DISPATCHER
  70     1804016   2340723        770  0.00%  0.00%  0.00%   0 SNMP ENGINE
  71           0         1          0  0.00%  0.00%  0.00%   0 SNMP ConfCopyPro
  72       17112     18587        920  0.00%  0.00%  0.00%   0 SNMP Traps
  73      493324    817593        603  0.00%  0.00%  0.00%   0 CEF Scanner
  74      387068   3949567         98  0.00%  0.00%  0.00%   0 Standby (HSRP)
  75        6916     23113        299  0.00%  0.00%  0.00%   0 SpanTree Helper
  76      859032  14115251         60  0.00%  0.00%  0.00%   0 NTP
  77         768     96415          7  0.00%  0.00%  0.00%   0 DHCPD Timer
  78       38296   5776824          6  0.00%  0.00%  0.00%   0 System polling
  79       29584   3276344          9  0.00%  0.00%  0.00%   0 DHCPD Database
  80   283269748 190471250       1487  0.16%  1.75%  2.05%   0 OSPF Router
  81          20        30        666  0.08%  0.02%  0.00%   3 Virtual Exec
  82      161248   1991241         80  0.00%  0.00%  0.00%   0 PAgP
  83         540      1390        388  0.00%  0.00%  0.00%   4 Virtual Exec
  84        2808      1750       1604  1.47%  0.71%  0.18%   5 Virtual Exec

Tks&Rgds

Senthil

Ganesh Hariharan Wed, 05/26/2010 - 00:20

Hi Senthil,

As per the output Cat4k Mgmt LoPri ,Cat4k Mgmt HiPri and STP is taking

high CPU utilization in your switch:-

In order to see which of the platform-specific processes use the CPU under

the context of Cat4k Mgmt HiPri and Cat4k Mgmt LoPri , issue the show

platform health command.

Each of the platform-specific processes has a target/expected utilization

of the CPU. When that process is within the target, the CPU executes the

process in the high-priority context. The show processes cpu command

output counts that utilization under Cat4k Mgmt HiPri . If a process

exceeds the target/expected utilization, that process runs under the

low-priority context. The show processes cpu command output counts that

additional utilization under Cat4k Mgmt LoPri . This Cat4k Mgmt LoPri is

also used to run background and other low-priority processes, such as

consistency check and reading interface counters. This mechanism allows

the CPU to run high-priority processes when necessary, and the idle CPU

cycles that remain are used for the low-priority processes. To exceed the

target CPU utilization by a small amount, or a momentary spike in

utilization, is not an indication of a problem that needs investigation.

Check out the below link on troubleshootinh high CPU in 4500 switches

http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note...

Note :- Do not shut any port channel in production network,just check the

mac-address which is getting flapped between the portchannel.

Hope to Help !!

Ganesh.H

Actions

This Discussion

Related Content