High CPU Utilization Problems 4507R

Luis Fernando Casanova · ‎11-01-2011

I have a problem with high cpu 4507R

CPU utilization for five seconds: 64%/1%; one minute: 57%; five minutes: 58%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

30 69325632 20179910 3435 44.21% 36.48% 38.28% 0 Cat4k Mgmt LoPri

29 23013384 45951032 500 11.89% 11.45% 11.48% 0 Cat4k Mgmt HiPri

52 13686072 3659895 3739 4.30% 4.74% 4.74% 0 Spanning Tree

58 2851068 7184408 396 0.95% 0.94% 0.96% 0 IP Input

18 2659888 7813807 340 0.79% 0.90% 0.87% 0 ARP Input

113 581416 1764899 329 0.23% 0.32% 0.33% 0 HSRP (Standby)

125 2184 171915 12 0.15% 0.01% 0.00% 2 Virtual Exec

71 180692 505824 357 0.15% 0.18% 0.16% 0 CEF: IPv4 proces

88 194608 447497 434 0.15% 0.10% 0.08% 0 DHCPD Receive

99 39684 2698161 14 0.07% 0.03% 0.02% 0 PM Callback

117 797884 488158 1634 0.07% 0.14% 0.18% 0 SNMP ENGINE

38 29988 846973 35 0.07% 0.00% 0.00% 0 Net Input

25 30476 448961 67 0.07% 0.01% 0.00% 0 Logger

%CPU %CPU RunTimeMax Priority Average %CPU Total

Target Actual Target Actual Fg Bg 5Sec Min Hour CPU

Lj-poll 1.00 0.03 2 1 100 500 0 0 0 1:19

GalChassisVp-review 3.00 1.68 10 42 100 500 1 1 1 68:41

S2w-JobEventSchedule 10.00 3.01 10 9 100 500 2 3 2 129:34

Stub-JobEventSchedul 10.00 2.50 10 38 100 500 1 2 1 94:31

StatValueMan Update 1.00 0.08 1 1 100 500 0 0 0 5:17

Pim-review 0.10 0.01 1 0 100 500 0 0 0 0:23

Ebm-host-review 1.00 0.08 8 4 100 500 0 0 0 3:21

Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:01

Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:00

Acl-Flattener 1.00 0.00 10 5 100 500 0 0 0 0:00

KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:13

KxAclPathMan update 2.00 0.20 10 9 100 500 0 0 0 4:42

KxAclPathMan reprogr 1.00 0.00 2 1 100 500 0 0 0 0:47

TagMan-RecreateMtegR 1.00 0.00 10 5 100 500 0 0 0 0:00

K2CpuMan Review 30.00 5.51 30 32 100 500 5 6 4 204:48

K2AccelPacketMan: Tx 10.00 1.70 20 1 100 500 1 2 1 72:49

K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00

K2AclMan-taggedFlatA 1.00 0.00 10 4 100 500 0 0 0 0:00

K2AclCamMan stale en 1.00 0.00 10 5 100 500 0 0 0 0:00

K2AclCamMan hw stats 3.00 1.14 10 5 100 500 0 0 0 21:03

K2AclCamMan kx stats 1.00 0.00 10 5 100 500 0 0 0 8:47

K2AclCamMan Audit re 1.00 0.00 10 5 100 500 0 0 0 10:00

K2AclPolicerTableMan 1.00 0.06 10 2 100 500 0 0 0 0:47

K2L2 Address Table R 2.00 0.71 12 5 100 500 0 0 0 37:10

K2L2 New Static Addr 2.00 0.00 10 5 100 500 0 0 0 0:06

K2L2 New Multicast A 2.00 0.00 10 5 100 500 0 0 0 0:36

K2L2 Dynamic Address 2.00 0.00 10 5 100 500 0 0 0 0:25

K2L2 Vlan Table Revi 2.00 0.00 12 8 100 500 0 0 0 2:08

K2 L2 Destination Ca 2.00 0.00 10 0 100 500 0 0 0 0:00

K2PortMan Review 2.00 4.57 15 11 100 500 2 3 2 139:21

I hope to help me

dominic.caron · ‎11-01-2011

can you post the complete sh platform health?

Reza Sharifi · ‎11-01-2011

It looks as a big percentage of your CPU is taken by Cat4k Mgmt LoPri

have a look at this document:

http://www.cisco.com/en/US/products/hw/switches/ps663/products_tech_note09186a00804cef15.shtml

HTH

Luis Fernando Casanova · ‎11-01-2011

950-NTSW-2#sho platform health

%CPU %CPU RunTimeMax Priority Average %CPU Tota

Target Actual Target Actual Fg Bg 5Sec Min Hour CPU

Lj-poll 1.00 0.01 2 1 100 500 0 0 0 1:2

GalChassisVp-review 3.00 1.69 10 42 100 500 1 1 1 69:

S2w-JobEventSchedule 10.00 3.70 10 9 100 500 2 3 2 130

Stub-JobEventSchedul 10.00 2.35 10 38 100 500 1 2 1 95:

StatValueMan Update 1.00 0.12 1 1 100 500 0 0 0 5:2

Pim-review 0.10 0.00 1 0 100 500 0 0 0 0:2

Ebm-host-review 1.00 0.00 8 4 100 500 0 0 0 3:2

Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:0

Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:0

Acl-Flattener 1.00 0.00 10 5 100 500 0 0 0 0:0

KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:1

KxAclPathMan update 2.00 0.01 10 9 100 500 0 0 0 4:4

KxAclPathMan reprogr 1.00 0.01 2 1 100 500 0 0 0 0:4

TagMan-RecreateMtegR 1.00 0.00 10 5 100 500 0 0 0 0:0

K2CpuMan Review 30.00 5.62 30 32 100 500 4 5 4 206

K2AccelPacketMan: Tx 10.00 2.27 20 1 100 500 1 2 1 73:

K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:0

K2AclMan-taggedFlatA 1.00 0.00 10 4 100 500 0 0 0 0:0

K2AclCamMan stale en 1.00 0.00 10 5 100 500 0 0 0 0:0

K2AclCamMan hw stats 3.00 0.18 10 5 100 500 0 0 0 21:

K2AclCamMan kx stats 1.00 0.53 10 5 100 500 0 0 0 8:5

K2AclCamMan Audit re 1.00 0.00 10 5 100 500 0 0 0 10:

K2AclPolicerTableMan 1.00 0.00 10 2 100 500 0 0 0 0:4

K2L2 Address Table R 2.00 0.00 12 5 100 500 0 0 0 37:

K2L2 New Static Addr 2.00 0.00 10 5 100 500 0 0 0 0:0

K2L2 New Multicast A 2.00 0.00 10 5 100 500 0 0 0 0:3

K2L2 Dynamic Address 2.00 0.00 10 5 100 500 0 0 0 0:2

K2L2 Vlan Table Revi 2.00 0.00 12 8 100 500 0 0 0 2:0

K2 L2 Destination Ca 2.00 0.00 10 0 100 500 0 0 0 0:0

K2PortMan Review 2.00 2.21 15 11 100 500 3 3 2 140

Gigaport0 Review 0.40 0.08 4 23 100 500 0 0 0 3:5

Gigaport1 Review 0.40 0.06 4 23 100 500 0 0 0 3:5

Gigaport2 Review 0.40 0.07 4 3 100 500 0 0 0 3:5

Gigaport3 Review 0.40 0.09 4 3 100 500 0 0 0 4:1

Gigaport4 Review 0.40 0.09 4 2 100 500 0 0 0 3:5

Gigaport5 Review 0.40 0.06 4 3 100 500 0 0 0 3:5

Gigaport6 Review 0.40 0.09 4 2 100 500 0 0 0 4:0

Gigaport7 Review 0.40 0.08 4 3 100 500 0 0 0 4:2

Gigaport8 Review 0.40 0.07 4 3 100 500 0 0 0 3:4

Gigaport9 Review 0.40 0.01 4 4 100 500 0 0 0 0:3

Gigaport10 Review 0.40 0.07 4 3 100 500 0 0 0 3:5

Gigaport11 Review 0.40 0.10 4 3 100 500 0 0 0 3:5

Gigaport12 Review 0.40 0.01 4 4 100 500 0 0 0 0:3

Gigaport13 Review 0.40 0.09 4 3 100 500 0 0 0 3:5

Gigaport14 Review 0.40 0.01 4 4 100 500 0 0 0 0:3

Gigaport15 Review 0.40 0.08 4 3 100 500 0 0 0 3:4

Gigaport16 Review 0.40 0.05 4 3 100 500 0 0 0 3:4

Gigaport17 Review 0.40 0.10 4 22 100 500 0 0 0 3:5

Gigaport18 Review 0.40 0.06 4 2 100 500 0 0 0 3:5

Gigaport19 Review 0.40 0.07 4 22 100 500 0 0 0 3:5

Gigaport20 Review 0.40 0.00 4 2 100 500 0 0 0 0:3

Gigaport21 Review 0.40 0.00 4 2 100 500 0 0 0 0:3

Gigaport22 Review 0.40 0.00 4 2 100 500 0 0 0 0:3

Gigaport23 Review 0.40 0.00 4 2 100 500 0 0 0 0:3

Gigaport24 Review 0.40 0.08 4 2 100 500 0 0 0 4:0

Gigaport25 Review 0.40 0.08 4 2 100 500 0 0 0 4:0

Gigaport26 Review 0.40 0.01 4 2 100 500 0 0 0 0:3

Gigaport27 Review 0.40 0.08 4 2 100 500 0 0 0 4:0

Gigaport28 Review 0.40 0.09 4 2 100 500 0 0 0 4:2

Gigaport29 Review 0.40 0.00 4 2 100 500 0 0 0 0:3

Gigaport30 Review 0.40 0.07 4 2 100 500 0 0 0 4:2

Gigaport31 Review 0.40 0.00 4 2 100 500 0 0 0 0:3

Gigaport32 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport33 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport34 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport35 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport36 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport37 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport38 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport39 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport40 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport41 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport42 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport43 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport44 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport45 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport46 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

Gigaport47 Review 0.40 0.00 4 0 100 500 0 0 0 0:0

K2Fib cam usage revi 2.00 0.00 15 0 100 500 0 0 0 0:0

K2Fib IrmFib Review 2.00 0.00 15 0 100 500 0 0 0 0:0

K2Fib Vrf Default Ro 2.00 0.00 15 0 100 500 0 0 0 0:0

K2Fib AdjRepop Revie 2.00 0.00 15 0 100 500 0 0 0 0:2

K2Fib Vrf Unpunt Rev 2.00 0.00 15 0 100 500 0 0 0 0:2

K2Fib Consistency Ch 1.00 0.00 5 3 100 500 0 0 1 66:

K2FibAdjMan Stats Re 2.00 0.31 10 7 100 500 0 0 0 22:

K2FibAdjMan Host Mov 2.00 0.00 10 7 100 500 0 0 0 27:

K2FibAdjMan Adj Chan 2.00 0.00 10 0 100 500 0 0 0 0:0

K2FibMulticast Signa 2.00 0.00 10 2 100 500 0 0 0 1:5

K2FibMulticast Entry 2.00 0.00 10 6 100 500 0 0 0 0:1

K2FibMulticast Irm M 2.00 0.00 10 7 100 500 0 0 0 0:0

K2FibFastDropMan Rev 2.00 0.00 7 0 100 500 0 0 0 0:0

K2FibPbr route map r 2.00 0.04 20 5 100 500 0 0 0 12:

K2FibPbr flat acl pr 2.00 0.02 20 7 100 500 0 0 0 2:1

K2FibPbr consolidati 2.00 0.00 10 1 100 500 0 0 0 0:3

K2FibPerVlanPuntMan 2.00 0.00 15 2 100 500 0 0 0 0:0

K2FibFlowCache flow 2.00 0.03 10 4 100 500 0 0 0 1:5

K2FibFlowCache flow 2.00 21.02 10 8 100 500 36 38 30 799

K2FibFlowCache adj r 2.00 0.04 10 3 100 500 0 0 0 2:3

K2FibFlowCache flow 2.00 0.00 10 3 100 500 0 0 0 1:0

K2MetStatsMan Review 2.00 0.04 5 2 100 500 0 0 0 16:

K2FibMulticast MET S 2.00 0.00 10 0 100 500 0 0 0 0:0

K2QosDblMan Rate DBL 2.00 0.12 7 0 100 500 0 0 0 5:3

IrmFibThrottler Thro 2.00 0.01 7 1 100 500 0 0 0 1:0

K2 VlanStatsMan Revi 2.00 0.29 15 2 100 500 0 0 0 15:

K2 Packet Memory Dia 2.00 0.00 15 8 100 500 0 0 0 43:

K2 L2 Aging Table Re 2.00 0.21 20 3 100 500 0 0 0 6:3

K2PacketBufMonitor-P 1.00 2.63 10 1 100 500 2 2 2 109

RkiosPortMan Port Re 2.00 0.20 12 8 100 500 0 0 0 9:3

Rkios Module State R 4.00 0.01 40 2 100 500 0 0 0 1:3

Rkios Online Diag Re 4.00 0.01 40 0 100 500 0 0 0 1:1

RkiosIpPbr IrmPort R 2.00 0.01 10 5 100 500 0 0 0 4:2

RkiosAclMan Review 3.00 0.05 30 1 100 500 0 0 0 2:4

MatMan Review 0.50 0.00 4 0 100 500 0 0 0 0:0

Slot 3 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 3 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 4 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 4 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 5 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 5 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 6 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 6 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 7 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:0

Slot 7 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:0

EthHoleLinecardMan(1 1.66 0.03 10 5 100 500 0 0 0 1:2

EthHoleLinecardMan(3 1.66 0.30 10 6 100 500 0 0 0 11:

EthHoleLinecardMan(4 1.66 0.18 10 6 100 500 0 0 0 9:5

EthHoleLinecardMan(5 1.66 0.16 10 7 100 500 0 0 0 11:

EthHoleLinecardMan(6 1.66 2.25 10 3 100 500 1 2 1 84:

EthHoleLinecardMan(7 1.66 0.25 10 2 100 500 0 0 0 11:

Grandprix 3-1 Stub R 4.00 0.13 20 20 100 500 0 0 0 6:3

Grandprix 4-1 Stub R 4.00 0.42 20 20 100 500 0 0 0 22:

Grandprix 5-1 Stub R 4.00 0.13 20 20 100 500 0 0 0 6:3

Grandprix 3-1 Stats 5.00 0.09 50 1 100 500 0 0 0 4:5

Grandprix 4-1 Stats 5.00 0.11 50 4 100 500 0 0 0 4:5

Grandprix 7-1 Stub R 4.00 0.25 20 21 100 500 0 0 0 14:

Grandprix 5-1 Stats 5.00 0.11 50 1 100 500 0 0 0 4:5

Grandprix 7-1 Stats 5.00 0.11 50 4 100 500 0 0 0 9:0

-------------

%CPU Totals 261.20 50.35

Allocation ceiling Current allocation

------------------ ------------------

kbytes % in use kbytes % in use

Linecard 1's Store 258.00 4% 12.48 100%

Linecard 2's Store 258.00 0% 0.00 0%

Linecard 3's Store 258.00 11% 30.50 100%

Linecard 4's Store 258.00 11% 30.59 100%

Linecard 5's Store 258.00 11% 30.50 100%

Linecard 6's Store 258.00 53% 137.77 100%

Linecard 7's Store 258.00 11% 30.53 100%

TSM objects ------------------ ------------------

RkiosSysPacketBuf 250.00 0% 0.53 0%

PacketBufRaw 24788.25 100% 24788.25 100%

PacketBufRawJumbo 732.81 25% 183.20 100%

Packet 1147.03 2% 30.11 99%

PacketInfoItem 390.62 0% 0.50 0%

VbufNodes2400 80.75 0% 0.00 0%

VbufNodes1600 55.75 0% 3.48 0%

VbufNodes400 292.00 4% 12.54 95%

VbufNodes64 62.00 0% 3.39 0%

GalGbicEntrys 11.43 5% 0.60 100%

PimPhyports 875.00 24% 217.00 100%

PimPorts 796.87 31% 248.62 100%

PimModules 150.00 2% 3.51 100%

PimSlots 5.00 2% 0.13 100%

PimChassis 33.62 6% 2.10 100%

EbmVlans 2720.00 1% 39.84 100%

EbmVlanGroupEntrys 1920.00 0% 6.91 100%

EbmPorts 184.00 30% 55.88 100%

EbmPortHostEntrys 384.00 0% 0.00 0%

EbmIeNodes 536.00 0% 4.18 100%

EbmPortVlanAclFeatur 1024.00 0% 0.00 0%

EbmSortedHostTableIt 1.87 0% 0.05 0%

EbmSortedGroupTableI 1.75 0% 0.05 0%

IrmVrfs 6.85 1% 0.10 100%

IrmFibAdjs 768.00 4% 38.03 98%

IrmPortEtherAddrEntr 500.00 0% 0.59 100%

IrmFibEntries 10240.00 0% 72.46 97%

AclL4Op 384.00 0% 0.05 100%

AclL4OpTriplet 256.00 0% 0.02 100%

AclClassifier 768.00 0% 0.49 95%

AclFeature 2512.68 0% 1.33 94%

Acl 384.00 0% 0.12 100%

Ace 1280.00 0% 3.08 100%

AceActionDescStorage 256.00 0% 0.00 0%

AclListNode 256.00 0% 0.16 95%

AceListNode 25600.00 0% 4.55 90%

AclClassifierActionL 512.00 0% 0.29 94%

AclLayerFeatureListN 512.00 0% 0.14 77%

AclClassifierListNod 256.00 0% 0.00 0%

TableMapMan NameToTa 27.00 0% 0.00 0%

TableMapAllocator 59.00 0% 0.00 0%

FlatAcl 512.00 0% 0.25 75%

FlatAce 3840.00 0% 33.98 90%

FlatAceActionListNod 76800.00 0% 13.59 90%

FlatAclL4OpSetStorag 1024.00 0% 0.00 0%

FlatAclCacheNode 1024.00 0% 0.37 100%

QoS Policers 1672.00 0% 0.25 100%

KxAclPath 1024.00 0% 1.00 100%

KxAclPathListNode 256.00 0% 0.00 0%

ConfigToHwAfMap 418.78 0% 0.00 0%

HwToCfgAceMap 192.00 0% 0.00 0%

CommandTables 48.00 23% 11.15 100%

K2FibPbrFlatRouteMap 481.68 0% 1.09 100%

K2FibPbrExpandedFlat 2304.00 0% 17.68 99%

K2FibPbrFlatRouteMap 320.00 0% 0.78 100%

K2FibVrfs 38.59 1% 0.59 100%

K2 Netflow FPT Entri 11.97 0% 0.00 0%

K2TxPacket 384.00 0% 0.02 0%

K2TxPacketInfo 256.00 0% 0.42 0%

EbmVlanHostEntrys 2048.00 4% 92.81 96%

MatEntrys 7680.00 0% 17.81 100%

MatEntryTableIterato 1.12 0% 0.03 0%

RkiosAclMan NamedGal 43.75 0% 0.01 100%

Rkios Acl VlanMaps 272.00 0% 0.00 0%

Rkios Acl VlanMapEnt 1015.62 0% 0.00 0%

Rkios QoS PolicyMaps 1315.93 0% 0.03 100%

Rkios QoS ClassMaps 896.00 0% 0.46 100%

AclToIosFilterMapLis 384.00 0% 0.01 100%

Rkios QoS Policers 380.00 0% 0.03 100%

RkiosTableMap Galios 3.00 0% 0.00 0%

Event Nodes 128.00 0% 1.81 25%

Event Nodes 128.00 1% 19.50 7%

KxAclTagPairNode 2176.00 0% 0.13 100%

KxAclMappingTableEnt 96.02 0% 0.37 0%

KxAclMappingTableEnt 64.01 0% 0.00 0%

KxAclTaggedFlatAcl 1152.00 0% 0.07 100%

RkisoIpPbrRouteMaps 97.65 0% 0.01 100%

IrmFlows 256.00 38% 106.62 91%

------------------ ------------------

TSM totals 189764.49 13% 26043.42 99%