High CPU on CSS 15003

Unanswered Question
Jul 29th, 2010

Hi, i have an issue with a CSS 15003 with a high load of CPU, it continously has some 60-70% of cpu, and it wont go down no matter the time of the day. The output of the spy shows no process taking the show but the one named "apidle" with 35%, all the other processes take about 2-4% of the Total, also if I issue a continous CPU hog the only process that always appears is the "apidle"

Is there any known issue about this process?, any input would be greatly appreciated

Thx

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 4.7 (3 ratings)
yushimaz Mon, 08/02/2010 - 00:17

High percentage of apIdle is not a problem since apIdle likes 'System Idle Process' of windows.

When work load is low, the value becomes high as below.

It seems that other process increases CPU utilization of 'show system resources'.

Please get spyReport for several times and check delta % again.

---

CSS11503-A(debug)# show system-resources cpu_summary

Chassis CPU Utilizations

Name                 Slot Sub 5Sec 1Min 5Min

--------------------------------------------

CSS5-SCM-2GE B0         1   1   0%   0%   0%

CSS5-IOM-8FE B0         2   1   0%   0%   0%

CSS5-SSL-C-K9 B0        3   1   0%   0%   0%

CSS11503-A(debug)# shell 1 1 spyReport | grep -v "0%"

NAME          ENTRY         TID   PRI   total % (ticks)  delta % (ticks)

--------     --------      -----  ---   ---------------  ---------------

apIdle                  8c184700  255    96% ( 1233127)   98% (     184)

TOTAL                                    98% ( 1272686)   98% (     187)

Return Value: 1,0x00000001

---

Regards,

Yuji

lozadaalberto Mon, 08/02/2010 - 07:01

I appreciate a lot your answer, I issued the spy report several times, however there is no indication of other system eating CPU resources, strangely, the total that is displayed in spy report is almost the percentage that the apidle has:

apIdle                  8c184850  255    36% (    5422)   37% (     172)

TOTAL                                    46% (   14955)   44% (     456)

That output is the same in several spy reports that i have issued, The show system resources show this

CSS5-SCM-2GE F0         1   1  64%  60%  60%

CSS5-IOM-16FE F0        2   1   0%   0%   0%

That is why it is kind of confusing that the total is displaying only  ~45, this pair of Css have ASR and redundant VIP redudancy, and both are showing the very same outputs. again any answer about this issue would be really helpful. 

yushimaz Mon, 08/02/2010 - 19:36

I understood your main question is why the TOTAL is around 45%.

Do you configure many keepalive?

I searched similar issue and some of them were fixed by reducing KAL config.

So, I created about 200 services which remain down state and then compared spyReport.

The result was as follows:

# before KAL config

CSS11503-A(debug)# shell 1 1 spyReport | grep -v "0%"

NAME          ENTRY         TID   PRI   total % (ticks)  delta % (ticks)

--------     --------      -----  ---   ---------------  ---------------

apIdle                  8c184700  255    96% ( 1233127)   98% (     184)

TOTAL                                    98% ( 1272686)   98% (     187)

# after adding KAL config

CSS11503-A(debug)# shell 1 1 spyReport | grep -v "0%"

NAME          ENTRY         TID   PRI   total % (ticks)  delta % (ticks)

--------     --------      -----  ---   ---------------  ---------------

apIdle                  8c184700  255    96% ( 7658722)   94% (     221)

TOTAL                                    98% ( 7938931)   94% (     233)

I'm not sure your issue is related to KAL config since I don't know your config and version.
If you use ASR, arrowpoint cookie and running 8.10(205) or before, you may hit CSCek57080.
For further investigation, could you please give me 'script play showtech' if possible?
Regards,
Yuji
lozadaalberto Wed, 08/04/2010 - 07:56

Thanks a lot yushimaz

It is a shame but I cannot give out the sh tech of the CSS due to some confidenciality rules, i would really like to but it just wont happen, im running 8.20.3.03 I run arrowpoint cookie and ASR, but the bug is supposedly fixed. About the Keep alives, i have 50 keepalive type script configured on different services, I dont know if those are a lot or not that many.

again any input on this would be greatly appreciated

yushimaz Wed, 08/04/2010 - 20:04

Hi Alberto

Thank you for your additional info regarding keepalive config

and sorry for my insensitive request regarding show tech.

Probably, many scripted keepalive config cause high cpu.

When I configured 50 services with scripted keepalive, cpu

utilization went 78%. (Since I configure keepalive frequesncy 2,

my CSS may waste cpu utilization than yours.)

Full configuration of my lab is attached. I checked this behavior with 8.20(402).

CSS11503-A(debug)# show system-resources cpu_summary

Chassis CPU Utilizations

Name                 Slot Sub 5Sec 1Min 5Min

--------------------------------------------

CSS5-SCM-2GE B0         1   1  78%  78%  69%

CSS5-IOM-8FE B0         2   1   0%   0%   0%

CSS5-SSL-C-K9 B0        3   1   0%   0%   0%

CSS11503-A(debug)# shell 1 1 spyReport | grep -v "0%"

NAME          ENTRY         TID   PRI   total % (ticks)  delta % (ticks)

--------     --------      -----  ---   ---------------  ---------------

apIdle                  8c184700  255    94% (24133648)   28% (      14)

KERNEL                                    1% (  358764)    2% (       1)

There is Guideline for scripted keepalive on the following URL.

You may be able to reduce cpu utilization to tune keepalive frequency.

http://www.cisco.com/en/US/products/hw/contnetw/ps792/prod_bulletin09186a008017bb55.html

Regarding high cpu bug, I could only find CSCtb45641 after 8.20(303).
However, it's very rare case. So, probably your problem doesn't hit this bug.
I guess your issue is related to scripted keepalive config (number, frequency
and interval) as with my test result.

Actions

Login or Register to take actions

This Discussion

Posted July 29, 2010 at 2:58 PM
Stats:
Replies:5 Avg. Rating:4.66667
Views:1449 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard

Rank Username Points
1 1,551
2 369
3 333
4 228
5 212
Rank Username Points
5