Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

High CPU usage on SMC during bulkstat gathering

This article explains potential cause of the SMC periodically showing high CPU usage during bulkstats collection

We often get calls for a condition of high cpu usage by the SMC/SPC. Quite often, the reason for the spike is due to bulkstats gathering. When bulkstats are gathered, a massive number of counters need to be queried and (s)ftp'd offline. The interval with which bulkstats are gathered is determined by the bulkstats mode variable "sample-interval", with the value in minutes. An example follows from a portion of the output of the "show configuration" command:

bulkstats collection
  bulkstats historical collection
  bulkstats mode
    sample-interval 2
    transfer-interval 15

If you are experiencing this bulkstat cpu behavior, please realize that this is an as designed behavior, and should not affect your subscriber traffic. In the example that follows, note that slot 8/0's cpu usage goes up just into the sample-interval timeframe, and goes down sometime thereafter. To determine whether your customers issue matches this behavior, please do the following:

- ssh to the mobile gateway in question.
- show configuration | grep sample
- timestamps
- show cpu table | grep 8/0

You will see that your cpu usage on slot 8/0 goes up for a small period of time every sample-interval.
In this annotated example, pertinent timestamps are noted, but clearly show the relationship of cpu usage to bulkstat sample-interval time.

Example:

[local]PDSN# show configuration | grep sample
    sample-interval 2
[local]PDSN# timestamps
[local]PDSN# sho cpu table | grep 8/0
             --------Load--------  ------CPU-Usage-----  ---------Memory--------
cpu  state     now   5min  15min     now   5min  15min    now  5min 15min total
----  -----  ------ ------ ------  ------ ------ ------  ----- ----- ----- -----
Friday December 04 20:52:02 UTC 2009
8/0  Actve    0.58   0.56   0.50   10.5%  19.5%  19.5%   413M  415M  411M 1024M
Friday December 04 20:52:10 UTC 2009
8/0  Actve    0.57   0.56   0.50   32.8%  19.6%  19.6%   418M  415M  411M 1024M
Friday December 04 20:52:15 UTC 2009
8/0  Actve    0.64   0.57   0.50   60.4%  21.1%  20.1%   417M  415M  411M 1024M
Friday December 04 20:52:27 UTC 2009
8/0  Actve    0.54   0.55   0.50   24.6%  21.4%  20.3%   416M  415M  411M 1024M
Friday December 04 20:52:37 UTC 2009
8/0  Actve    0.53   0.55   0.49   11.4%  21.4%  20.4%   416M  415M  411M 1024M
Friday December 04 20:52:49 UTC 2009
8/0  Actve    0.52   0.55   0.49   12.0%  21.6%  20.4%   414M  415M  411M 1024M
Friday December 04 20:53:02 UTC 2009
8/0  Actve    0.44   0.53   0.49   11.2%  21.7%  20.4%   414M  415M  411M 1024M
Friday December 04 20:53:06 UTC 2009
8/0  Actve    0.53   0.55   0.49   30.8%  21.8%  20.5%   414M  415M  411M 1024M
Friday December 04 20:53:12 UTC 2009
8/0  Actve    0.53   0.55   0.49   30.8%  21.8%  20.5%   414M  415M  411M 1024M
Friday December 04 20:53:16 UTC 2009
8/0  Actve    0.45   0.53   0.49   17.3%  19.7%  20.0%   414M  415M  411M 1024M
Friday December 04 20:53:21 UTC 2009
8/0  Actve    0.45   0.53   0.49   17.3%  19.7%  20.0%   414M  415M  411M 1024M
Friday December 04 20:53:25 UTC 2009
8/0  Actve    0.38   0.51   0.48   13.9%  19.4%  20.0%   413M  415M  411M 1024M
Friday December 04 20:53:31 UTC 2009
8/0  Actve    0.38   0.51   0.48   13.9%  19.4%  20.0%   413M  415M  411M 1024M
Friday December 04 20:53:40 UTC 2009
8/0  Actve    0.32   0.49   0.48   14.6%  19.5%  20.1%   413M  415M  411M 1024M
Friday December 04 20:53:44 UTC 2009
8/0  Actve    0.27   0.47   0.47   12.1%  19.7%  20.1%   413M  415M  412M 1024M
Friday December 04 20:53:51 UTC 2009
8/0  Actve    0.27   0.47   0.47   12.1%  19.7%  20.1%   413M  415M  412M 1024M
Friday December 04 20:53:57 UTC 2009
8/0  Actve    0.23   0.46   0.46   13.7%  19.8%  20.2%   413M  415M  412M 1024M
Friday December 04 20:54:01 UTC 2009
8/0  Actve    0.23   0.46   0.46   13.7%  19.8%  20.2%   413M  415M  412M 1024M
Friday December 04 20:54:05 UTC 2009
8/0  Actve    0.50   0.51   0.48   29.2%  20.0%  20.2%   415M  415M  412M 1024M
Friday December 04 20:54:08 UTC 2009
8/0  Actve    0.50   0.51   0.48   29.2%  20.0%  20.2%   415M  415M  412M 1024M
Friday December 04 20:54:10 UTC 2009
8/0  Actve    0.50   0.51   0.48   29.2%  20.0%  20.2%   415M  415M  412M 1024M
Friday December 04 20:54:13 UTC 2009
8/0  Actve    0.50   0.51   0.48   29.2%  20.0%  20.2%   415M  415M  412M 1024M
Friday December 04 20:54:16 UTC 2009
8/0  Actve    0.58   0.52   0.48   67.1%  21.6%  20.8%   421M  415M  412M 1024M
Friday December 04 20:54:19 UTC 2009
8/0  Actve    0.58   0.52   0.48   67.1%  21.6%  20.8%   421M  415M  412M 1024M
Friday December 04 20:54:22 UTC 2009
8/0  Actve    0.58   0.52   0.48   67.1%  21.6%  20.8%   421M  415M  412M 1024M
Friday December 04 20:54:26 UTC 2009
8/0  Actve    0.65   0.54   0.49   41.8%  22.3%  21.1%   423M  415M  412M 1024M
Friday December 04 20:54:30 UTC 2009
8/0  Actve    0.65   0.54   0.49   41.8%  22.3%  21.1%   423M  415M  412M 1024M
Friday December 04 20:54:34 UTC 2009
8/0  Actve    0.63   0.54   0.49   23.3%  22.6%  21.1%   415M  415M  412M 1024M
8/0  Actve    0.63   0.54   0.49   23.3%  22.6%  21.1%   415M  415M  412M 1024M
Friday December 04 20:54:57 UTC 2009
8/0  Actve    0.82   0.60   0.51   14.3%  23.3%  21.3%   414M  415M  412M 1024M
Friday December 04 20:55:03 UTC 2009
8/0  Actve    0.82   0.60   0.51   14.3%  23.3%  21.3%   414M  415M  412M 1024M
[local]PDSN#

Imported from Starent Networks Knowledgebase Article # 10880

Version history
Revision #:
1 of 1
Last update:
‎01-24-2012 02:54 PM
Updated by:
 
Everyone's tags (1)