Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

EEM Applet to Monitor CPU

I need to what the PID using CPU utilization at 50%.

All i see on the syslog server is " %HA_EM-2-LOG: highcpu: HIGH CPU"

EEM doesnt tell me what process is consuming the CPU at 50%

Any ideas?

event manager applet highcpu
event snmp oid 1.3.6.1.4.1.9.9.109.1.1.1.1.10.1 get-type exact entry-op ge entry-val 50 poll-interval 5
action 1.0 cli command "enable"
action 2.0 cli command "show proc cpu sorted"
action 3.0 syslog priority critical msg "HIGH CPU"

Francisco.

2 ACCEPTED SOLUTIONS

Accepted Solutions
Cisco Employee

Re: EEM Applet to Monitor CPU

Right now it does.  That is, the script will not execute unless the OVERALL CPU usage is at or above 50%.  It will then email all processes which have a five second CPU utilization above 0%.  This way, you can get an idea of ALL CPU consumers that contributed to the 50% overall CPU utilization.  I thought this is what you would want.  However, if you want to only see processes at or above 50% on their own, I can do that, but I do not think the script will provide useful data at that point.

--

Please support CSC Helps Haiti

https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

Cisco Employee

Re: EEM Applet to Monitor CPU

Your version of IOS may require another parameter.  Try this version.

28 REPLIES
Cisco Employee

Re: EEM Applet to Monitor CPU

You're running the command "show proc cpu sorted", but that output is not going anywhere.  You might consider sending the output out via email:

action 4.0 mail server 10.1.1.1 to user@company.com from router@company.com subject "Show Process Output" body $_cli_result

Re: EEM Applet to Monitor CPU

Joe,

The switches dont have access to the mail server. Without outputing the "show proc cpu sorted" via email, can i not syslog it?

Cisco Employee

Re: EEM Applet to Monitor CPU

Sure, you can, but that would a HUGE syslog message.  Some syslog servers have a 1024 byte maximum message size.  You could try something like "show proc cpu sorted | exc 0.00" to weed out some of the noise.

Re: EEM Applet to Monitor CPU

Joe,

I only need to run eem script for a short period time.

Should i expect to see the process PID consuming 50% of CPU on the syslog server with my eem script config?

All i see on the syslog server is the message " %HA_EM-2-LOG: highcpu: HIGH CPU". or there is a way to check event manager output on the switch to capture the process PID.

Thanks Joe

Re: EEM Applet to Monitor CPU

any one please.

Is the EEM the tool for what i am trying to do?

Any suggestion please.

Cisco Employee

Re: EEM Applet to Monitor CPU

Change action 3.0 to:

action 3.0 syslog priority critical msg "HIGH CPU: $_cli_result"

Re: EEM Applet to Monitor CPU

Joe,

That is very aggressive. it output to syslog all active processes even with 0.00% CPU utilization. All i want to capture is a process using 50% CPU.

Thanks

Francisco

Cisco Employee

Re: EEM Applet to Monitor CPU

That you can't do without Tcl or programmatic applets.  What version of IOS do you have?

Re: EEM Applet to Monitor CPU

12.2.18-SXF14 on C6500 and 12.2(24)T2.

Not possible to accomplish what i am trying to do with EEM?

Don't know much about tcl! Can you assist us?

Francisco

Cisco Employee

Re: EEM Applet to Monitor CPU

One facet of EEM is Tcl.  You are currently using an applet policy, but you could easily do what you want in Tcl.  The 12.4(24)T router would allow you to do what you want with an applet, but that wouldn't work on the 6500.  One Tcl policy could be made to do what you want on both.  I will post suhc a policy in a while.

Re: EEM Applet to Monitor CPU

Thank you Joe.

Cisco Employee

Re: EEM Applet to Monitor CPU

Here is a policy which will replace your existing applet policy.  Before you can register it, you must create a directory on flash to hold EEM Tcl policies.  For example, flash:/policies.  Then, copy the sn_alert_high_cpu.tcl script into this directory.  Then, you need to set three EEM environment variables in the running config.  For example, to maintain your current level of polling and threshold:

event manager environment high_cpu_cpu_id 1

event manager environment high_cpu_threshold 50

event manager environment high_cpu_poll_freq 5

Then configure:

event manager policy sn_alert_high_cpu.tcl


When the CPU crosses 50%, you should see a syslog which contains the top non-zero CPU consumers.

Re: EEM Applet to Monitor CPU

Joe,

Thanks for the script.

I think we are almost there.

Regarding the "event manager environment high_cpu_threshold 50", does the value 50 mean 50.00% or 0.50% of CPU?.

I am testing your tcl script in our lab before i use it in production. Since i am not generating lot of traffic on the switch i have changed the threshold to 8 instead. With a value of 8 i am seeing processes that's using 0.08% and above appearing on the syslog server (see below). if i change it to 50 i am see processes with 0.50% and above. That's good but i am interested in anything at 50.00% or above not 0.50%. Does that make sense?

Can we change the tcl file "High CPU threshold" to a number say between 0.00 and 100.00 to reflect 0.00% to 100.00%  ?

*Mar  1 01:14:02.379: %HA_EM-2-LOG: system:/lib/tcl/eem_scripts_registered/sn_alert_high_cpu.tcl: HIGH CPU: top processes: Compute load avg (0.25%) SSS Feature Time (0.08%) Virtual Exec (0.17%) Exec (0.08%) IGMP Input (0.34%) PIM Process (0.08%) EEM TCL Proc (25.74%) EEM ED SNMP (0.51%) Syslog (0.34%) RBSCP Background (0.08%) HC Counter Timer (0.08%) bsm_xmt_proc (0.17%) AAA SEND STOP EV (0.08%) EEM Server (0.76%) Mwheel Process (0.17%)

Cisco Employee

Re: EEM Applet to Monitor CPU

I configured the script to react when the overall CPU is high, then print out those top processes with non-zero CPU utilizations.  You may not be seeing one process at 50% when the overall CPU utilization is at 50.  I can certainly change the script, but I think the way it is working now is probably what you want.  Checking that an individual process' CPU utilization is at or above 50 may not yield anything.

Re: EEM Applet to Monitor CPU

Joe,

On one of your C6500 WAN switches with EBGP peering with our ISP provider appears to have BGP hold timer expiring frequently during the day (really annoying) . During those events we are seeing very high CPU. We are not sure at this stage what's causing high CPU since we are logging lots a ACL's filling logging buffer quickly hence that's why we am asking for your assitant to capture what's causing high CPU.

The tcl script you have provided us we will definately use in prod but any chance you could modify the script to capture overall CPU utilization only at 50%?

Francisco.

Cisco Employee

Re: EEM Applet to Monitor CPU

Right now it does.  That is, the script will not execute unless the OVERALL CPU usage is at or above 50%.  It will then email all processes which have a five second CPU utilization above 0%.  This way, you can get an idea of ALL CPU consumers that contributed to the 50% overall CPU utilization.  I thought this is what you would want.  However, if you want to only see processes at or above 50% on their own, I can do that, but I do not think the script will provide useful data at that point.

--

Please support CSC Helps Haiti

https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

Re: EEM Applet to Monitor CPU

Joe.

Thank you very much for your assistant.

I will test it on the weekend

Francisco

Cisco Employee

Re: EEM Applet to Monitor CPU

I made one modification which may help.  In this version, the processes will be listed in descending order relative to their CPU consumption.  Previously, the processes were listed by hash value.

--

Please support CSC Helps Haiti

https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

Cisco Employee

Re: EEM Applet to Monitor CPU

Sorry, I had a typo in that last version.  Try this one instead.

--

Please support CSC Helps Haiti

https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

Re: EEM Applet to Monitor CPU

Hey Joe,

I dont see much difference based on the output i am seeing on the syslog server!

The high cpu threshold "50" definately means 0.50% and higher not 50.00% and higher. The problem is we are receiving far too many syslogs mesaages for procceses using low CPU and we could easily missed out what we need to see!. Hence why i need the tcl script to trigger when CPU is at 50.00% or above. Not interested on any process using below 50.00%.

Any chance you could modify the script to do that? if not then no worries.

Thanks Joe.

Francisco.

Cisco Employee

Re: EEM Applet to Monitor CPU

The threshold is 50%.  That is, the script will not run at all unless the five second CPU utilization of the device as a whole is greater than or equal to 50%.  When that occurs, the script will parse the output of "show proc cpu sorted".  For every process which has a non-zero five second CPU utilization value, that process name will be sent out via a syslog with its five second CPU utilization value.  Again, the reason for this is that multuple processes could be contributing to the overall CPU utilization of 50%.  There may not be one single process taking 50% or more CPU.  In which case, if you only printed processes that had a 50% CPU utilization value, your syslog would have no processes.

Now, if you are only interested in processes which have a 50% utilization value, you need to change the design of your script from one that looks at the overall system CPU usage to one that runs periodically, parses the output of "show proc cpu sorted", and only sends a syslog when one or more processes are taking up at least 50% of the CPU.  IS this what you want?

Re: EEM Applet to Monitor CPU

Yes Joe. Only interested in processes which have a 50% utilization value

Francisco.

Cisco Employee

Re: EEM Applet to Monitor CPU

Try this policy instead.  You will no longer need to set the high_cpu_cpu_id variable.  It will only generate a syslog if there is at least one process with a CPU utilization value at or above the high_cpu_threshold value.

Re: EEM Applet to Monitor CPU

Joe,

I have uploaded the script in to flash and trying to register it, i get error below.

R1(config)#event manager policy tm_alert_high_cpu.tcl
Compile check and registration failed:Wrong # args, usage is "::cisco::eem::event_register_timer watchdog|countdown|absolute|cron name ? cron_entry ? time ? queue_priority normal|low|high maxrun ? nice ?"
    while executing
"::cisco::eem::event_register_timer watchdog time $high_cpu_poll_freq
"
Tcl policy execute failed: Wrong # args, usage is "::cisco::eem::event_register_timer watchdog|countdown|absolute|cron name ? cron_entry ? time ? queue_priority normal|low|high maxrun ? nice ?"

Embedded Event Manager configuration: failed to retrieve intermediate registration result for policy tm_alert_high_cpu.tcl: Unknown error 0
R1(config)#event manager policy tm_alert_high_cpu.tcl
Compile check and registration failed:Wrong # args, usage is "::cisco::eem::event_register_timer watchdog|countdown|absolute|cron name ? cron_entry ? time ? queue_priority normal|low|high maxrun ? nice ?"
    while executing
"::cisco::eem::event_register_timer watchdog time $high_cpu_poll_freq
"
Tcl policy execute failed: Wrong # args, usage is "::cisco::eem::event_register_timer watchdog|countdown|absolute|cron name ? cron_entry ? time ? queue_priority normal|low|high maxrun ? nice ?"

Embedded Event Manager configuration: failed to retrieve intermediate registration result for policy tm_alert_high_cpu.tcl: Unknown error 0
R1(config)#event manager policy tm_alert_high_cpu.tcl

Cisco Employee

Re: EEM Applet to Monitor CPU

Your version of IOS may require another parameter.  Try this version.

Re: EEM Applet to Monitor CPU

PERFECTO.  Excellent stuff Joe.

Another rating goes to you.

Thanks

Francisco.

New Member

Re: EEM Applet to Monitor CPU

Joe,

I'm in a similar position where it appears the "BGP Router" process may be running so frequently due our EBGP peer, whom we receieve the full routing table from.  They constantly run some algorithm that changes their routes for optimal routing which seems to be causing them to send us updates ranging from 15-30 on average with spikes as high as 100+.  On our other bgp router, it peers with two ISPs which it gets full routing tables from both.  It has twice as many routes as the one in question but rarely has the same problem. Its bgp updates are only around 1-5 on average with maybe a spike up to 20-25. Nothing like the one in question.  I was hoping to use EEM to get a better picture of what process is taking up the majority of the CPU when the cpu spikes to over 75% over the 5sec duration.  I want to confirm that its the BGP Router process that is causing our problem.  Whats making me look at bgp is that we use nagios as our NMS.  We defined a service so that every 5 mins it runs a perl script to get information on all our devices interfaces.  Periodically it will alert us that this router is not responding but 5 minutes later its ok. Now I know that I can adjust one setting in nagios that might stop it from alerting what I feel is a false alarm but I want to confirm if its the CPU spikes by the BGP Router process that stops the router from responding to nagios.

s72033-advipservicesk9_wan-mz.122-33.SXI1.bin  <---- IOS currently running on our 7200 router.

Any help would be appreciated.

Lee

Cisco Employee

Re: EEM Applet to Monitor CPU

Please start a new thread for your issue.

3157
Views
0
Helpful
28
Replies
CreatePlease to create content