Problem with Embedded Event Manager and Object Tracking

Answered Question
Dec 7th, 2009

Hi,

I have a 2801 running c2801-advipservicesk9-mz.124-24.T2.bin. It has the following configuration:

track 300 list boolean or
object 10
object 11
object 12
object 13

event manager applet clear_ipsec_tunnel
event track 300 state down
action 1.0 cli command "enable"
action 2.0 cli command "clear crypto session"
action 3.0 syslog msg "IPSec tunnel has been cleared by clear_ipsec_tunnel applet"

My problem is that after the tracked object number 300 transitions from an up state to a down state, nothing happens. It seems like the applet doesn't work with object tracking. Here's what I see in logs:

Dec  7 21:52:32.236 MCK: %TRACKING-5-STATE: 12 ip sla 12 reachability Up->Down
Dec  7 21:52:37.236 MCK: %TRACKING-5-STATE: 13 ip sla 13 reachability Up->Down
Dec  7 21:52:57.236 MCK: %TRACKING-5-STATE: 10 ip sla 10 reachability Up->Down
Dec  7 21:53:07.236 MCK: %TRACKING-5-STATE: 11 ip sla 11 reachability Up->Down
Dec  7 21:53:07.996 MCK: %TRACKING-5-STATE: 300 list boolean or Up->Down

That's it. For some reason, the applet won't execute the CLI commands when the EEM applet is triggered. Am I doing something wrong or I have encountered some bug? Thanks.

I have this problem too.
0 votes
Correct Answer by Joe Clarke about 7 years 2 days ago

Disabling AAA on one line can be done with this:

aaa authentication login noaaa none

aaa authorization exec noaaa none

aaa authorization command 0 noaaa none

aaa authorization commands 1 noaaa none

aaa authorization commands 15 noaaa none

!

line vty 0
login authentication noaaa
authorization exec noaaa
authorization commands 0 noaaa
authorization commands 1 noaaa
authorization commands 15 noaaa
transport input none
Correct Answer by Joe Clarke about 7 years 3 days ago

Change your event registration line to:

event track 300 state down maxrun 30
Correct Answer by Joe Clarke about 7 years 3 days ago

It looks like you might have command authorization enabled.  Try configuring the following:

event manager session cli username USER

Where USER is a username capable of executing the required CLI commands.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (5 ratings)
Loading.
Joe Clarke Mon, 12/07/2009 - 11:54

The applet as it's written looks okay.  I did a quick track list test, and it works.  Enable the following debug, reproduce the problem, then post the output.

debug event manager detector track

debug event manager action cli

saiiven07 Mon, 12/07/2009 - 12:09

Hi jclarke,

Thank you for your attention to my problem. Here's the output:

Dec  7 23:05:55.739 MCK: %LINK-5-CHANGED: Interface Loopback0, changed state to administratively down
Dec  7 23:05:56.739 MCK: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to down
Dec  7 23:06:32.283 MCK: %TRACKING-5-STATE: 12 ip sla 12 reachability Up->Down
Dec  7 23:06:37.283 MCK: %TRACKING-5-STATE: 13 ip sla 13 reachability Up->Down
Dec  7 23:06:57.283 MCK: %TRACKING-5-STATE: 10 ip sla 10 reachability Up->Down
Dec  7 23:07:07.283 MCK: %TRACKING-5-STATE: 11 ip sla 11 reachability Up->Down
Dec  7 23:07:08.083 MCK: %TRACKING-5-STATE: 300 list boolean or Up->Down
Dec  7 23:07:08.083 MCK: fh_track_object_changed: Track notification 300 state down
Dec  7 23:07:08.083 MCK: fh_fd_track_event_match: track ED pubinfo enqueue rc = 0
Dec  7 23:07:08.095 MCK: fh_send_track_fd_msg: msg_type=64
Dec  7 23:07:08.095 MCK: fh_send_track_fd_msg: sval=0
Dec  7 23:07:08.099 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : CTL : cli_open called.
Dec  7 23:07:08.107 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT : N001>
Dec  7 23:07:08.107 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : IN  : N001>enable
Dec  7 23:07:28.119 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : CTL : cli_close called.
Dec  7 23:08:27.283 MCK: %TRACKING-5-STATE: 1 ip sla 1 reachability Up->Down
Dec  7 23:09:52.283 MCK: %TRACKING-5-STATE: 1 ip sla 1 reachability Down->Up

Correct Answer
Joe Clarke Mon, 12/07/2009 - 12:40

It looks like you might have command authorization enabled.  Try configuring the following:

event manager session cli username USER

Where USER is a username capable of executing the required CLI commands.

Joe Clarke Mon, 12/07/2009 - 12:45

Actually, the problem may not be with command authorization, but rather with your enable prompt.  When you become enabled on this device, what does your prompt look like?

saiiven07 Mon, 12/07/2009 - 13:24

Yes, authorization is enabled on the device and the configuration looks like this:


aaa new-model
!
!
aaa authentication login default group tacacs+ local
aaa authentication ppp default group tacacs+ local
aaa authorization exec default group tacacs+ local
aaa authorization commands 0 default group tacacs+ local if-authenticated
aaa authorization commands 1 default group tacacs+ local if-authenticated
aaa authorization commands 2 default group tacacs+ local if-authenticated
aaa authorization commands 3 default group tacacs+ local if-authenticated
aaa authorization commands 4 default group tacacs+ local if-authenticated
aaa authorization commands 5 default group tacacs+ local if-authenticated
aaa authorization commands 6 default group tacacs+ local if-authenticated
aaa authorization commands 7 default group tacacs+ local if-authenticated
aaa authorization commands 8 default group tacacs+ local if-authenticated
aaa authorization commands 9 default group tacacs+ local if-authenticated
aaa authorization commands 10 default group tacacs+ local if-authenticated
aaa authorization commands 11 default group tacacs+ local if-authenticated
aaa authorization commands 12 default group tacacs+ local if-authenticated
aaa authorization commands 13 default group tacacs+ local if-authenticated
aaa authorization commands 14 default group tacacs+ local if-authenticated
aaa authorization commands 15 default group tacacs+ local if-authenticated
aaa authorization network default group tacacs+ local
aaa accounting exec default
action-type start-stop
group tacacs+
!
aaa accounting commands 0 default
action-type stop-only
group tacacs+
!
aaa accounting commands 1 default
action-type stop-only
group tacacs+
!
...


but at the moment the tacacs server is not reachable (timeout), so it all comes to logging into the device by using the local database, which contains one username. Something like this:

username cisco privilege 15 secret cisco

As for my prompt, it looks like this "Router#" after I login as cisco:cisco (local database).

I've added the "event manager session cli username cisco" command, but it didn't help. The debug output was the same.

Joe Clarke Mon, 12/07/2009 - 13:28

How long does it take after typing enable, and entering the enable password until you get back a prompt?

saiiven07 Mon, 12/07/2009 - 13:45

Oddly enough, I see the output of the commands I enter without any delay although I have the "tacacs-server timeout 20" command configured. After I ssh to the router by using cisco:cisco, I get the "Router#" prompt. I tried to issue the "disable" command, then "enable", typed in the password and got the prompt instantly. Something's wrong here.

edit: in fact, the tacacs server is reachable, but the key on the router doesn't match the one in the database of the ACS server.

Correct Answer
Joe Clarke Mon, 12/07/2009 - 13:59

Change your event registration line to:

event track 300 state down maxrun 30
saiiven07 Mon, 12/07/2009 - 14:15

Changed it, but still no luck...jclarke, if you need me to give you some more info, I'll provide you with it tommorow. Thanks for your help.

Joe Clarke Mon, 12/07/2009 - 16:02

At this point, I would need to see the full show run from this device, and the new debug output with the maxrun timer increased.

saiiven07 Tue, 12/08/2009 - 09:54

jclarke,

Today I added the router into the tacacs server database and the applet started working just fine by using my login name. So the working configuration looks like this:

event manager session cli username "my login name"
event manager applet clear_ipsec_tunnel
event track 300 state down maxrun 30
action 1.0 cli command "enable"
action 2.0 cli command "clear crypto session"
action 3.0 syslog msg "IPSec tunnel has been cleared by clear_ipsec_tunnel applet"

Then I tried to use a login name from the local database that has "privelege 15" access and of course the debug output showed me this:

Dec  8 18:12:58.203 MCK: %TRACKING-5-STATE: 300 list boolean or Up->Down
Dec  8 18:12:58.203 MCK: fh_track_object_changed: Track notification 300 state down
Dec  8 18:12:58.203 MCK: fh_fd_track_event_match: track ED pubinfo enqueue rc = 0
Dec  8 18:12:58.215 MCK: fh_send_track_fd_msg: msg_type=64
Dec  8 18:12:58.215 MCK: fh_send_track_fd_msg: sval=0
Dec  8 18:12:58.219 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : CTL : cli_open called.
Dec  8 18:12:58.227 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT : Router>
Dec  8 18:12:58.227 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : IN  : Router>enable
Dec  8 18:12:58.747 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT : Command authorization failed.
Dec  8 18:12:58.747 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT :
Dec  8 18:12:58.747 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT : Router>
Dec  8 18:12:58.747 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : IN  : Router>clear crypto session
Dec  8 18:12:58.771 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT :                                  ^
Dec  8 18:12:58.771 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT : % Invalid input detected at '^' marker.
Dec  8 18:12:58.771 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT :
Dec  8 18:12:58.771 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : OUT : Router>
Dec  8 18:12:58.775 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel: IPSec tunnel has been cleared by clear_ipsec_tunnel  applet
Dec  8 18:12:58.775 MCK: %HA_EM-6-LOG: clear_ipsec_tunnel : DEBUG(cli_lib) : : CTL : cli_close called.

So I guess this problem arises when you have command authorization enabled and the tacacs server is not reachable or something like that. I have tried to find a way to use the local database instead of using the aaa server but didn't succeed. Although I have found an interesting workaround. Here it is:

Link: http://blog.ioshints.info/2007/05/command-authorization-fails-with-eem.html

Workaround found after reading the "Executing IOS commands from Tcl shell" from the "Tclsh on Cisco IOS tutorial".

On the above article it is mentionned that the ios_config command is executed inside the context of another VTY line (also found with the AAA debug). The workaround is to define the FIRST VTY line with "transport input none" to prevent ssh or telnet to grab it and to configure the aaa authorization without any command authorization for this line.

Kind regards
Christian Chautems

Looks great, but I am not quite sure how to "configure the aaa authorization without any command authorization for this line".

Anyway, jclarke thank you so much for taking your time to look into my problem and for your help.

Correct Answer
Joe Clarke Tue, 12/08/2009 - 10:09

Disabling AAA on one line can be done with this:

aaa authentication login noaaa none

aaa authorization exec noaaa none

aaa authorization command 0 noaaa none

aaa authorization commands 1 noaaa none

aaa authorization commands 15 noaaa none

!

line vty 0
login authentication noaaa
authorization exec noaaa
authorization commands 0 noaaa
authorization commands 1 noaaa
authorization commands 15 noaaa
transport input none

Actions

This Discussion