Thanks for the response Joe.  I'm still getting up to speed on some of the newer EEM features, so I appologize for any ignorance.  If I am reading that correctly though, won't that put that applet into the config to restore the line, after the initial condition is met once?  What will happen when the event happens again a few hours/days later?

The sequence I am trying to automate is roughly this:

1. Detect condition (in this case a DDoS attack, via one of several possible triggers).

2. Remove a line from a prefix list, causing traffic attacking a public IP to no longer be sent to the router.

3. Wait for some period for the attack to subside (say 30 minutes)

4. Re-advertise the prefix which was being attacked.

5. Return to step 1

Will what you are suggesting return the router/config to the same state it was in before the initial trigger condition was detected?

Here is what I have got to work.  This uses polling of an SNMP OID for a rate limiter that I created to dump SYN traffic which exeeds our typical SYN traffic levels by a factor of 5.  (Might change the trigger later, but that isn't really important)  This seems to work, except that if I make the wait time more than 5 seconds it never gets past action 3.0.  I assume that is because the wait time is longer than the polling interval, and the whole thing "resets" on the polling interval.

event manager applet DDoS_RESPONSE01

event snmp oid 1.3.6.1.4.1.9.9.166.1.17.1.1.21.80.65538 get-type exact entry-op gt entry-val "0" entry-type increment exit-op eq exit-val "0" poll-interval 10

trigger

action 1.0 cli command "enable"

action 1.1 cli command "config term"

action 1.2 cli command "no ip prefix-list PUBLIC_NETWORKS seq 50 permit 216.1.1.0/24 le 32"

action 1.9 syslog msg "DDoS Attach Detected. Removing Web Srvr Subnet from PUBLIC_NETWORKS."

action 3.0 wait 5

action 4.0 cli command "enable"

action 4.1 cli command "config term"

action 4.2 cli command "ip prefix-list PUBLIC_NETWORKS seq 50 permit 216.1.1.0/24 le 32"

action 4.9 syslog msg "DDoS Attach Timeout reached. Re-adding Web Srvr Subnet to PUBLIC_NETWORKS."

!

What I'm suggesting will do what you want.  It will also handle the case where the DDoS attack happens again and again.  As a final measure, you can unconfigure the timer policy as the last thing it does.  For example:

action 010 cli command "action 6.0 $q no event manager applet restore-prefix$q"

Just insert your one CLI command into the proper place in my example list of actions.  Be sure to watch those $q markers.  You'll need to keep them.  What you'll find is that this solution works exactly like what you're doing, only it gives you a larger timer that runs asynchronously.

Ok I got it to work the way you suggested.  But I have a question.  After the initial event is triggered, the line in my prefix-list is removed, and the restore applet is created starting the restore timer with a value of X.  After the timer X expires the line is put back, but what would happen if the initial event is triggered again, before the timer X reaches zero?

I thought about using an environment variable for the timer to put the line back, then using that same environment variable with the "exit-time $tout" clause to make it harder for someone to do that if they start adjusting timer values later, but I can't get the exit-time clause to accept an environment variable as a parameter.

One more question, There seems to be a lot of variability in when my event fires.  The first time it fires it usually does so within about 10-15 seconds of the OID I'm monitoring reaching a non-zero value. After the first time it fires though, it can be anywhere from [exit-time + 30 seconds] to [exit-time + 250 seconds]

Thanks for all your help!  Sorry I missed your session at CiscoLive this year. It is always one of my favorites!

Here is my complete script:

ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.5.1..0/24 le 32
!
event manager environment q "
event manager environment ATimeout 30
!
no event manager applet RESTORE_PREFIX
no event manager applet DDOS_RESPONSE01
event manager applet DDOS_RESPONSE01
event snmp oid 1.3.6.1.4.1.9.9.166.1.17.1.1.21.80.65538 get-type exact entry-op gt entry-val "0" entry-type increment exit-time 60 poll-interval 10
trigger
action 001 cli command "enable"
action 002 cli command "config term"
action 003 cli command "no ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.5.1..0/24 le 32"
action 004 syslog msg "DDoS Attack Detected. Removing Web Srvr Subnet from PUBLIC_NETWORKS for $ATimeout seconds."

action 006 cli command "config term"
action 007 cli command "event manager applet RESTORE_PREFIX"
action 008 cli command "event timer countdown time $ATimeout "
action 009 cli command "action 001 cli command enable"
action 010 cli command "action 002 cli command $q config term $q"
action 011 cli command "action 003 cli command $q ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.5.1..0/24 le 32 $q"
action 014 cli command "action 004 syslog msg $q DDoS Attack Timeout ($ATimeout) reached. Re-adding Web Srvr Subnet to PUBLIC_NETWORKS. $q "
action 021 cli command "action 010 cli command enable"
action 022 cli command "action 011 cli command $q config term $q"
action 025 cli command "action 012 cli command $q no event manager applet RESTORE_PREFIX $q"
!

Change action 006 to:

action 006 cli command "no event manager applet RESTORE_PREFIX"

Remove actions 021 and 022.

By doing that, you will ensure that if something happens to trigger the first policy again before the restore policy has a chance to run, the timer will reset.

As for the trigger time, this will depend on the polling cycle and the exit-time.  You have set the entry-type to be increment, so the object will have to increment at least once in a polling cycle (compared to the last time the event fired).  And if it increments during the exit-time hold-down that won't count.

Thanks for all your help Joe. 

Ok, so here is my current script, which seems to be working pretty well (changing to entry-type "value" fixed the variability in detection times).  For testing in the script below, I'm using a 30 second timeout for when the line gets put back, and a 60 second timeout for when monitoring should resume after the event is triggered. The script checks the value of the OID every 5 seconds.

The only other thing I would like to do with it that I can't figure out, is how to use an environment variable for the exit-time.  Ideally, I would just add a value, like 10 seconds, to the ATimeout variable.  However I can't figure out the syntax to just use a var for the exit-time.  Anyone know the secret (or if it is possible?)

!

event manager environment ATimeout 30

event manager environment q "

!

no event manager applet DDOS_RESPONSE01

event manager applet DDOS_RESPONSE01

event snmp oid 1.3.6.1.4.1.9.9.166.1.17.1.1.21.80.65538 get-type exact entry-op gt entry-val "0" entry-type value exit-time 60 poll-interval 5

trigger

action 001 cli command "enable"

action 002 cli command "config term"

action 003 cli command "no ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.4.1.0/24 le 32"

action 004 syslog msg "DDoS Attack Detected. Removing Web Srvr Subnet from PUBLIC_NETWORKS for ($ATimeout) seconds."

action 005 cli command "event manager applet RESTORE_PREFIX"

action 006 cli command "event timer countdown time $ATimeout "

action 007 cli command "action 101 cli command $q enable $q"

action 008 cli command "action 102 cli command $q config term $q"

action 009 cli command "action 103 cli command $q no event manager applet RESTORE_PREFIX $q"

action 010 cli command "action 104 cli command $q ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.4.1.0/24 le 32$q"

action 011 cli command "action 105 syslog msg $q DDoS Attack Timeout ($ATimeout) reached. Re-adding Web Srvr Subnet to PUBLIC_NETWORKS. $q "

action 012 cli command "action 106 cli command $q no event manager applet RESTORE_PREFIX $q"

!

exit

! !
event manager environment ATimeout 30
event manager environment q "
!
event manager applet DDOS_RESPONSE01
event snmp oid 1.3.6.1.4.1.9.9.166.1.17.1.1.21.80.65538 get-type exact entry-op gt entry-val "0" entry-type value exit-time 60 poll-interval 5
trigger
action 001 cli command "enable"
action 002 cli command "config term"
action 003 cli command "no ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.4.1.0/24 le 32"
action 004 syslog msg "DDoS Attack Detected. Removing Web Srvr Subnet from PUBLIC_NETWORKS for ($ATimeout) seconds."
action 005 cli command "event manager applet RESTORE_PREFIX"
action 006 cli command "event timer countdown time $ATimeout "
action 007 cli command "action 101 cli command $q enable $q"
action 008 cli command "action 102 cli command $q config term $q"
action 009 cli command "action 103 cli command $q no event manager applet RESTORE_PREFIX $q"
action 010 cli command "action 104 cli command $q ip prefix-list PUBLIC_NETWORKS seq 140 permit 10.4.1.0/24 le 32$q"
action 011 cli command "action 105 syslog msg $q DDoS Attack Timeout ($ATimeout) reached. Re-adding Web Srvr Subnet to PUBLIC_NETWORKS. $q "
action 012 cli command "action 106 cli command $q no event manager applet RESTORE_PREFIX $q"
!
exit
!

Likely this is not possible.  Only certain fields were made variable in applets, and I'm betting something as infrequently used as exit-time was not one of them.  If this must be a variable, convert your applet to a Tcl policy and then you can use the variable exit-time.