Hello forum,
We have routers that use EEM scripts tied to tracked objects for the purposes of internet circuit failover. These routers have a primary cable connection and a cellular backup. The EEM scripts are supposed to monitor connectivity through the primary connection and if it fails for whatever reason then remove some static routes and fail over to the cellular card. When the primary connection comes back online the EEM scripts are supposed to shutdown the cellular card and reinstate the routes.
Lately we've noticed higher than normal usage on the cellular cards and when I logged into some of the routers the routes pointing to the primary connection were not in the config. This tells me that the EEM script for bringing the primary connection back isn't firing at all. I am not sure how to tell if the script ran at all or has some other issue that would prevent it from installing the routes.
Here are the EEM scripts and tracked objects:
event manager applet PRIMARY_UP
event track 6 state up maxrun 240
action 1000 if $FLAPPING eq "1" goto 1019"
action 1001 cli command "enable"
action 1002 cli command "configure terminal"
action 1003 cli command "ip route vrf CORP 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 253"
action 1004 cli command "ip route vrf OFFICE 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1005 cli command "ip route vrf STORE 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1006 cli command "ip route vrf GUEST 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1007 cli command "ip route vrf POS 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1008 cli command "ip route vrf VENDOR 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1009 cli command "interface Cellular0/0/0"
action 1010 cli command "shutdown"
action 1011 wait 5
action 1012 cli command "no shutdown"
action 1013 cli command "end"
action 1014 cli command "clear policy-firewall session"
action 1016 cli command "clear ip nat translation forced"
action 1017 syslog msg "NAT Translations Flushed"
action 1018 syslog priority critical msg "PRIMARY_CIRCUIT_UP"
action 1019 exit
event manager applet PRIMARY_DOWN
event track 6 state down maxrun 240
action 1000 if $FLAPPING eq "1" goto 1024"
action 1001 cli command "enable"
action 1002 cli command "configure terminal"
action 1003 cli command "interface gigabitEthernet 0/0"
action 1004 cli command "shutdown"
action 1005 cli command "exit"
action 1006 cli command "no ip route vrf CORP 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp 253"
action 1007 cli command "no ip route vrf OFFICE 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1008 cli command "no ip route vrf STORE 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1009 cli command "no ip route vrf GUEST 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1010 cli command "no ip route vrf POS 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1011 cli command "no ip route vrf VENDOR 0.0.0.0 0.0.0.0 GigabitEthernet0/0 dhcp"
action 1012 cli command "interface gigabitEthernet 0/0"
action 1013 cli command "no shutdown"
action 1014 cli command "end"
action 1015 cli command "clear policy-firewall session"
action 1017 policy FLAPPING_ACTIVATE
action 1018 cli command "clear ip nat translation forced"
action 1019 syslog msg "NAT Translations Flushed"
action 1020 cli command "clear ip bgp all 65000"
action 1022 syslog priority critical msg "PRIMARY_CIRCUIT_DOWN"
action 1023 policy FLAPPING_ACTIVATE
action 1024 exit
track 1 ip sla 1
delay down 60 up 60
!
track 2 interface Tunnel301 line-protocol
delay down 60 up 60
!
track 3 interface Tunnel302 line-protocol
delay down 60 up 60
!
track 4 list threshold percentage
object 2
object 3
!
track 5 ip sla 2
delay down 60 up 60
!
track 6 list threshold percentage
object 1
object 5
!
ip sla 1
dns google.com name-server 8.8.8.8
frequency 30
ip sla schedule 1 life forever start-time now
ip sla 2
dns microsoft.com name-server 208.67.222.222
frequency 30
ip sla schedule 2 life forever start-time now
I am not an EEM guru but the lines that have me concerned are the two lines with $FLAPPING that have an extra set of double-quotes at the end. I don't know if that is tripping up the interpreter or not.
Also, I don't know how to make sure these scripts actually fire or not. I suppose I could try to down one of the primary ports and see what syslog says, but I thought I would check the forums for any obvious mistakes first.
Please let me know if there is anything else I can provide to help solve this issue.
