EEM nat overload weirdness

[Patrick McHenry]

I've configured the EEM to bring up a backup interface. In the EEM config, I have - action A37 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/2 overload". This makes g0/2 the source of the nat translations. When I fail back to the primary interface g0/0, I  have action A35 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload", but it doesn't seem to take.

And when I do a "show run | i nat", I get: " ip nat inside source list nat_acl interface GigabitEthernet0/2 overload'

But the weird thing is that it still works. I have a phone and a laptop connected to it and can make calls and the laptop can traverse the Internet. Does the router just use the interface that is available?     

Any ideas why this still works and any ideas how I can get EEM to change the nat list?

Thanks, Pat.

[John Blakley]

Patrick,

My first thought is that when you try to remove a nat statement and there are translations currently in use, it will prompt you to remove those translations (basically clear the nat table). If you answer anything other than "Y", it will kick you back to the prompt without clearing them. I've not tested this yet, but you may be able to answer the prompt in your eem script that you're clearing it in:

R2(config)#no ip nat insid source list 101 inter fa0/0 overload

Dynamic mapping in use, do you want to delete all entries? [no]:

So, in your eem script you would need to answer this to delete the line. Create a new action after your "no ip nat ..." line:

action a36 cli command "no ip nat inside source list nat_acl interface GigabitEthernet0/2 overload" pattern "delete all entries"

action a37 cli command "yes"

This would delete the line completely. You then could add the new line after this action....

HTH,

John

[Patrick McHenry]

I tried this manually but it still wouldn't allow me to delete the line.

But, why does this stll work? Does the router just find a way out? It kinda bothers me. Is there any other config you would like to see?

Thanks, Pat

[John Blakley]

Pat,

Sure...can you post your ip nat lines and your eem script? It shouldn't work if the backup interface is truly not being used. How do you have the backup interface configured? Is it configured on the primary or is it an interface that's always up, but you prefer it only when your primary goes down? What happened when you tried to delete the line manually?

HTH,

John

[Patrick McHenry]

Thanks John

sh run | i nat

ip nat outside

ip nat inside

ip nat outside

ip nat inside source list nat_acl interface GigabitEthernet0/2 overload

ip access-list extended nat_acl

action A35 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload"

action A37 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/2 overload"

track 1 ip sla 10 reachability

delay down 180 up 180

!

ip sla 10

icmp-echo 4.2.2.2 source-interface GigabitEthernet 0/0

frequency 30

exit

ip sla schedule 10 life forever start-time now

!

logging buffered 6

sh run | b event man

!

event manager applet FAIL_Back_TO_PRIMARY_INTERFACE

event syslog pattern "1 ip sla 10 reachability Down->Up"

action A11 cli command "enable"

action A12 cli command "configure terminal"

action A13 cli command "interface tunnel0"

action A14 cli command "no tunnel source GigabitEthernet0/2"

action A15 cli command "interface tunnel 1"

action A16 cli command "no tunnel source GigabitEthernet0/2"

action A17 cli command "interface tunnel0"

action A18 cli command "tunnel source GigabitEthernet0/0"

action A19 cli command "interface tunnel 1"

action A21 cli command "tunnel source GigabitEthernet0/0"

action A22 cli command "exit"

action A23 cli command "interface GigabitEthernet0/2"

action A24 cli command "shut"

action A25 cli command "exit"

action A26 cli command "no ip ftp source-interface GigabitEthernet0/2"

action A27 cli command "no ip tftp source-interface GigabitEthernet0/2"

action A28 cli command "no ip http client source-interface GigabitEthernet0/2"

action A29 cli command "no ip radius source-interface GigabitEthernet0/2"

action A31 cli command "no logging source-interface GigabitEthernet0/2"

action A32 cli command "ip ftp source-interface GigabitEthernet0/0"

action A33 cli command "ip tftp source-interface GigabitEthernet0/0"

action A34 cli command "ip http client source-interface GigabitEthernet0/0"

action A35 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload"

action A36 cli command "ip radius source-interface GigabitEthernet0/0"

action A37 cli command "logging source-interface GigabitEthernet0/0"

action A38 cli command "no ip route 0.0.0.0 0.0.0.0 dhcp"

action A39 cli command "ip route 0.0.0.0 0.0.0.0 dhcp"

action A41 cli command "exit"

action A42 cli command "end"

event manager applet FAIL__TO_BACKUP_INTERFACE

event syslog pattern "1 ip sla 10 reachability Up->Down"

action A11 cli command "enable"

action A12 cli command "configure terminal"

action A13 cli command "interface tunnel0"

action A14 cli command "no tunnel source GigabitEthernet0/0"

action A15 cli command "interface tunnel 1"

action A16 cli command "no tunnel source GigabitEthernet0/0"

action A17 cli command "interface tunnel0"

action A18 cli command "tunnel source GigabitEthernet0/2"

action A19 cli command "interface tunnel 1"

action A21 cli command "tunnel source GigabitEthernet0/2"

action A22 cli command "exit"

action A23 cli command "interface GigabitEthernet0/0"

action A24 cli command "shut"

action A25 cli command "exit"

action A26 cli command "no ip ftp source-interface GigabitEthernet0/0"

action A27 cli command "no ip tftp source-interface GigabitEthernet0/0"

action A28 cli command "no ip http client source-interface GigabitEthernet0/0"

action A29 cli command "no ip radius source-interface GigabitEthernet0/0"

action A31 cli command "no logging source-interface GigabitEthernet0/0"

action A32 cli command "no ip ftp source-interface GigabitEthernet0/2"

action A33 cli command "no ip tftp source-interface GigabitEthernet0/2"

action A34 cli command "no ip http client source-interface GigabitEthernet0/2"

action A35 cli command "ip radius source-interface GigabitEthernet0/2"

action A36 cli command "logging source-interface GigabitEthernet0/2"

action A37 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/2 overload"

action A38 cli command "interface GigabitEthernet0/2"

action A39 cli command "no shut"

action A41 cli command "interface GigabitEthernet0/0"

action A42 cli command "no shut"

action A43 cli command "end"

ip route 4.2.2.2 255.255.255.255 GigabitEthernet0/0 dhcp

I had to do the ip route 0.0.0.0 0.0.0.0 dhcp and ip route 0.0.0.0 0.0.0.0 dhcp. I had to do this to the failover back to the primary to make it work. Not exactly sure why.

I have the Event Manager FAIL__TO_BACKUP_INTERFACE shut go/2, and it is shut at the moment. I just noticed though that my laptop cannot ping public addresses though but, I can through the router.

I have the backup interface down when not used. It comes up and builds a tunnel when the primary pings fail.

Thanks again.

[John Blakley]

Pat,

When you are pinging from the router, it's locally generated and sources from the outside interface which would explain why you get a response while doing this from the router but not doing from a pc. You can also test this by pinging a public address from the router and then sourcing from the inside interface that's part of the subnet to be natted. It should fail if nat truly isn't working.

What happens when you try to remove your nat statement manually? Does it do anything at all? Aside from your script, what I would do is, unless I'm not seeing it, is to remove the nat statement during failover before applying the new nat statement along with the pattern command from my earlier post. We first need to figure out what the router does when trying to remove the nat statement manually.

HTH,

John

[Patrick McHenry]

mchenry#clear ip nat trans *

mchenry#config t

Enter configuration commands, one per line.  End with CNTL/Z.

mchenry(config)#$list nat_acl interface GigabitEthernet0/0 overload

%Dynamic mapping in use, cannot change

mchenry(config)#exit

mchenry#sh run | i nat

crypto isakmp nat keepalive 10

ip nat outside

ip nat inside

ip nat outside

ip nat inside source list nat_acl interface GigabitEthernet0/2 overload

ip access-list extended nat_acl

action A35 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload"

action A37 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/2 overload"

mchenry#sh ip nat trans

mchenry#

mchenry(config)#no ip nat source list nat_acl interface GigabitEthernet0/2 ove$

%Dynamic mapping not found

Anything else you would like to see?

Thank you, Pat.

[Patrick McHenry]

Just reloaded the router:

sh run | i nat

crypto isakmp nat keepalive 10

ip nat outside

ip nat inside

ip nat outside

ip nat inside source list nat_acl interface Gigab:itEthernet0/0 overload

Pinged google form attached lapptop

Pro Inside global      Inside local       Outside local      Outside global

icmp 192.168.168.46:7  172.17.0.20:7      173.194.43.33:7    173.194.43.33:7

[Patrick McHenry]

After I failed back to the Primary interface:

Aug 28 10:46:12.270 EST-DST: %TRACKING-5-STATE: 1 ip sla 10 reachability Down->Up

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : CTL : cli_open called.

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry>

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry>enable

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry#configure terminal

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line.  End with CNTL/Z.

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#interface tunnel0

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#no tunnel source GigabitEthernet0/2

Aug 28 10:46:12.310 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down

Aug 28 10:46:12.310 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 DOWN on interface Tunnel0 non DR

Aug 28 10:46:12.310 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from FULL to DOWN, Neighbor Down: Interface down or detached

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel 1

Aug 28 10:46:12.318 EST-DST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=192.168.25.185, prot=50, spi=0x89C92382(2311660418), srcaddr=65.199.155.102, input interface=GigabitEthernet0/2

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#no tunnel source GigabitEthernet0/2

Aug 28 10:46:12.334 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to down

Aug 28 10:46:12.334 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 DOWN on interface Tunnel1 non DR

Aug 28 10:46:12.334 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from FULL to DOWN, Neighbor Down: Interface down or detached

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel0

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#tunnel source GigabitEthernet0/0

Aug 28 10:46:12.358 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel 1

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#tunnel source GigabitEthernet0/0

Aug 28 10:46:12.382 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#exit

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#interface GigabitEthernet0/2

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#shut

Aug 28 10:46:13.590 EST-DST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.20.68.12 on interface Tunnel0

Aug 28 10:46:13.590 EST-DST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 10.3.68.12 on interface Tunnel1

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#exit

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip ftp source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip tftp source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip http client source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip radius source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no logging source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip ftp source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip tftp source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip http client source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip nat inside source list nat_acl interface GigabitEthernet0/0 overload

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : %Dynamic mapping in use, cannot change

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip radius source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#logging source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip route 0.0.0.0 0.0.0.0 dhcp

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip route 0.0.0.0 0.0.0.0 dhcp

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#exit

.Aug 28 10:46:18.194 EST-DST: %SYS-5-CONFIG_I: Configured from console by  on vty0 (EEM:FAIL_Back_TO_PRIMARY_INTERFACE)

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry#end

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Translating "end"

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT :

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : % Bad IP address or host name

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Translating "end"

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT :

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : % Unknown command or computer name, or unable to find computer address

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : CTL : cli_close called.

.Aug 28 10:46:19.970 EST-DST: %LINK-5-CHANGED: Interface GigabitEthernet0/2, changed state to administratively down

.Aug 28 10:46:20.970 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/2, changed state to down

.Aug 28 10:46:25.371 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 UP on interface Tunnel0

.Aug 28 10:46:28.639 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 UP on interface Tunnel1

.Aug 28 10:46:34.295 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from LOADING to FULL, Loading Done

.Aug 28 10:46:40.751 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from LOADING to FULL, Loading Done

This time it stayed with g0/0/

[Patrick McHenry]

Failed back to the Primary interface:

Pro Inside global      Inside local       Outside local      Outside global

tcp 192.168.168.46:51796 172.17.0.19:51796 10.20.64.40:2000  10.20.64.40:2000

icmp 192.168.168.46:7  172.17.0.20:7      173.194.43.33:7    173.194.43.33:7

Aug 28 10:46:12.270 EST-DST: %TRACKING-5-STATE: 1 ip sla 10 reachability Down->Up

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : CTL : cli_open called.

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry>

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry>enable

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry#configure terminal

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line.  End with CNTL/Z.

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#interface tunnel0

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#no tunnel source GigabitEthernet0/2

Aug 28 10:46:12.310 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down

Aug 28 10:46:12.310 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 DOWN on interface Tunnel0 non DR

Aug 28 10:46:12.310 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from FULL to DOWN, Neighbor Down: Interface down or detached

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel 1

Aug 28 10:46:12.318 EST-DST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=192.168.25.185, prot=50, spi=0x89C92382(2311660418), srcaddr=65.199.155.102, input interface=GigabitEthernet0/2

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#no tunnel source GigabitEthernet0/2

Aug 28 10:46:12.334 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to down

Aug 28 10:46:12.334 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 DOWN on interface Tunnel1 non DR

Aug 28 10:46:12.334 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from FULL to DOWN, Neighbor Down: Interface down or detached

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel0

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#tunnel source GigabitEthernet0/0

Aug 28 10:46:12.358 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel 1

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#tunnel source GigabitEthernet0/0

Aug 28 10:46:12.382 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#exit

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#interface GigabitEthernet0/2

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 10:46:12 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#shut

Aug 28 10:46:13.590 EST-DST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.20.68.12 on interface Tunnel0

Aug 28 10:46:13.590 EST-DST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 10.3.68.12 on interface Tunnel1

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#exit

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip ftp source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip tftp source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip http client source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip radius source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no logging source-interface GigabitEthernet0/2

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip ftp source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip tftp source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip http client source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip nat inside source list nat_acl interface GigabitEthernet0/0 overload

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : %Dynamic mapping in use, cannot change

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip radius source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#logging source-interface GigabitEthernet0/0

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip route 0.0.0.0 0.0.0.0 dhcp

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip route 0.0.0.0 0.0.0.0 dhcp

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#exit

.Aug 28 10:46:18.194 EST-DST: %SYS-5-CONFIG_I: Configured from console by  on vty0 (EEM:FAIL_Back_TO_PRIMARY_INTERFACE)

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry#end

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Translating "end"

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT :

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : % Bad IP address or host name

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Translating "end"

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT :

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : % Unknown command or computer name, or unable to find computer address

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

.Aug 28 10:46:18 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : CTL : cli_close called.

.Aug 28 10:46:19.970 EST-DST: %LINK-5-CHANGED: Interface GigabitEthernet0/2, changed state to administratively down

.Aug 28 10:46:20.970 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/2, changed state to down

.Aug 28 10:46:25.371 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 UP on interface Tunnel0

.Aug 28 10:46:28.639 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 UP on interface Tunnel1

.Aug 28 10:46:34.295 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from LOADING to FULL, Loading Done

.Aug 28 10:46:40.751 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from LOADING to FULL, Loading Done

mchenry#

[John Blakley]

Pat,

I see the following line:

%Dynamic mapping in use, cannot change

What I would do is change the eem script to delete the line completely before applying the new line. I would have to test this, but have you tried that? I could set up a lab this afternoon if needed.

So, what you're doing is bringing down g0/2 and then adding

action A35 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload"

Then I think you're getting the above error because you're trying to nat the same acl to different outside interfaces (again, I'd have to test). What I would do is try deleting the ip nat statement and see if you can delete the mappings as in my above post. If that works, then change your script to:

action A33 cli command "no ip nat inside source list nat_acl interfac g0/2 overload" pattern "entries"

action A34 cli command "yes"

action A35 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload"

Same thing for your fail_to_backup script:

action A34 cli command "no ip nat inside source list nat_acl interface GigabitEthernet0/0 overload" pattern "entries"

action A35 cli command "yes"

action A36 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/2 overload"

HTH,

John

[Patrick McHenry]

I tried it and it worked going from the primary to the backup. The nat statement changed to g0/2 and all was good but, when I went to the primary again I got the death sequence when I tried to delete the g0/2 nat statement - I highlighted in bold. After, I entered the nat statement manually for g0/0 and it took but then, I had to down and up interface g0/0 for it to build the tunnel again.

Aug 28 13:34:42.243 EST-DST: %TRACKING-5-STATE: 1 ip sla 10 reachability Down->Up

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : CTL : cli_open called.

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry>

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry>enable

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry#configure terminal

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line.  End with CNTL/Z.

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#interface tunnel0

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#no tunnel source GigabitEthernet0/2

Aug 28 13:34:42.283 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down

Aug 28 13:34:42.283 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 DOWN on interface Tunnel0 non DR

Aug 28 13:34:42.283 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from FULL to DOWN, Neighbor Down: Interface down or detached

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel 1

Aug 28 13:34:42.291 EST-DST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=192.168.25.185, prot=50, spi=0x66C284F1(1724024049), srcaddr=65.199.155.102, input interface=GigabitEthernet0/2

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#no tunnel source GigabitEthernet0/2

Aug 28 13:34:42.307 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to down

Aug 28 13:34:42.307 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 DOWN on interface Tunnel1 non DR

Aug 28 13:34:42.307 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from FULL to DOWN, Neighbor Down: Interface down or detached

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel0

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#tunnel source GigabitEthernet0/0

Aug 28 13:34:42.331 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#interface tunnel 1

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#tunnel source GigabitEthernet0/0

Aug 28 13:34:42.355 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#exit

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#interface GigabitEthernet0/2

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

Aug 28 13:34:42 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#shut

Aug 28 13:34:43.647 EST-DST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.20.68.12 on interface Tunnel0

Aug 28 13:34:43.647 EST-DST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 10.3.68.12 on interface Tunnel1

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config-if)#

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config-if)#exit

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip ftp source-interface GigabitEthernet0/2

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip tftp source-interface GigabitEthernet0/2

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip http client source-interface GigabitEthernet0/2

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:47 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip radius source-interface GigabitEthernet0/2

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no logging source-interface GigabitEthernet0/2

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip ftp source-interface GigabitEthernet0/0

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip tftp source-interface GigabitEthernet0/0

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#ip http client source-interface GigabitEthernet0/0

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : OUT : mchenry(config)#

.Aug 28 13:34:48 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : IN  : mchenry(config)#no ip nat inside source list nat_acl interface GigabitEthernet0/2 overload

.Aug 28 13:34:49.887 EST-DST: %LINK-5-CHANGED: Interface GigabitEthernet0/2, changed state to administratively down

.Aug 28 13:34:50.887 EST-DST: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/2, changed state to down

.Aug 28 13:34:55.643 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 UP on interface Tunnel1

.Aug 28 13:34:56.423 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 UP on interface Tunnel0

.Aug 28 13:35:02 EST-DST: %HA_EM-6-LOG: FAIL_Back_TO_PRIMARY_INTERFACE : DEBUG(cli_lib) : : CTL : cli_close called.

.Aug 28 13:35:02 EST-DST: tty is now going through its death sequence

.Aug 28 13:35:10.524 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from LOADING to FULL, Loading Done

.Aug 28 13:35:12.372 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from LOADING to FULL, Loading Done

.Aug 28 13:35:16.041 EST-DST: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel0, addr 172.20.68.1 - looped chain attempting to stack

.Aug 28 13:37:09.655 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 DOWN on interface Tunnel1 non DR

.Aug 28 13:37:11.155 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 DOWN on interface Tunnel0 non DR

.Aug 28 13:37:14.979 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from FULL to DOWN, Neighbor Down: Dead timer expired

.Aug 28 13:37:18.599 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from FULL to DOWN, Neighbor Down: Dead timer expired

.Aug 28 13:37:30.188 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 10.3.5.100 on Tunnel1 from LOADING to FULL, Loading Done

.Aug 28 13:37:32.316 EST-DST: %OSPF-5-ADJCHG: Process 1, Nbr 172.20.64.102 on Tunnel0 from LOADING to FULL, Loading Done

.Aug 28 13:37:35.496 EST-DST: %ADJ-5-PARENT: Midchain parent maintenance for IP midchain out of Tunnel1, addr 10.3.68.1 - looped chain attempting to stack

mchenry#

.Aug 28 13:37:52.353 EST-DST: %PIM-5-NBRCHG: neighbor 10.3.68.1 UP on interface Tunnel1

.Aug 28 13:37:54.033 EST-DST: %PIM-5-NBRCHG: neighbor 172.20.68.1 UP on interface Tunnel0

.Aug 28 13:37:57.706 EST-DST: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=192.168.168.46, prot=50, spi=0xE2C1EE54(3804360276), srcaddr=65.199.155.102, input interface=GigabitEthernet0/0

Thanks, Pat

[John Blakley]

Pat,

Can you post the newly revised eem script Fail_Back_To_Primary_Interface?

John

[Patrick McHenry]

event manager applet FAIL_Back_TO_PRIMARY_INTERFACE

event syslog pattern "1 ip sla 10 reachability Down->Up"

action A11 cli command "enable"

action A12 cli command "configure terminal"

action A13 cli command "interface tunnel0"

action A14 cli command "no tunnel source GigabitEthernet0/2"

action A15 cli command "interface tunnel 1"

action A16 cli command "no tunnel source GigabitEthernet0/2"

action A17 cli command "interface tunnel0"

action A18 cli command "tunnel source GigabitEthernet0/0"

action A19 cli command "interface tunnel 1"

action A21 cli command "tunnel source GigabitEthernet0/0"

action A22 cli command "exit"

action A23 cli command "interface GigabitEthernet0/2"

action A24 cli command "shut"

action A25 cli command "exit"

action A26 cli command "no ip ftp source-interface GigabitEthernet0/2"

action A27 cli command "no ip tftp source-interface GigabitEthernet0/2"

action A28 cli command "no ip http client source-interface GigabitEthernet0/2"

action A29 cli command "no ip radius source-interface GigabitEthernet0/2"

action A31 cli command "no logging source-interface GigabitEthernet0/2"

action A32 cli command "ip ftp source-interface GigabitEthernet0/0"

action A33 cli command "ip tftp source-interface GigabitEthernet0/0"

action A34 cli command "ip http client source-interface GigabitEthernet0/0"

action A35 cli command "no ip nat inside source list nat_acl interface GigabitEthernet0/2 overload" pattern "entries"

action A36 cli command "yes"

action A37 cli command "ip nat inside source list nat_acl interface GigabitEthernet0/0 overload"

action A38 cli command "ip radius source-interface GigabitEthernet0/0"

action A39 cli command "logging source-interface GigabitEthernet0/0"

action A41 cli command "no ip route 0.0.0.0 0.0.0.0 dhcp"

action A42 cli command "ip route 0.0.0.0 0.0.0.0 dhcp"

action A43 cli command "exit"

!

Thanks, Pat.

[John Blakley]

Pat,

I couldn't find anything about the error that you're receiving, but here's a thought. The run time for a script is 20 seconds. It looks like you're getting close to that threshold. I'd recommend, if you can, to raise the max run time on your event line and see if that helps.

Try changing:

event syslog pattern "1 ip sla 10 reachability Down->Up"

To:

event syslog pattern "1 ip sla 10 reachability Down->Up" maxrun 45

Let's see if that keeps the script from erroring out.

HTH,

John