ASA 5520 failover question!!!! - Just put in questions & troubleshooting...

Ryan Tian · ‎12-14-2010

Two ASA 5520 (ASA SSM-10) configured as active/standby failover, they can sync configuration no problem. The only issue is that one box will show Failed after a while -- whateve I configured it as primary or secondary, always the same box. I used debug fo ifc, it shows HW failed (mate=0) in this box. I suspect it is hardware issue, any idea?

AND Another big issue:

PRIMAY box won't get ACTIVE back once rebooted, stay STANDBY as if it were SECONDARY.

Ryan Tian · ‎12-14-2010

id-asa-01# sh run fail
failover
failover lan unit secondary
failover lan interface lan-failover GigabitEthernet0/2
failover link state-failover GigabitEthernet0/3
failover interface ip lan-failover 10.254.0.1 255.255.255.248 standby 10.254.0.2

failover interface ip state-failover 10.254.0.9 255.255.255.248 standby 10.254.0
.10
id-asa-01# conf t
**** WARNING ****
Configuration Replication is NOT performed from Standby unit to Active u
nit.
Configurations are no longer synchronized.
id-asa-01(config)# debug fo ifc
fover event trace on
id-asa-01(config)# fover_health_monitoring_thread: ifc_check() group: 0, - time
= 3261720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3264220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3266720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3269220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3271720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3274220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3276720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3279220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3281720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3284220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3286720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
uncfover_health_monitoring_thread: ifc_check() group: 0, - time = 3289220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
fover_health_monitoring_thread: ifc_check() group: 0, - time = 3291720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
undeufover_health_monitoring_thread: ifc_check() group: 0, - time = 3294220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
b allfover_health_monitoring_thread: ifc_check() group: 0, - time = 3296720
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)

                       ^
ERROR: % Invalid input detected at '^' marker.
id-asa-01(config)# undfover_health_monitoring_thread: ifc_check() group: 0, - ti
me = 3299220
fover_health_monitoring_thread: ifc_check() - group 0 HW failed 1 (mate 0)
ebug all
id-asa-01(config)# end
id-asa-01# sh fail
Failover On
Failover unit Secondary
Failover LAN Interface: lan-failover GigabitEthernet0/2 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 11 of 160 maximum
Version: Ours 8.3(1), Mate 8.3(1)
Last Failover at: 11:07:20 EST Dec 14 2010
        This host: Secondary - Failed
                Active time: 99 (sec)
                slot 0: ASA5520 hw/sw rev (2.0/8.3(1)) status (Up Sys)
                  Interface outside (216.191.185.206): Normal
                  Interface backbone1 (10.96.20.2): Normal
                  Interface mrcg (10.1.1.2): Normal
                  Interface mpls (10.97.4.2): Normal
                  Interface mrts1 (10.1.5.2): Normal
                  Interface DMZ (10.1.7.2): Normal
                  Interface bpic (10.1.12.2): Normal
                  Interface internet (0.0.0.0): Normal (Waiting)
                  Interface infrastructure (0.0.0.0): Normal (Waiting)
                  Interface building (0.0.0.0): Normal (Not-Monitored)
                  Interface security (0.0.0.0): Normal (Not-Monitored)
                  Interface vpn (0.0.0.0): Normal (Waiting)
                  Interface voip (0.0.0.0): Normal (Not-Monitored)
                  Interface management (192.168.1.2): Failed (Waiting)
                slot 1: ASA-SSM-10 hw/sw rev (1.0/CSC SSM 6.3.1172.0) status (Up
/Up)
                  CSC SSM, 6.3.1172.0, Down
        Other host: Primary - Active
                Active time: 5210 (sec)
                slot 0: ASA5520 hw/sw rev (2.0/8.3(1)) status (Up Sys)
                  Interface outside (216.191.185.205): Normal
                  Interface backbone1 (10.96.20.1): Normal
                  Interface mrcg (10.1.1.1): Normal
                  Interface mpls (10.97.4.1): Normal
                  Interface mrts1 (10.1.5.1): Normal
                  Interface DMZ (10.1.7.1): Normal
                  Interface bpic (10.1.12.1): Normal
                  Interface internet (0.0.0.0): Normal (Waiting)
                  Interface infrastructure (0.0.0.0): Normal (Waiting)
                  Interface building (0.0.0.0): Normal (Not-Monitored)
                  Interface security (0.0.0.0): Normal (Not-Monitored)
                  Interface vpn (0.0.0.0): Normal (Waiting)
                  Interface voip (0.0.0.0): Normal (Not-Monitored)
                  Interface management (192.168.1.1): Normal (Waiting)
                slot 1: ASA-SSM-10 hw/sw rev (1.0/CSC SSM 6.3.1172.0) status (Up
/Up)
                  CSC SSM, 6.3.1172.0, Down

Stateful Failover Logical Update Statistics
        Link : state-failover GigabitEthernet0/3 (up)
        Stateful Obj    xmit       xerr       rcv        rerr
        General         461        0          1433       0
        sys cmd         431        0          431        0
        up time         0          0          0          0
        RPC services    0          0          0          0
        TCP conn        0          0          0          0
        UDP conn        0          0          0          0
        ARP tbl         30         0          1002       0
        Xlate_Timeout   0          0          0          0
        IPv6 ND tbl     0          0          0          0
        VPN IKE upd     0          0          0          0
        VPN IPSEC upd   0          0          0          0
        VPN CTCP upd    0          0          0          0
        VPN SDI upd     0          0          0          0
        VPN DHCP upd    0          0          0          0
        SIP Session     0          0          0          0

        Logical Update Queue Information
                        Cur     Max     Total
        Recv Q:         0       13      9301
        Xmit Q:         0       1       563
id-asa-01#id-asa-01# sh ver

Cisco Adaptive Security Appliance Software Version 8.3(1)
Device Manager Version 6.3(1)

Compiled on Thu 04-Mar-10 16:56 by builders
System image file is "disk0:/asa831-k8.bin"
Config file at boot was "startup-config"

id-asa-01 up 56 mins 38 secs
failover cluster up 10 days 20 hours

Hardware: ASA5520, 2048 MB RAM, CPU Pentium 4 Celeron 2000 MHz
Internal ATA Compact Flash, 256MB
BIOS Flash M50FW016 @ 0xfff00000, 2048KB

Encryption hardware device : Cisco ASA-55x0 on-board accelerator (revision 0x0)
                             Boot microcode   : CN1000-MC-BOOT-2.00
                             SSL/IKE microcode: CNLite-MC-SSLm-PLUS-2.03
                             IPSec microcode : CNlite-MC-IPSECm-MAIN-2.06
0: Ext: GigabitEthernet0/0 : address is 1cdf.0f2e.cbec, irq 9
1: Ext: GigabitEthernet0/1 : address is 1cdf.0f2e.cbed, irq 9
2: Ext: GigabitEthernet0/2 : address is 1cdf.0f2e.cbee, irq 9
3: Ext: GigabitEthernet0/3 : address is 1cdf.0f2e.cbef, irq 9
4: Ext: Management0/0       : address is 1cdf.0f2e.cbeb, irq 11
5: Int: Internal-Data0/0    : address is 0000.0001.0002, irq 11
6: Int: Internal-Control0/0 : address is 0000.0001.0001, irq 5

Licensed features for this platform:
Maximum Physical Interfaces    : Unlimited      perpetual
Maximum VLANs                  : 150            perpetual
Inside Hosts                   : Unlimited      perpetual
Failover                       : Active/Active perpetual
VPN-DES                        : Enabled        perpetual
VPN-3DES-AES                   : Enabled        perpetual
Security Contexts              : 2              perpetual
GTP/GPRS                       : Disabled       perpetual
SSL VPN Peers                  : 2              perpetual
Total VPN Peers                : 750            perpetual
Shared License                 : Disabled       perpetual
AnyConnect for Mobile          : Disabled       perpetual
AnyConnect for Cisco VPN Phone : Disabled       perpetual
AnyConnect Essentials          : Disabled       perpetual
Advanced Endpoint Assessment   : Disabled       perpetual
UC Phone Proxy Sessions        : 2              perpetual
Total UC Proxy Sessions        : 2              perpetual
Botnet Traffic Filter          : Disabled       perpetual
Intercompany Media Engine      : Disabled       perpetual

This platform has an ASA 5520 VPN Plus license.

Failover cluster licensed features for this platform:
Maximum Physical Interfaces    : Unlimited      perpetual
Maximum VLANs                  : 150            perpetual
Inside Hosts                   : Unlimited      perpetual
Failover                       : Active/Active perpetual
VPN-DES                        : Enabled        perpetual
VPN-3DES-AES                   : Enabled        perpetual
Security Contexts              : 4              perpetual
GTP/GPRS                       : Disabled       perpetual
SSL VPN Peers                  : 4              perpetual
Total VPN Peers                : 750            perpetual
Shared License                 : Disabled       perpetual
AnyConnect for Mobile          : Disabled       perpetual
AnyConnect for Cisco VPN Phone : Disabled       perpetual
AnyConnect Essentials          : Disabled       perpetual
Advanced Endpoint Assessment   : Disabled       perpetual
UC Phone Proxy Sessions        : 4              perpetual
Total UC Proxy Sessions        : 4              perpetual
Botnet Traffic Filter          : Disabled       perpetual
Intercompany Media Engine      : Disabled       perpetual

This platform has an ASA 5520 VPN Plus license.

Serial Number: JMX1446L2GA
Running Permanent Activation Key: 0x6932c04e 0xdce1664e 0x3cb23934 0x8b5444c0 0x
001e2ea9
Configuration register is 0x1
Configuration last modified by enable_1 at 10:21:01.619 EST Tue Dec 14 2010
id-asa-01#

jubetz · ‎12-14-2010

Your management interface has failed on Secondary:

Interface management (192.168.1.2): Failed (Waiting)

Is the switchport in the right vlan? Fix that and you should be all set.

Ryan Tian · ‎12-15-2010

Thank you Justin, that is very helpful! I put them into one vlan, Normal now. No HW error at this moment, I am going to monitor the stability.

Another big issue:

PRIMAY box won't get aACTIVE back once rebooted, stay STANDBY as if it were SECONDARY.

jubetz · ‎12-15-2010

Hi Ryan,

That will be expected. The either unit can be active. You have to manually fail back to the Primary if you want it to be active. What you are looking for is called preemption and is not available with active/standby failover - only active/active.

There shouldn't be any problem with your Secondary unit being active.

-jb

Ryan Tian · ‎12-15-2010

Hi Justin,

Thank you for the quick response, you are so great! So everything is fine now, I have to continue with my pre-8.3 -> 8.3 NAT converting (migrate from old PIX 515)... thousands of static NAT policies...no converting software?

Moalghar · ‎04-26-2020

good