ASA5510 - Frequent loss on various interfaces [asa821-11-k8.bin]

paulwhite1977 · ‎01-15-2010

Hi Guys,

I've a test pair of ASA 5510 firewalls that run across sites in an active standby configuration both running asa821-11-k8.bin.

However I've noticed that at random times every few days the following messages appear in the log of the primary unit suggesting it has lost its IP connection on that particular interface to the secondary unit and it returns within the same second.

Of the various interfaces on this device it does appear to only affect two of them as shown below.

The interface traffic levels are fine and the layer two path is consistant and stable with no corresponding log entries present on the secondary unit.

Jan 14 2010 08:52:21 : %ASA-1-105005: (Primary) Lost Failover communications with mate on interface Test
Jan 14 2010 08:52:21 : %ASA-1-105008: (Primary) Testing Interface Test
Jan 14 2010 08:52:21 : %ASA-1-105009: (Primary) Testing on interface Test Passed

Jan 15 2010 05:20:32 : %ASA-1-105005: (Primary) Lost Failover communications with mate on interface Inside
Jan 15 2010 05:20:32 : %ASA-1-105008: (Primary) Testing Interface Inside
Jan 15 2010 05:20:32 : %ASA-1-105009: (Primary) Testing on interface Inside Passed

Traffic flows appear not to be disrupted but has anyone else experienced this and if so what was the resolution to remove these messages?

Kind Regards

P.

Kureli Sankar · ‎01-15-2010

I believe you are monitoring the interfaces and the polltime interface may be too soon (ms).

monitor interface.

http://www.cisco.com/en/US/docs/security/asa/asa80/command/reference/m.html#wp1984795

polltime interface:

http://www.cisco.com/en/US/docs/security/asa/asa80/command/reference/p.html#wp1883986

-KS

paulwhite1977 · ‎02-01-2010

Hi KS,

I've already looked at that, units are running in Active / Standby.

Device currently set as

failover polltime unit 10 holdtime 30

vilaxmi · ‎01-17-2010

Hello,

Could you please paste show failover and show interface output here ?

In a failover pair, there are some standard tests, which is done to check the failover pair health. By default , Interface health check for both active and standby units is enabled and that is the reason you get such logs in your firewall. As long, as the ifc pass the test ( they are passing traffic normally) there is nothing to worry about.

Thanks

Vijaya

paulwhite1977 · ‎02-01-2010

Hi Guys,

Please see below both are subinterfaces off a main.

The drops on interface 512 had been examined previously and are L2 drops of Microsoft NLB traffic and there is no congestion on either interface 512 or 647

interface Ethernet0/1
speed 100
duplex full
no nameif
no security-level
no ip address
!

interface Ethernet0/1.512
description xx
vlan 512
nameif xx
security-level 50
ip address 10.123.221.1 255.255.255.0 standby 10.123.221.1
!

interface Ethernet0/1.647
description yy
vlan 647
nameif yy
security-level 100
ip address 10.111.222.1 255.255.255.0 standby 10.111.222.1

show int E0/1
Interface Ethernet0/1 "", is up, line protocol is up
Hardware is i82546GB rev03, BW 1000 Mbps, DLY 10 usec
        Full-Duplex(Full-duplex), 100 Mbps(100 Mbps)
        Input flow control is unsupported, output flow control is unsupported
        Available but not configured via nameif
        MAC address 001d.7066.859b, MTU not set
        IP address unassigned
        116509693 packets input, 91809388395 bytes, 0 no buffer
        Received 15523894 broadcasts, 0 runts, 0 giants
        0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
        0 L2 decode drops
        107126525 packets output, 53659297026 bytes, 0 underruns
        0 pause output, 0 resume output
        0 output errors, 0 collisions, 3 interface resets
        0 late collisions, 0 deferred
        1 input reset drops, 0 output reset drops, 0 tx hangs
        input queue (blocks free curr/low): hardware (255/236)
        output queue (blocks free curr/low): hardware (255/105)

show int e0/1.512
Interface Ethernet0/1.512 "xx", is up, line protocol is up
Hardware is i82546GB rev03, BW 1000 Mbps, DLY 10 usec
        VLAN identifier 512
        Description: xx
        MAC address 001d.7066.859b, MTU 1500
        IP address 10.123.221.1, subnet mask 255.255.255.0
Traffic Statistics for "xx":
        59467758 packets input, 46681561868 bytes
        43694808 packets output, 24148878068 bytes
        12169985 packets dropped


show int e0/1.647
Interface Ethernet0/1.647 "yy", is up, line protocol is up
Hardware is i82546GB rev03, BW 1000 Mbps, DLY 10 usec
        VLAN identifier 647
        Description: yy
        MAC address 001d.7066.859b, MTU 1500
        IP address 10.111.222.1, subnet mask 255.255.255.0
Traffic Statistics for "yy":
        24885558 packets input, 20957775471 bytes
        31997238 packets output, 9210514868 bytes
        1731 packets dropped

======================================================
sh failover
======================================================
Failover On
Failover unit Primary
Failover LAN Interface: Stateful Ethernet0/3 (up)
Unit Poll frequency 10 seconds, holdtime 30 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 3 of 110 maximum
Version: Ours 8.2(1)11, Mate 8.2(1)11
Last Failover at: 15:15:56 UTC Nov 16 2009
        This host: Primary - Active
                Active time: 7017525 (sec)
                slot 0: ASA5510 hw/sw rev (2.0/8.2(1)11) status (Up Sys)
                  Interface 11(11.11.11.11): Normal
                  Interface xx ( 10.123.221.1): Normal
                  Interface yy (10.111.222.1): Normal
                  Interface 22 (22.22.22.22): Normal (Not-Monitored)
                slot 1: empty
        Other host: Secondary - Standby Ready
                Active time: 144528 (sec)
                slot 0: ASA5510 hw/sw rev (2.0/8.2(1)11) status (Up Sys)
                  Interface 11 (11.11.11.12): Normal
                  Interface xx( 10.123.221.2): Normal
                  Interface yy (10.111.222.2): Normal
                  Interface 22 (22.22.22.23): Normal (Not-Monitored)
                slot 1: empty

Stateful Failover Logical Update Statistics
        Link : Stateful Ethernet0/3 (up)
        Stateful Obj    xmit       xerr       rcv        rerr
        General         26580177   0          1150445    3
        sys cmd         937883     0          937516     0
        up time         0          0          0          0
        RPC services    0          0          0          0
        TCP conn        19139795   0          94820      0
        UDP conn        3139899    0          27995      0
        ARP tbl         3362600    0          90114      3
        Xlate_Timeout   0          0          0          0
        VPN IKE upd     0          0          0          0
        VPN IPSEC upd   0          0          0          0
        VPN CTCP upd    0          0          0          0
        VPN SDI upd     0          0          0          0
        VPN DHCP upd    0          0          0          0
        SIP Session     0          0          0          0

        Logical Update Queue Information
                        Cur     Max     Total
        Recv Q:         0       4       1290608
        Xmit Q:         0       27      33483858

shailesh.h · ‎04-21-2010

I gone through the couple of reading related to error messages mentioned below and views were

The loss of communication between primary and secondary firewall if error mentioned for all the interfaces
High CPU utilization in the firewall (either primary or secondary)
Incorrect polling time OR synchronization time.

To isolate the problem you may

change the cable between two firewall or for time being you may use cross cable between failover interface of two firewall
Observe the processes utilization during problem scenario
Check synchronization status and failover timers between two firewalls.

Hope with this you can make some progress

Shailesh