Solved: C2921 Router doesn't receive HSRP packets: Really weird problem!

lap · ‎10-14-2010

Hi,

I have a really weird issue. A really simple setup:

Router 1------Switch-------Router 2

Router 1 is a C881

Router 2 is a C2921

Switch is a C3750

Router 1 can see the HSRP hello packets from Router 2: ( I can see that Router 1 is sending and receiving Hello HSRP packets)

Router 1#sh standby brief
                     P indicates configured to preempt.
                     |
Interface   Grp Pri P State   Active          Standby         Virtual IP
Vl100       10   100 P Standby 192.168.107.3   local           192.168.107.254

But Router 2 doesn't receive any HSRP hello packets from Router 1 (if I debug stanby I only see HSRP packets send out by the router itself)

Router 2#sh standby brief
                     P indicates configured to preempt.
                     |
Interface   Grp Pri P State   Active          Standby         Virtual IP
Gi0/1       10   150 P Active local           unknown         192.168.107.254

The ports on the switch are configured exactly the same and in same VLAN.

If I do some sniffing on the switch I can see both Hello packets from Router 1 and Router 2.

I though this was a bug on C2921 so I have upgrade the router to : c2900-universalk9-mz.SPA.150-1.M3.bin but it didn't help. I have shut, unshut the interface without success!

Here the HSRP config of Router 2:

interface GigabitEthernet0/1
description ***Conected to LAN***$ES_LAN$$ETH-LAN$
ip address 192.168.107.3 255.255.255.0
ip access-group INSIDE_OUT in
no ip redirects
ip nat inside
ip inspect FWINSPECT_LAN in
ip virtual-reassembly
delay 10
duplex auto
speed auto
standby 10 ip 192.168.107.254
standby 10 timers 1 3
standby 10 priority 150
standby 10 preempt

Here the HSRP config of Router 1:

interface Vlan100
description ***LAN-VLAN***

ip address 192.168.107.4 255.255.255.0

no ip redirects
ip nat inside
ip virtual-reassembly
standby 10 ip 192.168.107.254
standby 10 timers 1 3
standby 10 preempt

Switch config:

Switch#sh cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
D - Remote, C - CVTA, M - Two-port Mac Relay

Device ID        Local Intrfce     Holdtme    Capability Platform Port ID
Router2
                 Gig 1/0/23        155             R S I CISCO2921 Gig 0/1
Router1
                 Gig 1/0/21        167             R S I 881       Fas 0
SW-SRV3#sh run in
SW-SRV3#sh run interface g1/0/23
Building configuration...

Current configuration : 87 bytes
!
interface GigabitEthernet1/0/23
switchport mode access
spanning-tree portfast
end

SW-SRV3#sh run interface g1/0/21
Building configuration...

Current configuration : 87 bytes
!
interface GigabitEthernet1/0/21
switchport mode access
spanning-tree portfast
end

No errors on the interfaces at all!

Any ideas guys?

Regards,

Laurent

milan.kulik · ‎10-15-2010

Hi,

I believe the incoming ACL might be blocking the HSRP packets.

Don't forget they are sent to a multicast (224.0.0.2, if I remember correctly) destination IP address.

BR,

Milan

View solution in original post

Jon Marshall · ‎10-14-2010

Laurent

You have inspect running on router 2 ie.

int gi0/0

ip inspect FWINSPECT_LAN in

could you temporarily remove this and retest.

Jon

lap · ‎10-14-2010

Hi Jon,

I have tried but the problem is the same!

Regards,

Laurent

milan.kulik · ‎10-15-2010

Hi,

how does the ACL INSIDE_OUT look like?

HTH,

Mian

Timothy Stewart · ‎10-15-2010

Hello,

Based on what you are describing, the 2921 is not seeing the hsrp hello's from the 881. We know the hello's are being sent to the switch since you saw them on the sniffer. When you setup the sniffer, did you span port g1/0/23 on the switch or did you just connect it into the vlan? Making the assumption that you spanned the port that the 2921 is connected to, we can safely assume the hello's from the 881 are are leaving the switch via that port. For the 2921 to bring the hello's in, the interface's software address filter must be programmed to listen to the hsrp destinationmac address. To check to see if this was done properly, issue the "show controller gig 0/1" command on the 2921. Look for the mac address "0100.5e00.0002" in the "Software MAC address Filter" section, for example:

F340.06.23-2900-12#sho controller gig 0/1 | be Software
Software MAC Address Filter (hash:length/addr/mask/hits)
--------------------------------------------------------
0x000: 0 ffff.ffff.ffff 0000.0000.0000         1
0x05C: 0 0100.5e00.0002 0000.0000.0000         0   <--------this is the hsrp mac address
0x080: 0 8843.e1b2.7661 0000.0000.0000         0
0x0C0: 0 0100.0ccc.cccc 0000.0000.0000       569
0x0C0: 1 0180.c200.0002 0000.0000.0000         0
0x0C5: 0 0180.c200.0007 0000.0000.0000         0

Software filtered frames: 56
Unicast overflow mode: 0
Multicast overflow mode: 1
Promiscuous mode: 0
Total Number of CAM entries: 1
Port Stopped: N

Internal Loopback Set: N

If the mac address is not in the table, then the interface will ignore the hsrp hello's. To fix this, try removing the hsrp config from the interface, then re-adding it. Once this is done, check the filter again. While we are at it, please also verify that a show cdp neighbor on the 2921 shows the switch.

Thanks

Tim

lap · ‎10-15-2010

Hi tim,

Thanks for your detailed e-mail. Here is the result of the command on the router:

Software MAC Address Filter (hash:length/addr/mask/hits)
--------------------------------------------------------
0x000: 0 ffff.ffff.ffff 0000.0000.0000       178
0x03D: 0 68ef.bdb6.f380 0000.0000.0000         0
0x0C0: 0 0180.c200.0002 0000.0000.0000         0
0x0C0: 1 0100.0ccc.cccc 0000.0000.0000         0
0x0C5: 0 0180.c200.0007 0000.0000.0000         0

Software filtered frames: 31267
Unicast overflow mode: 0
Multicast overflow mode: 1
Promiscuous mode: 0
Total Number of CAM entries: 1
Port Stopped: N

Internal Loopback Set: N

So no hsrp mac address!

Show cdp neighbor from C2921:

C2921#sh cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater

Device ID Local Intrfce Holdtme Capability Platform Port ID

Switch
Gig 0/1 137 R S I WS-C3750G Gig 1/0/23
C2921#

I remove HSRP config from interface and on again:

C2921(config)#int g0/1
C2921(config-if)#no standby 10 ip 192.168.107.254
C2921(config-if)#no standby 10 timers 1 3
C2921(config-if)#no standby 10 priority 150
C2921(config-if)#no standby 10 preempt
C2921(config-if)#end
C2921#sh stand
C2921#sh standby bri
C2921#sh standby brief

DK-STILLING1#

And then reply again:

DK-STILLING1#sh standby brief
                     P indicates configured to preempt.
                     |
Interface   Grp Pri P State   Active          Standby         Virtual IP
Gi0/1       10   150 P Active local           unknown         192.168.107.254
DK-STILLING1#

Software MAC Address Filter (hash:length/addr/mask/hits)
--------------------------------------------------------
0x000: 0 ffff.ffff.ffff 0000.0000.0000       182
0x03D: 0 68ef.bdb6.f380 0000.0000.0000         0
0x0C0: 0 0180.c200.0002 0000.0000.0000         0
0x0C0: 1 0100.0ccc.cccc 0000.0000.0000         0
0x0C5: 0 0180.c200.0007 0000.0000.0000         0

Software filtered frames: 31517
Unicast overflow mode: 0
Multicast overflow mode: 1
Promiscuous mode: 0
Total Number of CAM entries: 1
Port Stopped: N

Internal Loopback Set: N

Weird! And yes I did span the port where C2921 is connected.

Regards,

Laurent

hardiklodhia · ‎10-15-2010

Hi,

Basic Check:

Can u ping R1 from R2? You have created Vlan interface on 881 and fast ethernet port is connected to switch. Is that port a trunk port carrying other VLANs too?

milan.kulik · ‎10-15-2010

Hi,

I believe the incoming ACL might be blocking the HSRP packets.

Don't forget they are sent to a multicast (224.0.0.2, if I remember correctly) destination IP address.

BR,

Milan

lap · ‎10-15-2010

Hi Milan,

You are right!!! Thank you very much. I didn´t think that packets destined to the router itself will be denied by the inbound ACL.

Thanks to all of you for your help.

I add following statement in the ACL:

permit udp host 192.168.107.4 host 224.0.0.2 eq 1985

And now I can see the HSRP MAC address:

Software MAC Address Filter (hash:length/addr/mask/hits)
--------------------------------------------------------
0x000: 0 ffff.ffff.ffff 0000.0000.0000     18214
0x006: 0 0000.0c07.ac0a 0000.0000.0000         0
0x03C: 0 68ef.bdb6.f381 0000.0000.0000         0
0x054: 0 0100.5e00.000a 0000.0000.0000      6379
0x05C: 0 0100.5e00.0002 0000.0000.0000     16695
0x0C0: 0 0100.0ccc.cccc 0000.0000.0000       246
0x0C0: 1 0180.c200.0002 0000.0000.0000         0
0x0C5: 0 0180.c200.0007 0000.0000.0000         0

Regards,

Laurent

lap · ‎10-15-2010

Yes no problem I can ping between the two routers.

I did forget to mention that but the two routers are EIGRP neighbors.

Regards,

Laurent