cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
457
Views
0
Helpful
8
Replies

Strange HSRP Behavior

lamav
Level 8
Level 8

OK, this is a strange one. I must be missing something.

I have 3 routers in a lab connected to the same switch with the ethernet interfaces for all 3 routers in the same vlan on the switch -- vlan 2.

I placed all 3 routers in an HSRP group, with R! as active and R2 as standby and R3 with a priority lower than R2. No biggie -- textbook set up.

I cant get the environment to stabilize. R3, with the most inferior priority, keeps declaring itself the active router and R2 and R1 do receive that hello from R3. R1 and R2 respond with Coup messages declaring that R1 is active and R2 is standby. This goes on for ever. Then R2 gets into the act and declares itself Active.

Most bizarre are the interface resets on R2. They keep incrementing until I shut down R3s ethernet interface....???? When I shut down R3's interface, all goes normal, the environment stabilizes, the elections for active router end, no more weird stuff occurs, and R2s interface reset stop incrementing completely.

Take note that it did stabilize for a few hours and then I shut down R3's interface and brought it back up, and now it hasnt stopped with this cycle of events.

Please see attached router config and log info.

PS the switch is configured with each switchport facing the routers in vlan 2 and no errors on the interfaes.

Any thoughts?

Victor

8 Replies 8

andrew.prince
Level 10
Level 10

Just tried this in a lab on an dif ver of codes -

rt1 primary - ver 12.3(15b)

rt2 - ver 12.3(13a)

rt3 secondary - ver 12.3(15b)

failed rt1 - rt3 primary, rt2 secondary!

default timers, 3 sec helo 10 hold

try to check to see if all routers are same version?

Giuseppe Larosa
Hall of Fame
Hall of Fame

Hello Victor,

what kind of switch are you using ?

the only thing I notice is that R3 is using a FE port at 100 full while R1's and R2's are using an eth 10 full.

I also see that R3's FE has never received a unicast frame but only some broadcast frames.

and : 0 multicast frames !

R1's eth and R2's eth don't distinguish broadcast and multicast received but:

the input and output packets are quite the same.

Instead R3's FE is declaring 0 multicast received and broadcast input packets much lesser then the output packets

I think R3's FE is not actually receiving multicast hellos from the other two routers so it keeps to declare itself the Active router even if its priority is the lowest.

I would investigate on this issue

Hope to help

Giuseppe

try changing:-

1) rt1 priority 105

2) rt2 priority 90

3) rt3 priority 80

and check ios vers are the same.

HTH>

Thanks, guys.

I dont think each of the 3 routers has to be running the same version of code, Andrew.

Giuseppe:

Interesting observation, but I am not sure how this would present me with the results Im getting.

Also, I cant see the connection between R2 interface resets and R3s interface...? I shut R3's FE and R2's Eth stops resetting.

Victor

Hi,

Sounds very strange - I've not heard of interface resets in this context. If you look at the HSRP state machine here:

http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a0080094afd.shtml#topic17

You should be able to work out what events are driving the state changes.

If the routers are old then you can get issues with the older ethernet hardware, another possibility could be access-lists blocking the hsrp packets in one direction. (although that still wouldn't explain the resets).

Might help having a "show ver" and the full config for all the routers (and the switch) but without any more info it does look hardware related.

HTH

Andrew.

Mr Burns:

I agree, its bizarre.

In the process of troubleshooting, I created ACLs that would trap the HSRP traffic coming into and out of the ethernet interfaces and then logging them. I want to see who is talking to who and how often.

A weird thing, though, a 'sh access-lists" shows that the ACL is hitting up, but when I do a "sh log" to see the particulars, it isnt there.

Example:

R2#sh access-lists

Extended IP access list viewtraffic

permit udp any any eq 1985 log (214 matches)

permit ip any any

R2#

R2#

R2#sh access-lists

Extended IP access list viewtraffic

permit udp any any eq 1985 log (215 matches)

permit ip any any

R2#

R2#

R2#

R2#sh log | in %SEC-6-IPACCESSLOGP: list

R2#

R2#

R2#

After 5 minutes or so, though, I will see 2 entries of HSRP packets being received, and instead of 1 packet, it will be 100 or so, maybe less. Then 2 seconds later I will do a "sh log | in %SEC-6-IPACCESSLOGP: list" (the sh log command gets more specific just to filter out the other messages), but I get no return output that time.

Example:

R1#sh log | in %SEC-6-IPACCESSLOGP: list

Aug 22 14:51:28 UTC: %SEC-6-IPACCESSLOGP: list viewtraffic permitted udp 192.168.2.2(1985) -> 224.0.0.2(1985), 141 packets

Aug 22 14:51:28 UTC: %SEC-6-IPACCESSLOGP: list viewtraffic permitted udp 192.168.2.3(1985) -> 224.0.0.2(1985), 9 packets

R1#

R1#sh log | in %SEC-6-IPACCESSLOGP: list

R1#

Is the log buffer getting filled so fast that the ACL messages are being aged-out after only 2 minutes? I increased the buffer to 16000.

I HATE when Ciscos monitoring tools dont work the way they say they should. Ive seen variations of this so many times with logging and ACLs. Whats the point of creating ACLs to trap traffic and logging them if the log cant handle it?

Any thoughts?

Thanks

Victor

Hello Victor,

the R3 interface fas0/0 in 5h 39' should have received 13560 packets from the other two routers and they should be classified as multicast : instead it received only 1971 broadcasts.

(hello time 3 sec).

R1 shows the capability to filter the VIP MAC address:

Aug 22 09:36:30 UTC: HSRP: Et0/0 API arp proto filter, 0000.0c07.ac01 is active vMAC for grp 1 - filter

is R2 capable of this ?

No, because R2 isn't the owner of the VIP MAC address.

So R2 accepts hello messages from both R1 and R3.

Messages are not synchronized : so when receives R1's hello moves from active to standby just few msec later it receives R3's hello and makes transition standby ---> active and so on.

I think this can be disturbing the R2's eth0 because it is always adding and removing the VIP MAC from the packet filter of the NIC (the list of MAC addresses that have to be received on the wire).

Try for example to remove preemption from R2 eth0: in this case it should stop to experience HSRP states transitions and to modify the packet filter.

So I think a relation exists between R3's f0/0 not receveing other routers's HSRP hellos and the effect you see on R2.

R1 is not affected because it never changes its HSRP state and being active protects itself from messages sourced by the VIP MAC.

Hope to help

Giuseppe

For some inexplicable reason, everything is working the way it should be now.

Nothing has changed. No config changes. No hardware changes. Nothing.

It just started working again

I hate when Ciscos equipment does this. I see this all too often. I can say though that this does seem to happen with older equipment that is running old code, especially when its interacting with routers that have newer code.

Sigh...

Victor

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco