Unusual VRRP Activities

Answered Question
Feb 3rd, 2010

Hi Guys,


What is the probable cause of this logs in our Core switches, I haven't change the vrrp settings, it is always showed up every night.


*Feb  2 07:27:12: %VRRP-6-STATECHANGE: Vl1 Grp 1 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl2 Grp 2 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl3 Grp 3 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl2 Grp 2 state Master -> Backup
*Feb  2 07:27:14: %VRRP-6-STATECHANGE: Vl3 Grp 3 state Master -> Backup
*Feb  2 07:27:14: %VRRP-6-STATECHANGE: Vl1 Grp 1 state Master -> Backup


Thanks in advance!


regards,

Gagamboy

Correct Answer by Ganesh Hariharan about 7 years 3 weeks ago

Hi Guys,


Fyi. These Vlans belongs to Server farm, high traffic are expected on these Vlans. What would be the solution for this problem? How can we avoid it? I believe it might cause future problem.


here are the VRRP brief info:


Vl1                1  150 3609       Y  Master  10.243.10.253    10.243.10.254  
Vl2                2  150 3414       Y  Master  10.243.20.253    10.243.20.254  
Vl3                3  150 3609       Y  Master  10.243.30.253    10.243.30.254


Thanks in advance!


Hi Gagamboy,


Try to do the following things to troubleshoot this type of problem :-


1) Try a shut / no shut on the VLAN on both switches.


2) Check the interface counters for the physical interface over which the VRRP hellos are received. Is there a physical interface issue which is inhibiting  reliable communication between the two switches? 


3) Check the CPU utilization, in particular the utilization for the processor labeled RP2 in the show processes cpu output. RP2 processes VRRP hellos and several additional traffic types, including ARP requests. A large number of ARP requests can disrupt the ability of RP2 to receive VRRP hellos.


4) Use the "Gratuitous ARP sent" counter in the output of the show vrrp command to determine the number of times that a standby router has become the VRRP master. With each status change from standby to master, the router sends a gratuitous ARP.


and if possible if both the switches are directly connected i would suggest you to configure etherchannel trunk 4 to 8 port between them as heavy amount of traffic is flowing between the switches.


Hope to help


Ganesh.H

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.5 (2 ratings)
Loading.
Reza Sharifi Wed, 02/03/2010 - 19:12

Hello Gagamboy,


The VRRP states for vlan 1,2,3 is flapping.  Do you have priority and preemption configured?

Also can you post the configs?


Reza

Ganesh Hariharan Wed, 02/03/2010 - 21:35

Hi Guys,


What is the probable cause of this logs in our Core switches, I haven't change the vrrp settings, it is always showed up every night.


*Feb  2 07:27:12: %VRRP-6-STATECHANGE: Vl1 Grp 1 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl2 Grp 2 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl3 Grp 3 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl2 Grp 2 state Master -> Backup
*Feb  2 07:27:14: %VRRP-6-STATECHANGE: Vl3 Grp 3 state Master -> Backup
*Feb  2 07:27:14: %VRRP-6-STATECHANGE: Vl1 Grp 1 state Master -> Backup


Thanks in advance!


regards,

Gagamboy

Hi Gagamboy,


VRRP router has changed the state for vlans 1,2 and 3,Is this error message is coming frequently and continous. If yes as reza suggested share your vrrp configuration for both switches.


HTH


Ganesh.H

gagamboy15 Thu, 02/04/2010 - 01:12

Hi Guys,


Below are the configs for both Core switches:


Core 1:

interface Vlan1
ip address 10.10.10.253 255.255.255.0
vrrp 9 ip 10.10.10.254
vrrp 9 preempt delay minimum 5
vrrp 9 priority 150


Core2:

interface Vlan1
ip address 10.10.10.252 255.255.255.0
vrrp 9 ip 10.10.10.254
vrrp 9 preempt delay minimum 5



Core 1:

interface Vlan2
ip address 10.10.20.253 255.255.255.0
vrrp 9 ip 10.10.20.254
vrrp 9 preempt delay minimum 5
vrrp 9 priority 150


Core 2:

interface Vlan2
ip address 10.10.20.252 255.255.255.0
vrrp 9 ip 10.10.20.254
vrrp 9 preempt delay minimum 5



Core 1:

interface Vlan3
ip address 10.10.30.253 255.255.255.0
vrrp 9 ip 10.10.30.254
vrrp 9 preempt delay minimum 5
vrrp 9 priority 150


Core 2:

interface Vlan3
ip address 10.10.30.252 255.255.255.0
vrrp 9 ip 10.10.30.254
vrrp 9 preempt delay minimum 5


Thanks in advance.

Ganesh Hariharan Thu, 02/04/2010 - 01:39

Hi,


Configuration seems to be ok can you share the show vrrp brief for core sw 1 and core sw 2


Ganesh.H

Jon Marshall Thu, 02/04/2010 - 01:49

gagamboy15 wrote:


Hi Guys,


What is the probable cause of this logs in our Core switches, I haven't change the vrrp settings, it is always showed up every night.


*Feb  2 07:27:12: %VRRP-6-STATECHANGE: Vl1 Grp 1 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl2 Grp 2 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl3 Grp 3 state Backup -> Master
*Feb  2 07:27:13: %VRRP-6-STATECHANGE: Vl2 Grp 2 state Master -> Backup
*Feb  2 07:27:14: %VRRP-6-STATECHANGE: Vl3 Grp 3 state Master -> Backup
*Feb  2 07:27:14: %VRRP-6-STATECHANGE: Vl1 Grp 1 state Master -> Backup


Thanks in advance!


regards,

Gagamboy


Gagamboy


Is this a regular occurence at roughly the same time every night. If so can you tie it in with anything else such as a large backup for example. It could simply be because the interconnect between the switches is very heavily utilised at this time and some of the VRRP hello packets are getting dropped.


Jon

gagamboy15 Thu, 02/04/2010 - 07:39

Hi Guys,


Fyi. These Vlans belongs to Server farm, high traffic are expected on these Vlans. What would be the solution for this problem? How can we avoid it? I believe it might cause future problem.


here are the VRRP brief info:


Vl1                1  150 3609       Y  Master  10.243.10.253    10.243.10.254  
Vl2                2  150 3414       Y  Master  10.243.20.253    10.243.20.254  
Vl3                3  150 3609       Y  Master  10.243.30.253    10.243.30.254


Thanks in advance!

Correct Answer
Ganesh Hariharan Thu, 02/04/2010 - 08:52

Hi Guys,


Fyi. These Vlans belongs to Server farm, high traffic are expected on these Vlans. What would be the solution for this problem? How can we avoid it? I believe it might cause future problem.


here are the VRRP brief info:


Vl1                1  150 3609       Y  Master  10.243.10.253    10.243.10.254  
Vl2                2  150 3414       Y  Master  10.243.20.253    10.243.20.254  
Vl3                3  150 3609       Y  Master  10.243.30.253    10.243.30.254


Thanks in advance!


Hi Gagamboy,


Try to do the following things to troubleshoot this type of problem :-


1) Try a shut / no shut on the VLAN on both switches.


2) Check the interface counters for the physical interface over which the VRRP hellos are received. Is there a physical interface issue which is inhibiting  reliable communication between the two switches? 


3) Check the CPU utilization, in particular the utilization for the processor labeled RP2 in the show processes cpu output. RP2 processes VRRP hellos and several additional traffic types, including ARP requests. A large number of ARP requests can disrupt the ability of RP2 to receive VRRP hellos.


4) Use the "Gratuitous ARP sent" counter in the output of the show vrrp command to determine the number of times that a standby router has become the VRRP master. With each status change from standby to master, the router sends a gratuitous ARP.


and if possible if both the switches are directly connected i would suggest you to configure etherchannel trunk 4 to 8 port between them as heavy amount of traffic is flowing between the switches.


Hope to help


Ganesh.H

Actions

This Discussion