STP or HSRP problem

Unanswered Question
Dec 16th, 2008
User Badges:

Hi NetPro,


anyone who has the experience about the below logs ? i have configured the root bridge to the switch and HSRP configured as well.


Dec 12 15:40:24.556 CCT: %STANDBY-6-STATECHANGE: Vlan10 Group 1 state Standby -> Active

Dec 12 15:40:24.564 CCT: %STANDBY-6-STATECHANGE: Vlan10 Group 1 state Active -> Speak



your reply will be highly appreciated.


thanks.


regards,

jack

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 2.9 (9 ratings)
Loading.
Giuseppe Larosa Tue, 12/16/2008 - 02:09
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello Jack,

the local router has changed state from standby to active because it has missed three HSRP hellos from neighbor.

Then it receives again a better hello from the neighbor and it reverts to non active state


This can be seen sometimes for different reasons: the neighbor router cpu is high or the link is full. or the neighbor interface has flapped


Hope to help

Giuseppe


lamav Tue, 12/16/2008 - 02:10
User Badges:
  • Blue, 1500 points or more

Jack, it means that the routers running HSRP for this vlan experienced a change in state. The standby router went active, and then to speak.


Did the other router in the HSRP group fail? Is tracking enabled on an interface that may have gone down? Was another router added to the HSRP group?


HTH


Victor


ney25 Tue, 12/16/2008 - 02:52
User Badges:

Hi Victor,


both switches in the hsrp group. but, funny thing is only switch 1 had this logs . whereby sw 2 not even received any fail msg.

Giuseppe Larosa Tue, 12/16/2008 - 03:03
User Badges:
  • Super Silver, 17500 points or more
  • Hall of Fame,

    Founding Member

Hello Jack,

this means that only the local missed the other's switch hellos

no link faiure on the other device.


check the incoming traffic volume and if you have input errors of any type


Hope to help

Giuseppe



ney25 Tue, 12/16/2008 - 03:15
User Badges:

Hi Qiuslar,


thanks for your information, what can i do after checking the input traffic volume ?


this logs show on the vlan10 having the flapping from Active->Standby.


your information will be helping me a lots.


thanks.


regards,

jack

viyuan700 Tue, 12/16/2008 - 05:49
User Badges:
  • Silver, 250 points or more

This HSRP member has changed from standby to active (means this was standby other was active), So if this is also active now can you check are both active for vlan 10?


If only one is active and one is standby for vlan 10 then you have just missed the error message on other switch.


Do you have only vlan 10 here or there are more vlan?

ney25 Tue, 12/16/2008 - 06:16
User Badges:

hi viyuan,


yes, this switch only contained 3 vlans (vlan 10, 20 and 30).

so, only sw 1 had received the Active->Standby logs. sw 2 has no suspicious logs shown.


so, now the sw 1 is standby and sw 2 is active.


thanks.


regards,

jack

viyuan700 Tue, 12/16/2008 - 23:46
User Badges:
  • Silver, 250 points or more

"These error messages describe a situation in which a standby HSRP router did not receive three successive HSRP hello packets from its HSRP peer. The output shows that the standby router moves from the standby state to the active state. Shortly thereafter, the router returns to the standby state. Unless this error message occurs during the initial installation, an HSRP issue probably does not cause the error message. The error messages signify the loss of HSRP hellos between the peers. When you troubleshoot this issue, you must verify the communication between the HSRP peers. A random, momentary loss of data communication between the peers is the most common problem that results in these messages. HSRP state changes are often due to High CPU Utilization. If the error message is due to high CPU utilization, put a sniffer on the network and the trace the system that causes the high CPU utilization.


There are several possible causes for the loss of HSRP packets between the peers. The most common problems are physical layer problems or excessive network traffic caused by spanning tree issues"



These error messages can be due to high CPU utilization or physical/spanning tree problem.


U can check CPU utilization?

What to check in physical problem see this link


http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a0080094afd.shtml#t26



ney25 Wed, 12/17/2008 - 19:16
User Badges:

Hi Viyuan,


thanks for your information.


below is my sw config.


sw 1:

------


Vlan10 - Group 1

Local state is Standby, priority 90, may preempt

Hellotime 3 sec, holdtime 10 sec

Next hello sent in 2.394

Virtual IP address is 192.168.29.1 configured

Active router is 192.168.29.3, priority 100 expires in 8.072

Standby router is local

193 state changes, last state change 5d19h

IP redundancy name is "hsrp-Vl10-1" (default)

Vlan20 - Group 2

Local state is Standby, priority 90, may preempt

Hellotime 3 sec, holdtime 10 sec

Next hello sent in 1.910

Virtual IP address is 192.168.30.252 configured

Active router is 192.168.30.254, priority 100 expires in 7.804

Standby router is local

130 state changes, last state change 23:44:00

IP redundancy name is "hsrp-Vl20-2" (default)



Sw 2:

------


Vlan10 - Group 1

Local state is Active, priority 100, may preempt

Hellotime 3 sec, holdtime 10 sec

Next hello sent in 2.588

Virtual IP address is 192.168.29.1 configured

Active router is local

Standby router is 192.168.29.2 expires in 8.288

Virtual mac address is 0000.0c07.ac01

23 state changes, last state change 1y1w

IP redundancy name is "hsrp-Vl10-1" (default)

Vlan20 - Group 2

Local state is Active, priority 100, may preempt

Hellotime 3 sec, holdtime 10 sec

Next hello sent in 1.536

Virtual IP address is 192.168.30.252 configured

Active router is local

Standby router is 192.168.30.253 expires in 9.752

Virtual mac address is 0000.0c07.ac02

17 state changes, last state change 1y1w

IP redundancy name is "hsrp-Vl20-2" (default)



thanks.


regards,

jack

viyuan700 Wed, 12/17/2008 - 21:57
User Badges:
  • Silver, 250 points or more

sw 1:

193 state changes, last state change 5d19h

130 state changes, last state change 23:44:00


Sw2


23 state changes, last state change 1y1w

17 state changes, last state change 1y1w


Your switch 1 is changing state too much, can you check the followiong command for Sw1


show interface vlan10

show interface vlan20

show processes cpu

show ip interface brief


Also check the interface for errors to which this Sw1 is connected


ney25 Wed, 12/17/2008 - 22:35
User Badges:

Hi viyuan,


as requested. sw 1 config as below.


Vlan10 is up, line protocol is up

Hardware is Ethernet SVI, address is 0012.00f4.0f3f (bia 0012.00f4.0f3f)

Internet address is 192.168.29.2/24

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 1000 bits/sec, 3 packets/sec

5 minute output rate 1000 bits/sec, 2 packets/sec

L3 in Switched: ucast: 260886453 pkt, 61897410344 bytes - mcast: 0 pkt, 0 bytes

L3 out Switched: ucast: 126079495 pkt, 73856057436 bytes - mcast: 0 pkt, 0 bytes

278729315 packets input, 64051007405 bytes, 0 no buffer

Received 17842862 broadcasts (66184 IP multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

250976519 packets output, 96033769983 bytes, 0 underruns

0 output errors, 0 interface resets

0 output buffer failures, 0 output buffers swapped out


****************************************************************************************************************


Vlan20 is up, line protocol is up

Hardware is Ethernet SVI, address is 0012.00f4.0f3f (bia 0012.00f4.0f3f)

Internet address is 192.168.30.253/24

MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:00, output never, output hang never

Last clearing of "show interface" counters never

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 0 bits/sec, 0 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

L3 in Switched: ucast: 127517032 pkt, 74021215211 bytes - mcast: 0 pkt, 0 bytes

L3 out Switched: ucast: 117686801 pkt, 36920089185 bytes - mcast: 0 pkt, 0 bytes

127517032 packets input, 74021215211 bytes, 0 no buffer

Received 0 broadcasts (34781289 IP multicast)

0 runts, 0 giants, 0 throttles

0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

165793313 packets output, 41477436208 bytes, 0 underruns

0 output errors, 0 interface resets

0 output buffer failures, 0 output buffers swapped out


****************************************************************************************************************


CPU utilization for five seconds: 6%/0%; one minute: 6%; five minutes: 8%

PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process

27 10260104001515314164 677 3.11% 2.58% 2.73% 0 Cat4k Mgmt HiPri

28 13514389242215706604 0 1.75% 3.19% 4.87% 0 Cat4k Mgmt LoPri

45 64846364 61693175 1051 0.39% 0.30% 0.27% 0 CDP Protocol

82 19490452 103660725 188 0.39% 0.03% 0.02% 0 IP SNMP

46 92 166 554 0.15% 0.07% 0.02% 1 Virtual Exec

38 14573036 95319458 152 0.07% 0.14% 0.10% 0 Spanning Tree



and i dont see any error on the sh ip int bri.



thanks.


regards,

jack

viyuan700 Wed, 12/17/2008 - 22:45
User Badges:
  • Silver, 250 points or more

193 state changes, last state change 5d19h

130 state changes, last state change 23:44:00


Vlan10 was changed 5d19h before

vlan20was changed nearly a day before.


Stable at this moment. Looks like you have to log CPU utilization high or errors when HSRP state change to troubleshoot it.

ney25 Wed, 12/17/2008 - 22:52
User Badges:

i just curious, vlan 10 and 20 were using for years, nobody has changed the configuration.


any possibilities reason ?

viyuan700 Wed, 12/17/2008 - 23:24
User Badges:
  • Silver, 250 points or more

Can't tell what is going on? Why sw1 is missing hello packets after 4-5 days suddenly and then normal. Maybe u can further investigate is something happening on sw1 ?


Your sw2 has changed state an year before. Pretty stable

ney25 Wed, 12/17/2008 - 23:46
User Badges:

hi viyuan.


thanks for your information.


regards,

jack

Actions

This Discussion