interface resets causes Port channel to flap nexus 5010

Unanswered Question
Feb 21st, 2012

Hi all,

I am having the issues explained below , I did not find any bug related to the below error messages , your help is appreciated.

Here is an example of what each switch logs when a server drops offline.  Sample logs taken between 5:32am and 5:35am on Feb 20.   This particular one was having problems all weekend.  Switch #1 encountered over 2000 interface resets.  The corresponding VPC port on Switch #2 only had 13 resets.

I’d be happy to start a web meeting session with someone to show you more if it helps.

NEXUS SWITCH #1

2012 Feb 20 05:32:09 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from Ethernet1/10 to none

2012 Feb 20 05:32:09 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel10: Ethernet1/10 is down

2012 Feb 20 05:32:09 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel10: port-channel10 is down

2012 Feb 20 05:32:09 q91-sw01-5010 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel10 is down (

No operational members)

2012 Feb 20 05:32:09 q91-sw01-5010 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/10 is down (Link failure)

2012 Feb 20 05:32:09 q91-sw01-5010 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel10 is down (

No operational members)

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-SPEED: Interface Ethernet1/10, operational speed changed to 10 Gbps

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/10, operational duplex mode changed to

Full

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/10, operational Receive Flow

Control state changed to off

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/10, operational Transmit Flow

Control state changed to off

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-SPEED: Interface port-channel10, operational speed changed to 10 Gbps

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-IF_DUPLEX: Interface port-channel10, operational duplex mode changed

to Full

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel10, operational Receive Flo

w Control state changed to off

2012 Feb 20 05:32:19 q91-sw01-5010 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel10, operational Transmit Fl

ow Control state changed to off

2012 Feb 20 05:32:20 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel10: Ethernet1/10 is up

2012 Feb 20 05:32:20 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from none to Ethernet1/10

2012 Feb 20 05:32:20 q91-sw01-5010 %ETHPORT-5-IF_UP: Interface Ethernet1/10 is up in mode trunk

2012 Feb 20 05:32:20 q91-sw01-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

2012 Feb 20 05:32:20 q91-sw01-5010 %ETHPORT-5-IF_UP: Interface port-channel10 is up in mode trunk

2012 Feb 20 05:33:22 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from Ethernet1/10 to none

2012 Feb 20 05:33:22 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel10: Ethernet1/10 is down

2012 Feb 20 05:33:22 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel10: port-channel10 is down

2012 Feb 20 05:33:22 q91-sw01-5010 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel10 is down (

No operational members)

2012 Feb 20 05:33:22 q91-sw01-5010 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/10 is down (Link failure)

2012 Feb 20 05:33:22 q91-sw01-5010 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel10 is down (

No operational members)

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-SPEED: Interface Ethernet1/10, operational speed changed to 10 Gbps

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/10, operational duplex mode changed to

Full

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/10, operational Receive Flow

Control state changed to off

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/10, operational Transmit Flow

Control state changed to off

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-SPEED: Interface port-channel10, operational speed changed to 10 Gbps

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-IF_DUPLEX: Interface port-channel10, operational duplex mode changed

to Full

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel10, operational Receive Flo

w Control state changed to off

2012 Feb 20 05:33:32 q91-sw01-5010 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel10, operational Transmit Fl

ow Control state changed to off

2012 Feb 20 05:33:33 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel10: Ethernet1/10 is up

2012 Feb 20 05:33:33 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from none to Ethernet1/10

2012 Feb 20 05:33:33 q91-sw01-5010 %ETHPORT-5-IF_UP: Interface Ethernet1/10 is up in mode trunk

2012 Feb 20 05:33:33 q91-sw01-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

2012 Feb 20 05:33:33 q91-sw01-5010 %ETHPORT-5-IF_UP: Interface port-channel10 is up in mode trunk

2012 Feb 20 05:34:35 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from Ethernet1/10 to none

2012 Feb 20 05:34:35 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel10: Ethernet1/10 is down

2012 Feb 20 05:34:35 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel10: port-channel10 is down

2012 Feb 20 05:34:35 q91-sw01-5010 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel10 is down (

No operational members)

2012 Feb 20 05:34:35 q91-sw01-5010 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/10 is down (Link failure)

2012 Feb 20 05:34:35 q91-sw01-5010 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel10 is down (

No operational members)

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-SPEED: Interface Ethernet1/10, operational speed changed to 10 Gbps

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/10, operational duplex mode changed to

Full

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/10, operational Receive Flow

Control state changed to off

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/10, operational Transmit Flow

Control state changed to off

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-SPEED: Interface port-channel10, operational speed changed to 10 Gbps

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_DUPLEX: Interface port-channel10, operational duplex mode changed

to Full

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel10, operational Receive Flo

w Control state changed to off

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel10, operational Transmit Fl

ow Control state changed to off

2012 Feb 20 05:34:46 q91-sw01-5010 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel10: Ethernet1/10 is up

2012 Feb 20 05:34:46 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from none to Ethernet1/10

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_UP: Interface Ethernet1/10 is up in mode trunk

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

2012 Feb 20 05:34:46 q91-sw01-5010 %ETHPORT-5-IF_UP: Interface port-channel10 is up in mode trunk

2012 Feb 20 05:35:48 q91-sw01-5010 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel10: first operational port changed

from Ethernet1/10 to none

NEXUS SWITCH #2

2012 Feb 20 05:32:20 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_REMOVED: VLANs 999 on Interface port-channel10 are rem

oved from suspended state.

2012 Feb 20 05:32:20 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

2012 Feb 20 05:33:33 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_REMOVED: VLANs 999 on Interface port-channel10 are rem

oved from suspended state.

2012 Feb 20 05:33:33 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

2012 Feb 20 05:34:46 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_REMOVED: VLANs 999 on Interface port-channel10 are rem

oved from suspended state.

2012 Feb 20 05:34:46 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

2012 Feb 20 05:35:59 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_REMOVED: VLANs 999 on Interface port-channel10 are rem

oved from suspended state.

2012 Feb 20 05:35:59 q91-sw02-5010 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 999 on Interface port-channel10 are b

eing suspended. (Reason: Vlan is not allowed on Peer-link)

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 0 (0 ratings)
onestero Wed, 02/22/2012 - 11:38

Hi Saleem

It seems like port-channel 10 has only one link ethernet1/10 - and it flaps. Most probable cause - cble defect or sfp module defect. Start troubleshooting from replacing them.

If po10 is peerlink, and vlan 999 is only allowed vlan,  it flaps -  secondary peer will suspend vlan because it's not allowed on peerlink.

But there might be other scenarios.

Try to replace mentioned parts and then check.

HTH,

Alex

Actions

Login or Register to take actions

This Discussion

Posted February 21, 2012 at 10:05 AM
Stats:
Replies:1 Avg. Rating:
Views:1562 Votes:0
Shares:0
Tags: No tags.
 

Discussions Leaderboard

Rank Username Points
1 16,104
2 9,218
3 8,197
4 7,566
5 7,558