Two Nexus 5020 vPC etherchannel with Two Catalyst 6500 VSS

Unanswered Question
Mar 17th, 2011

Hi,

we are fighting with an 40 Gbps etherchannel between 2 Nx 5000 and 2 Catalyst 6500 but the etherchannel never comes up. Here is the config:

NK5-1

interface port-channel30

  description Trunk hacia VSS 6500

  switchport mode trunk

  vpc 30

  switchport trunk allowed vlan 50-54

  speed 10000

!

interface Ethernet1/3

  switchport mode trunk

  switchport trunk allowed vlan 50-54

  beacon

  channel-group 30

!

interface Ethernet1/4

  switchport mode trunk

  switchport trunk allowed vlan 50-54

  channel-group 30

NK5-2

interface port-channel30

  description Trunk hacia VSS 6500

  switchport mode trunk

  vpc 30

  switchport trunk allowed vlan 50-54

  speed 10000

!

interface Ethernet1/3

  switchport mode trunk

  switchport trunk allowed vlan 50-54

  beacon

  channel-group 30

!

interface Ethernet1/4

  switchport mode trunk

  switchport trunk allowed vlan 50-54

  beacon

  channel-group 30

********************************************************

********************************************************

Catalyst 6500 VSS

interface Port-channel30

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 50-54

!
interface TenGigabitEthernet2/1/2
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 50-54
channel-protocol lacp
channel-group 30 mode passive
!
interface TenGigabitEthernet2/1/3
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 50-54
channel-protocol lacp
channel-group 30 mode passive
!
interface TenGigabitEthernet1/1/2
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 50-54
channel-protocol lacp
channel-group 30 mode passive
!
interface TenGigabitEthernet1/1/3
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 50-54
channel-protocol lacp
channel-group 30 mode passive
The "Show vpc 30" is as follows
N5K-2# sh vpc 30
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
30     Po30        down*  success     success                    -         
But the "Show vpc Consistency-parameters vpc 30" is
N5K-2# sh vpc consistency-parameters vpc 30
    Legend:
        Type 1 : vPC will be suspended in case of mismatch
Name                             Type  Local Value            Peer Value            
-------------                         ----  ---------------------- -----------------------
Shut Lan                              1     No                     No                   
STP Port Type                    1     Default                Default              
STP Port Guard                  1     None                   None                 
STP MST Simulate PVST 1     Default                Default              
mode                                    1     on                     -                    
Speed                                  1     10 Gb/s                -                    
Duplex                                   1     full                   -                    
Port Mode                            1     trunk                  -                    
Native Vlan                           1     1                      -                    
MTU                                       1     1500                   -                    
Allowed VLANs                    -     50-54                  50-54                
Local suspended VLANs    -     -                      -         
We will apreciate any advice,
Thank you very much for your time...
Jose
I have this problem too.
1 vote
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 4 (2 ratings)
bruno.fernandes Thu, 03/17/2011 - 16:50

Hi,

Have you checked that you are using LACP in active mode in the 5k ?

If you are using LACP passive mode in both sides the Channel will not came UP,

show port-channel Po30

Regards,

Bruno

Lucien Avramov Thu, 03/17/2011 - 17:11

the port-channel part is a good clue.

Change the cat6k to active mode for LACP.

Also attach your show vpc brief from both 5ks.

jecheand@pe.ibm.com Thu, 03/17/2011 - 17:19

Hi Lucien,

here is the "show vpc brief"

N5K-2# sh vpc brief

Legend:

                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                   : 5  

Peer status                     : peer adjacency formed ok     

vPC keep-alive status           : peer is alive                

Configuration consistency status: success

Per-vlan consistency status     : success                      

Type-2 consistency status       : success

vPC role                        : secondary                    

Number of vPCs configured       : 2  

Peer Gateway                    : Disabled

Dual-active excluded VLANs      : -

Graceful Consistency Check      : Enabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans   

--   ----   ------ --------------------------------------------------

1    Po5    up     50-54                                                   

vPC status

----------------------------------------------------------------------------

id     Port        Status Consistency Reason                     Active vlans

------ ----------- ------ ----------- -------------------------- -----------

30     Po30        down*  success     success                    -         

31     Po31        down*  failed      Consistency Check Not      -         

                                      Performed                            

*************************************************************************+

*************************************************************************+

N5K-1# sh vpc brief

Legend:

                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                   : 5  

Peer status                     : peer adjacency formed ok     

vPC keep-alive status           : peer is alive                

Configuration consistency status: success

Per-vlan consistency status     : success                      

Type-2 consistency status       : success

vPC role                        : primary                      

Number of vPCs configured       : 2  

Peer Gateway                    : Disabled

Dual-active excluded VLANs      : -

Graceful Consistency Check      : Enabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans   

--   ----   ------ --------------------------------------------------

1    Po5    up     50-54                                                   

vPC status

----------------------------------------------------------------------------

id     Port        Status Consistency Reason                     Active vlans

------ ----------- ------ ----------- -------------------------- -----------

30     Po30        down*  failed      Consistency Check Not      -         

                                      Performed                            

31     Po31        down*  failed      Consistency Check Not      -         

                                      Performed             

I have changed the lacp on both devices to active:

On Nexus N5K-1/-2

interface Ethernet1/3

  switchport mode trunk

  switchport trunk allowed vlan 50-54

  channel-group 30 mode active

!

interface Ethernet1/4

  switchport mode trunk

  switchport trunk allowed vlan 50-54

  channel-group 30 mode active    

On Catalyst 6500

interface TenGigabitEthernet2/1/2-3

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 50-54

switchport mode trunk

channel-protocol lacp

channel-group 30 mode active

!

interface TenGigabitEthernet1/1/2-3

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 50-54

switchport mode trunk

channel-protocol lacp

channel-group 30 mode active

Thanks for your time.

Jose

Lucien Avramov Thu, 03/17/2011 - 17:45

Thanks!

Now send me the following:

show port-channel summary (from each 5k)

show etherchannel summary (from cat6k)

jecheand@pe.ibm.com Fri, 03/18/2011 - 12:44

Hi Lucien,

sorry for the delay on my response. Here are what yo ask:

N5K-1

********

N5K-1# sh port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

        I - Individual  H - Hot-standby (LACP only)

        s - Suspended   r - Module-removed

        S - Switched    R - Routed

        U - Up (port-channel)

--------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

      Channel

--------------------------------------------------------------------------------

5     Po5(SU)     Eth      LACP      Eth1/1(P)    Eth1/2(P)   

30    Po30(SD)    Eth      LACP      Eth1/3(D)    Eth1/4(D)   

31    Po31(SD)    Eth      NONE      Eth105/1/1(r) 

105   Po105(SD)   Eth      NONE      Eth1/5(D)   

N5K-1#

N5K-2

********

N5K-2# sh port-channel summary

Flags:  D - Down        P - Up in port-channel (members)

        I - Individual  H - Hot-standby (LACP only)

        s - Suspended   r - Module-removed

        S - Switched    R - Routed

        U - Up (port-channel)

--------------------------------------------------------------------------------

Group Port-       Type     Protocol  Member Ports

      Channel

--------------------------------------------------------------------------------

1     Po1(SD)     Eth      NONE      --

5     Po5(SU)     Eth      LACP      Eth1/1(P)    Eth1/2(P)   

30    Po30(SD)    Eth      LACP      Eth1/3(D)    Eth1/4(D)   

31    Po31(SD)    Eth      NONE      Eth109/1/1(r) 

109   Po109(SD)   Eth      NONE      Eth1/5(D)   

N5K-2#

VSS

******

Core_VSS#show etherchannel summary

Flags:  D - down        P - bundled in port-channel

        I - stand-alone s - suspended

        H - Hot-standby (LACP only)

        R - Layer3      S - Layer2

        U - in use      N - not in use, no aggregation

        f - failed to allocate aggregator

        M - not in use, no aggregation due to minimum links not met

        m - not in use, port not aggregated due to minimum links not met

        u - unsuitable for bundling

        d - default port

        w - waiting to be aggregated

Number of channel-groups in use: 7

Number of aggregators:           7

Group  Port-channel  Protocol    Ports

------+-------------+-----------+-----------------------------------------------

1      Po1(SD)          -       

2      Po2(SD)          -       

3      Po3(SD)          -       

4      Po4(SD)          -       

10     Po10(RU)         -        Te1/1/1(P)     Te1/5/4(P)    

20     Po20(RU)         -        Te2/1/1(P)     Te2/5/4(P)    

30     Po30(SD)        LACP      Te1/1/2(D)     Te1/1/3(D)     Te2/1/2(D)    

                                 Te2/1/3(D)    

Core_VSS#

Thanks for your time.

Jose

Lucien Avramov Fri, 03/18/2011 - 13:04

Jose all the ports are down.

1. Are you sure it's cabled right? What does give you a show cdp neighbor on the 3 devices?

2. What is the reason for the interface to be down? (show interface e1/3, show interface e1/4 etc...)

jecheand@pe.ibm.com Fri, 03/18/2011 - 13:23

Lucien,

1.- the interfaces are cabled with a single mode fiber using in the VSS X2-10GB-SR and in the N5K SFP-10G-SR but the interfaces never goes down. With CDP only see the peer-link interfaces (mgmt0)

2.- When we change the etherchannel configuration the interfaces on the N5K or VSS goes up but after a while one of them enters on error disable and the ports goes down.

Any advice?

Thanks

Lucien Avramov Fri, 03/18/2011 - 20:36

This should be working, it's almost like the VSS or the 5k switches think the other pair of devices are not the same logical one.

Can you enable CDP on the cat6k? Id really like to confirm we are indeed going to the same catalyst vss pair.

What happens if you first only bring up 1 pair of 20 GB in vpc 30 that goes to one of the two cat6k?

Finally, what code are you running on the n5k? Can you send me the following from each n5k:

-show vpc consistency-parameters global

-show queuing int e1/1

jecheand@pe.ibm.com Sat, 03/19/2011 - 20:07

Lucien,

i had found what was happening. Change the single mode by multi mode fiber and the port-channels works instantly. Now i have another question:

Can we connect servers and blade switches to fabric extenders using vPC and port-channels between different N2K?

Thanks for your time.

Jose

Lucien Avramov Sat, 03/19/2011 - 21:59

Ok, this makes more sense now. Good to hear you got the right cable for the SR optics.

Yes you can surely connect servers, switches are tricky behind the FEX. The FEX is a nic card extender, it doesnt support spanning tree.

If you introduce a switch be very careful to not introduce a loop in your network. (You will have to enable spanning-tree bpdufilter).

For vPC a few references:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/configuration_guide_c07-543563.html

And video:

https://supportforums.cisco.com/videos/1243

jecheand@pe.ibm.com Sun, 03/27/2011 - 15:50

Hi Lucien,

how do i ask for the FCoE license? We have bougth the FC module and tranceivers FC 8Gbps but when i put the show license usage the FC license is temporary.

How else do i need to buy to activate this or how do i know if we had bougth but we didn't ask for the licenses?

Thanks

Jose

Lucien Avramov Sun, 03/27/2011 - 16:00

Jose,

You need to get a PAK number from your account team / partner in order to request the license via www.cisco.com/go/license.

There is a grace period that is enabled for a while on the switch, so you can find out the PAK and receive your license.

Once you have your license file, you copy it to the switch and use the command install license.

mohammedah Mon, 03/28/2011 - 07:38

I have an ESX Server (migrated to Nexus 1000v)  attached to two 10G ports of two Nexus 5Ks using vPC. This same server has a 1G NIC card attached FEX (2248) for redundancy. Under normal circumstances the 10G links are forwarding the the 1G link is just hot stadby. When I shut down the 10G link the 1 G link immediatly starts forwarding which is exactly what I want. The problem is when I bring up the 10G links, they will not preempt the 1G link. I have to manually shut the 1G so that the 10Gs start forwarding. Then I will bring up the 1G and it will be in hot-stand by mode. I want the 10Gs to preempt the 1G without me intervening manually. How can I do this??? Below is the port-profile configuration on the 1000v.

port-profile type ethernet uplink_1G_Nexus
  description to applied on 1G NICs_FEX
  vmware port-group
  switchport mode trunk
  switchport trunk native vlan 74
  switchport trunk allowed vlan 73-74,165,180,811,813-814,816-822,824,826-830
  channel-group auto mode on sub-group cdp
  no shutdown
  system vlan 74,180,811,819-821
  state enabled

port-profile type ethernet system_uplink
  description to be applied on the two 10G vmnics
  vmware port-group
  switchport mode trunk
  switchport trunk native vlan 74
  switchport trunk allowed vlan 73-74,153,165,180,811,813-814,816-822,824,826-830
  channel-group auto mode active
  no shutdown
  system vlan 74,180,811,819-821
  state enabled

aijazbeigh Fri, 04/27/2012 - 00:16

Hi,

I have got two DCs. Each DC has a pair or 5596. We have setup VPC Peering within each DC between two 5596s

Now I need to connect two DCs to each other. I have connected them like  normal trunk ports. Is it possible If I can connect them together as  etherchannel using VPC at both ends. VPC domain at both DCs is  different.Is it possible to make 2 10gbps links one.

.

Regards

Aijaz

iker.santamaria... Wed, 11/23/2011 - 11:23

Hi,

first you need configure LACP mode active in VSS-N5k Port-Channel side and active or pasive in N5K-VSS.

Be sure that the vPC domain in N5K is different that the VSS domain.

The N5K has Bridge Assurance activated by default, therefore you has two ways:

     Configure both Port-Channels with "spanning-tree port type network", this is the Bridge Assurance  mode or

     Configure N5K Port-Channel with "spanning-tree port type normal".

We have it configured and working.

I hope I've helped.

Actions

Login or Register to take actions

This Discussion

Posted March 17, 2011 at 4:29 PM
Stats:
Replies:16 Avg. Rating:4
Views:13272 Votes:1
Shares:0
Tags: No tags.

Discussions Leaderboard