Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Silver

Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdown ?!

Hi.

I have a pretty basic design with FCoE running on a Nexus 5548 with NPV (true FC to a Brocade fabric) and Nexus 4000.

Everything works fine until we have an ESX trying to reach a datastore shared by another ESX. Once the second ESX is coming, here is what happens :

N5K-01# 2012 May 15 15:39:48 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interfac e vfc107, vsan 620 is down (Gracefully shutdown)

2012 May 15 15:39:48 N5K-01 %PORT-5-IF_DOWN_NONE: %$VSAN 620%$ Interface vfc107 is down (None)

2012 May 15 15:39:48 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc107 , vsan 620 is down (waiting for flogi)

N5K-01#

N5K-01#

N5K-01# 2012 May 15 15:40:40 N5K-01 %NPV-3-ACL_UPDATE_FAILED: Device FLOGI entry update failed for pwwn 21:00:00:c0:dd:1b:82:c5 on server interface vfc123 : (null)

2012 May 15 15:40:40 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc123, vsan 620 is down (Gracefully shutdown)

2012 May 15 15:40:40 N5K-01 %PORT-5-IF_DOWN_NONE: %$VSAN 620%$ Interface vfc123 is down (None)

2012 May 15 15:40:41 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc110, vsan 620 is down (Gracefully shutdown)

2012 May 15 15:40:41 N5K-01 %PORT-5-IF_DOWN_NONE: %$VSAN 620%$ Interface vfc110 is down (None)

2012 May 15 15:40:41 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc123, vsan 620 is down (waiting for flogi)

2012 May 15 15:40:41 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc110, vsan 620 is down (waiting for flogi)

2012 May 15 15:40:48 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc106, vsan 620 is down (Gracefully shutdown)

2012 May 15 15:40:48 N5K-01 %PORT-5-IF_DOWN_NONE: %$VSAN 620%$ Interface vfc106 is down (None)

2012 May 15 15:40:48 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc108, vsan 620 is down (Gracefully shutdown)

2012 May 15 15:40:48 N5K-01 %PORT-5-IF_DOWN_NONE: %$VSAN 620%$ Interface vfc108 is down (None)

2012 May 15 15:40:48 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc106, vsan 620 is down (waiting for flogi)

2012 May 15 15:40:49 N5K-01 %PORT-5-IF_TRUNK_DOWN: %$VSAN 620%$ Interface vfc108, vsan 620 is down (waiting for flogi)

The vfc of the incoming ESX goes down (graceful shutdown ?) and then 1 minute later, all vfc also go down.

Everyone's tags (7)
6 REPLIES
Cisco Employee

Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdown ?

Hi Surya

What NX-OS are you running on Nexus 4000? Can you also paste output of "show interface flowcontrol" from the Nexus 4k?

Silver

Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdown ?

Hi.

All switches run the last version available on CCO :

4.1(2)E1(1h) for Nexus 4000 and 5.1.3 N2 for 5548.

In fact this is the same installation as described here

https://supportforums.cisco.com/thread/2121164

but another issue

I'll retrieve the output of the command.

Silver

Re: Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdo

Here it is; we can see a specific interface which receives a lot of pause frames from the associated server, but why do all the vfc go down ?

NX-BCH1-M7# sh interface flowcontrol

--------------------------------------------------------------------------------

Port       Send FlowControl  Receive FlowControl  RxPause TxPause

           admin    oper     admin    oper

--------------------------------------------------------------------------------

Eth1/1     off      off      off      off         0       8710974

Eth1/2     off      off      off      off         0       8710990

Eth1/3     off      off      off      off         0       8710978

Eth1/4     off      off      off      off         0       8711002

Eth1/5     off      off      off      off         0       8711015

Eth1/6     off      off      off      off         0       8711026

Eth1/7     off      off      off      off         1753895 8710893

Eth1/8     off      off      off      off         0       8710966

Eth1/9     off      off      off      off         0       8711056

Eth1/10    off      off      off      off         0       8711028

Eth1/11    off      off      off      off         0       8711059

Eth1/12    off      off      off      off         0       8711061

Eth1/13    off      off      off      off         0       8711062

Eth1/14    off      off      off      off         0       8711047

Eth1/15    off      off      off      off         0       8710658

Eth1/16    off      off      off      off         0       8710632

Eth1/17    off      off      off      off         0       0

Eth1/18    off      off      off      off         0       0

Eth1/19    off      off      off      off         0       0

Eth1/20    off      off      off      off         0       0

Po2        off      off      off      off         0       1742130

Cisco Employee

Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdown ?

Hello Surya

We have seen instances where if a rogue server sends too many pause frames, FIP snooping on Nexus 4000 would stop working. This would cause the upstream N5k not to see FIP Keep alives and it would bring the vFCs down. You might want to investigate why server blade/CNA on Eth1/7 is doing this.

Thanks

-Prashanth

Silver

Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdown ?

ok, maybe faulty NIC or bad driver ?

Surya

Cisco Employee

Nexus 5548UP - issue with FCoE / NPV - vfc gracefully shutdown ?

Quite possible

2326
Views
0
Helpful
6
Replies
CreatePlease to create content