Nexus 7k's with 2k fex connected - cannot communicate with devices connected to different FEX

[deyster94]

I have two Nexus 7k's that each have two FEX's connected to them.  The VPC between the 7k's is up and operational.  The FEX connections are online as well.  Here is my issue.  If I have device connected to FEX, it cannot communicate with a device in another FEX that has a different parent 7k.  If I am on one of the 7k's, I can communicate with any device connected to any FEX, even if the FEX is directly connected to that 7k.  There has to be something simple I am missing to get this to work. 

TIA for any help and let me know if you need parts of the config or show commands posted.

Dan

[Mike Williams]

This really should go in the datacenter forum, you will get a better response there. You can move the post if you desire.

With that said, are these single or dual homed FEXs? Do you have the necessary VLANs allowed on the VPC peer-link? I know there are a lot of caveats to peer-link traffic, but I think you would be ok in this scenario.

Also, are you having problems within a layer 2 boundary, or across a layer 3 boundary?

If they are single homed, you don't happen to be using the same FEX ID on both 7k's, are you? That would definitely cause a problem.

Regards,
Mike

Sent from Cisco Technical Support Android App

[deyster94]

I ended up calling TAC since this was causing our project to stall.  I hit a bug in version 6.2(2) since I only have an F2E card in the 7k's.  The card can do layer2 and layer3, but in the VDC settings, it was defined as only a layer 2 module.  This line under the VDC is what had to be changed:

limit-resource module-type f2e

It has to only have the 'f2e' module type listed.  If there was another module listed before the f2e that is a layer 3 module, the 7k make the f2e module layer 2 only.  This is a known bug in the current release and the BU is aware of this issue.  The TAC engineer told me there should be a fix in the next release.  The major catch was when this change was made, was after I restarted the 7k's for this change to take effect, I lost some parts of my config.  Hopefully they can get this bug fixed and the new release out soon so someone else doesn't have to deal with this mess.

[nikolasgeyer]

deyster94 wrote:
I ended up calling TAC since this was causing our project to stall.  I hit a bug in version 6.2(2) since I only have an F2E card in the 7k's.  The card can do layer2 and layer3, but in the VDC settings, it was defined as only a layer 2 module.  This line under the VDC is what had to be changed:
limit-resource module-type f2e
It has to only have the 'f2e' module type listed.  If there was another module listed before the f2e that is a layer 3 module, the 7k make the f2e module layer 2 only.  This is a known bug in the current release and the BU is aware of this issue.  The TAC engineer told me there should be a fix in the next release.  The major catch was when this change was made, was after I restarted the 7k's for this change to take effect, I lost some parts of my config.  Hopefully they can get this bug fixed and the new release out soon so someone else doesn't have to deal with this mess.

Just got bitten by this bug... fun times. Ironically nobody knew about it or why this was occurring until I found your post. Do you have the bug ID by chance?

[glen.grant]

    I find these to be much buggier than they should be in this point in their evolution.  They have been out at least 4 years now ,we shouldn't be seeing this stuff at this point.  I will have to keep this in mind because I believe we have the same setup with fexes and f2e cards and running 6.2.2 .