Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

ASK THE EXPERT - Nexus 5000 & 2000 series

with Lucien Avramov

Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to learn how to troubleshoot Nexus 5000 & 2000 series with Lucien Avramov.  Lucien is a customer support engineer at the Cisco Technical  Assistance Center. He currently works in the data center switching team supporting customers on the Cisco Nexus 5000 and 2000. He was previously a technical leader within the network management team. Lucien holds a  bachelor's degree in general engineering and a master's degree in  computer science from Ecole des Mines d'Ales. He also holds the  following certifications: CCIE #19945 in Routing and Switching, CCDP,  DCNIS, and VCP #66183.

Remember to use the rating system to let Lucien know if you have received an adequate response.

Lucien might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Security discussion forums shortly after the event. This event lasts through March 11, 2011. Visit this forum often to view responses to your questions and the questions of other community members.

49 REPLIES
New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Lucien

I have a 2232 and I am not able to configure it. I am running NX-OS 4.1(3)N1(1). Can you help? Thanks.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

I'm glad you asked.

The 2200 series (2232, 2248) need you to run NX-OS version 4.2 or later.

The actual version you are running on your Nexus 5000 is not capable to communicate with the 2232. Please consider an upgrade.

To decide on which version to go with, I recommend you to take a look at the release notes:

http://www.cisco.com/en/US/products/ps9670/prod_release_notes_list.html

Also here is the upgrade instructions, in the upgrade / downgrade guide:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/upgrade/421_n2_1/Cisco_Nexus_5000_Series_NX-OS_Software_Upgrade_and_Downgrade_Guide__Release_421_chapter1.html

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien,

I have a pair of new 5548's running 5.0(2)N2(1). I am trying to use Dell R810 servers with Qlogic QLE8150 CNA's. Under Windows 2008R2 I can't seem to pass a packet larger than 251 bytes through the switch from a 1Gb device to a 10Gb devices. Between 1Gb devices everything works fine. I have not tried between two 10G devices yet. From Windows a simple "ping 10.40.3.227 -l 251" works but using "-l 252" fails. This happens with multiple CNA's. The CNA itself appears to work since small pings work and the FC interface is up (but I haven't tried using any FC resources yet). Possibly related, Using the same model of server and CNA with VMware when the driver for the CNA is loaded the switch port will error out after 45 minutes or so with a "DCX-No ACK in 100 PDUs"  I'm scratching my head trying to figure out how to troubleshoot this so I can figure out if it is the switch, CNA, server or ??? Do you have any suggestions on things to look at before I call TAC? Since I don't have other cards I'm having a hard time pointing the finger at the right component. So far the 5548 makes a fine 1Gb switch but we haven't been able to do anything with 10Gb or FCoE.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

What you observe is normal, the 5548 with the actual software revision actually does not support yet the speed 1 GB, even though you can configure it on the CLI. From version 5.0.3 that will be released in the next few days, the speed 1 GB will be supported and then I think your issue will go away. At this point you were able to configure 1 GB and have 1 GB to 1 GB work, however the issues start happening when you go from 1 GB to a 10 GB.

In the meantime, while you wait for 5.0.3 release, a  quick test for you to confirm this would be to remove the speed 1000 from each physical 5548 port and try 10 gb to 10 gb connection.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

5.0(3)N1(1) just got released on cisco.com and supports 1 GB. Can you please upgrade? This should resolve your issue.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hello Lucien,

I have a question about Nexus 5548P Switches and the 1GB ports. Once I upgrade the switches, to 5.0(3)N1(1) - will I be able to direct connect a Cisco switch (maybe a 2960G) to the 5548 via SFP? Please let me know.

Also, a doubt regarding the Layer 3 upgrade to the 5548P, using the daughter card - When I am performing the initial configuration on the 5K - can I upgrade them to the 5.0.3.N1.1 and then setup vPC etc - so that adding the daughter card can be done one switch at a time (so that there will be no downtime)?

Thanks in advance!!


Re: ASK THE EXPERT - Nexus 5000 & 2000 series

1. Yes if you upgrade to 5.0(3)N1(1) you will be able to use any port on the 5548 as 1 GB speed and connect for example a 2960G.

2. You can configure your switches and then upgrade to 5.0(3)N1(1), or first upgrade and then configure vPC. Remember that in the 5548P, you will need to remove the power supplies in the back place the card and replace the power supplies. This upgrade can be done in rack, you dont need to open the chassis.

Let me know if you have any other questions.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien,

I am trying to upgrade the image on the 5548P to the 5.0(3)N1(1) code - I've downloaded the kickstart and system images, but when I run the "install all ...." command, I get an error - when I searches on some forums - it said the image could be bad. Any suggestions on why I am getting this?

NX5K-1#dir bootflash:

    4497974    Mar 08 16:03:46 2011  n5000-uk9-kickstart.5.0.3.N1.1.bin
  188747641    Mar 08 16:08:22 2011  n5000-uk9.5.0.3.N1.1.bin
NX5K-1(config)# install all kickstart bootflash:n5000-uk9-kickstart.5.0.3.N1.1.bin system bootflash:n5000-uk9.5.0.3.N1.1.bin
Verifying image bootflash:/n5000-uk9-kickstart.5.0.3.N1.1.bin for boot variable "kickstart".
[#                   ]   0% -- FAIL. Return code 0x40450009 (cannot read image info).
Pre-Upgrade check failed. Return code 0x40930011 (Image verification failed).
NX5K-1(config)# sh install all status
This is the log of last installation.
Verifying image bootflash:/n5000-uk9-kickstart.5.0.3.N1.1.bin for boot variable "kickstart".
FAIL. Return code 0x40450009 (cannot read image info).
Pre-Upgrade check failed. Return code 0x40930011 (Image verification failed).
Any help would be appreciated!

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

The size of your kickstart image does not seems right, it should be: 25148416 bytes, the MD5 checksum for it is: e5c733cdd70feac0ee746924d0372709

Your system image has the correct size.

I advise you to download again the kickstart from our website and compare the size, MD5 if need be and then run install all again.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Yes sir!

I downloaded the kickstart again from CCO - and it worked. Thanks for your help!

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien,

I have 5020s and a 5548, can I use the ports with a speed of 1 GB? Thanks Steve.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Yes you can, here are the caveats:

1. On the 5520s, you can use the first 16 ports as a speed 1 GB (port e1/1 to e1/16).

2. On the 5548, you can use any port with a speed of 1 GB, however you need run the latest NX-OS version: 5.0(3)N1(1). That code was released yesterday on cisco.com, so you may download it.

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien,

Thanks for being here with us for our help.

Nexus 5K can support only 12 N2K fiber extenders at a time. That is out of 20 ports, only 12 ports can be used to connect to N2Ks. Is this a true statement, if yes then whats the purpose of remaining 8 ports on N5k?

How many (and which) ports on 5520 can be used for VPC with N2K?

As per my understanding, VPC is configured on the downlink ports, plz confirm.

Nexus 2K fiber extender modules cuurently do not support link aggregation. Why do we have it like this, and is there any roadmap for availability fo this feature?

If it is required to dual-home the downstream switch to N2K, Flex Link feature should be used. This way spanning-tree can be avoided. Can N2K be used to connect the downstream switch in this way, and would there be more cons than the pros?

regards,

Mohsin

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Great questions.

As you say, the 5000, can support up to 12 FEX, and the 5500 up to 16 as per:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration_limits/limits_503a/nexus5k_config_limits_503_chapter1.html

Keep in mind that the 2148,2248 have 4 10 GE ports, the 2232 have 8 10 GE ports and the 2224 2 10 GE ports, so depending on model and the bandwitdh you will allocate to the FEX you will use between 1 to 8 10 GE ports for each FEX.

The ports on the Nexus 5000 series can be used for multiple type of connections: connection to other switches (Nexus 7K, Catalyst switches etc..), connection to a FEX, connection to 10 GB servers.

The FEX do support LACP. There are two scenarios:

-2148 series: a port-channel with LACP accross 2 2148 works when you configure vPC. Port-channel within 2 physical ports on the same 2148 will not work.
-2200 series: LACP will work with either vPC or multiple connections on the same FEX.

The N2K can have a switch connected downstream, however you need to be very careful with not introducing a loop in your network since there is no spanning-tree running on the FEX ports. To get your link between a FEX and a switch to come up, you will have to enable spanning-tree bpdufilter, so the link doesnt error disable.

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien,

I have 2 Nexus 5020 running NX-OS 5.0(2)N1(1) and 1 N2248..

The VPC architecture is: FEX Dual-Connected to N5K switches

VPC Peer link between both N5K is UP.

N5K ports connected to the N2K are in “"fex-fabric DOWN” reason= SDP timeout/SFP Mismatch

I’ve tried to configure the same ports in “access mode” and the port comes UP.

As soon as I configure them in “fex-fabric mode”, the ports go down.

Here is my N5K configuration:

int port-channel 102

description FEX-102

switchport mode fex-fabric

fex associate 102

vpc 102

int e1/20

description FEX-102

switchport mode fex-fabric

fex associate 102

channel-group 102

Can you help ?

Thanks ;-)

Xavière

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Thanks for your question. I assume you have only e1/20 connected on each N5K to this fex.

Can you make sure of the following, and feel free to attach the outputs back:

-show interface e1/20 transceiver details

I suspect you are not using matching SFPs on each N5K. You can use any supported SFP from the matrix, however we need them to match on each N5K when going to the same FEX.

http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/compatibility/matrix/OL_6974.html#wp55411

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Thx a lot !

SFP didn't match :-) It works now !

Xavière

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hello,

I am trying to extend the fabric from a 5020 to a 2248TP and 2232PP.

I have the fet-10g but my 5020 doesn't like it. Link light is orange on the 5020 port and off on the 2000 switches.


Eth1/17       --                 sfpInvali 1         full    10G     10g
Eth1/18       --                 sfpInvali 1         full    10G     10g

Ethernet1/17
    transceiver is present
    type is Fabric Extender Transceiver
    name is CISCO-FINISAR
    part number is FTLX8570D3BCL-C1
    revision is A
    serial number is ......

    nominal bitrate is 10300 MBit/sec
    Link length supported for 50/125um fiber is 20 m
    Link length supported for 50/125um fiber is 100 m
    Link length supported for 62.5/125um fiber is 10 m
    cisco id is --
    cisco extended id number is 4

           Transceiver Detail Diagnostics Information (internal calibration)
  ----------------------------------------------------------------------------
                Current              Alarms                  Warnings
                Measurement     High        Low         High          Low
  ----------------------------------------------------------------------------
  Temperature   35.61 C        75.00 C      5.00 C     70.00 C       10.00 C
  Voltage        3.27 V         3.63 V      2.97 V      3.46 V        3.13 V
  Current        0.40 mA  --   11.80 mA     4.00 mA    10.80 mA       5.00 mA
  Tx Power      -14.81 dBm --    2.69 dBm  -11.30 dBm   -1.30 dBm     -7.30 dBm

  Rx Power        N/A     --    2.99 dBm  -13.97 dBm   -1.00 dBm     -9.91 dBm
  ----------------------------------------------------------------------------
  Note: ++  high-alarm; +  high-warning; --  low-alarm; -  low-warning

FET-10G is documented to be support on the Nexus 5020 with the 2000 switches. I tried with NX-OS of 4.2 and latest 5.0.3 neither work.

Any ideas what is wrong?

Thanks

A.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Have you configured the port as switchport mode fex-fabric? The FET-10G only works for this mode.

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Thanks,

switchport mode fex-fabric

did the trick.

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien! Do you have any information if the SFP-10G-ER transceiver will be supported on Nexus 5548/5596 later (and if yes, when)? It seems not to be compatible at the moment according to http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/compatibility/matrix/OL_6974.html.

Markku

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

You are correct  SFP-10G-ER is not supported at this time on the 5500. It is being evaluated for support in future code releases, no more details to share as far as when at this time.

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hello Lucien,

I just want to know if ERSPAN is supported on the Nexus N5K-C5020P-BF which is running 5.0(2)N1(1). It seems the commands are there to set up ERSPAN source however the parameters like destination IP and source interfaces are not saved in  and the ERSPAN indicates as being down.

Here is sample output:

D45-N5K1-25# sh monitor session 1
   session 1
---------------
type              : erspan-source
state             : down (No src/dst configured)
ERSPAN ID         : 100
ERSPAN TTL        : 3
ERSPAN IP Prec.   : 0
ERSPAN DSCP       : 0
ERSPAN MTU        : 1500
source intf       :
    rx            :
    tx            :
    both          :
    rx            :
destination IP    :

D45-N5K1-25#

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Thanks for asking. The issue is reported in CSCtn84269, you can view this in our bug tool kit page:

http://tools.cisco.com/Support/BugToolKit/action.do?hdnAction=searchBugs

Basically, in 5.0.2 code this shows in the parser, however the feature is still not supported. In 5.0.3 and later code, this has been removed from the parser and is not an option.

Rest assured, the feature ERSPAN is in the works but is just not yet released.

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien,

is it possible to have FEX 2148 connected to two Nexus 5k for redundancy? If I look at FEX like a linecard I think it's not possible but

can you confirm this please?

Tnx

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

You can surely have a FEX (2148) or any other model dual homed to two Nexus 5ks.

To achieve such you need to configure vPC (virtual port-channel).

Here is a quick start guide:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/configuration_guide_c07-543563.html

And a step by step video:

https://supportforums.cisco.com/videos/1243

https://supportforums.cisco.com/videos/1197

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi lucien,

i had received my two first Nexus 5000 but one of them arrived without an NX-OS image and in Loader> prompt.

I tried to load by TFTP the kickstart image but the system gave me the following error message:

loader> boot tftp://172.16.1.3/n5000-uk9-kickstart.4.2.1.N2.1a.bin

Address: 172.16.1.2

Netmask: 255.255.255.0

Server: 172.16.1.3

Gateway: 172.16.1.1

Booting: /n5000-uk9-kickstart.4.2.1.N2.1a.bin console=ttyS0,9600n8nn quiet load

er_ver="pr-1.3"....ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

ALERT: got a fragmented packet - reconfigure your server

TFTP error 0 (Unknown request code.)

Error 15: File not found

I tried this process with 2 different TFTP software 3cDaemon and Solarwinds but the problem persists.

Do you have any advice on these.

Thanks,

Pepe

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Do you mind trying with tftpd32:

http://tftpd32.jounin.net/tftpd32_download.html

New Member

Re: ASK THE EXPERT - Nexus 5000 & 2000 series

Hi Lucien, it works! but we have to enable the parameter PXE Compatibility in the Advanced TFTP Options to work.

Thank you very much.

Pepe

56795
Views
30
Helpful
49
Replies