Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Read the bioWith Lucien Avramov

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn about design, configuration and troubleshooting of the Cisco Nexus 2000, 3000, 5000 Series with cisco Expert Lucien Avramov. Lucien Avramov is a technical marketing engineer in the Server Access Virtualization Business Unit at Cisco, where he supports the Cisco Nexus 5000, 3000 and 2000 Series. He has several industry certifications including CCIE #19945 in Routing and Switching, CCDP, DCNIS, and VCP #66183

Remember to use the rating system to let Lucien know if you have received an adequate response. 

Lucien might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Data Center sub-community   discussion forum shortly after the event.   This event lasts through October 21, 2011. Visit this forum often to view responses to your questions and the questions of other community members.

69 REPLIES
New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Lucien,

Can you suggest any drawbacks or any potentiall pitfalls in using a pair of Nexus 3064s for core routing/switching and a pair of 5596 switches for the access layer?

Also, do you know when to expect VPC to be available on the Nexus 3064 switches?

Thank you

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Thanks for your question.

There are limitations for L3 multicast type of traffic.

What would be the applications / usage of this type of design?

Any reason why you would prefer a higher tier with N3K instead of N5K with L3?

VPC is already shipping on the latest N3K code on CCO.

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

We have a small server room that requires 10Gig connectivity plus several 6513s.  A Nexus 7000 doesn't seem ideal for this location so I decided to use a pair 3064 switches that will be the aggregation point for approximately10 6500s via 2 x 1Gig uplinks.  The 6500s are for dense user access requirements.  I would have used the 5596 switches as an alternative for the 3064s as the fiber aggregation but I was told they will not support Netflow in the routing module nor is it in the road map.

The 5596 switches will server 10Gig access for my virtual servers.  If the Netflow feature is offered down the road I may use the 5500s instead.

Thank you Lucien!

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi out there

We have a couple of nexus 5010p and are low on ports - I was considering this 10 gig 6 ports expansion-module but on the other hand I can get many extra ports trough a 10 gig fex instead - are there any drawbacks by this? how many fex can I expect a 5010 can handle and will it add som extra latency compared with the "build-in ports" ?

best regards /ti

Ask the Expert: Nexus 5000, 3000 and 2000 Series

You can have up to 12 Fabric Extenders on the 50x0 family. So if you go with 10 2232, that will give you up to 320 10GE ports, modulo you tolerate a 4:1 oversubscription. No drawbacks as far as capability: the 10 GE fabric extender port will provide you 10 GE ethernet or FCoE capacity as a 10GE port on the Nexus 5000. If will add around 800 ns latency. Do you have specific latency requirements?

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

hi again

Well specific latency requirements - just as fast as possibly - we are running iSCSI instead of FCoE (or FC) so as fast as possibly might be the best answer here...

My concern is just that I have a pretty fast cut-though switch here and if I add some extra latency with the fex instead of using the build-in ports it might have been better to buy the expansion module instead. I don't expect that I am going to use all the ports in the fex (we are cabling directly from the nexus to a 10gig pass-through module of a Dell M1000 bladecenter so we 10 gig all the way though to the servers) - but one newer know - better have to many ports than be low on ports..

best regards /ti

Ask the Expert: Nexus 5000, 3000 and 2000 Series

The latency on the 5010 is around 3.4 usec, a fex will add you 0.8 usec if you add 10G ports on the FEX instead of the 5010, so that would be a choice for you to make. So there is a trade off.. port count / port cost or port latency.

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi Lucien,

i've got an problem with the Ports which are configured as 1G-Ports.

The Problem is, that the connection to other Devices are "timeouted". Ping works fine, but when data is transfered to other Devices via the 1G Port, the transfer stopped because of a timeout.

Have you got any idea? FLowcontrol?

Thanks!

Ask the Expert: Nexus 5000, 3000 and 2000 Series

What type of end to end speed do you have? Are you oversubcribing a 1 GE link? Flow control does not create timeouts.

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi Lucien,

the end-to-end connection ist 1Gbit to 10Gbit.

Torsten

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Do you have a 5500 serie? If so, can you  make sure you are running 5.0.3 code on Nexus 5000, since that is the version that supports 1GE on 5500 series.

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi again

Another question - I have added a vpc between 2 nx5010 where I have a 20 g peerlink between and a mgmt-interface acting as keepalive link. Now I add another vpc between a remote set of nexus (similary) and has tested if this can be used as failover between two sites - works fine. but - what happens if my keepaliave link fails (uplinket it to a 3. switch where we run all the management links on)

best regards /ti

Ask the Expert: Nexus 5000, 3000 and 2000 Series

If you peer-keepalive fails and your peer-link is up you are fine. If your peer-link fails and your keep-alive is up the secondary will suspend it's vpc ports.

Here is a useful document the vPC operations guide:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/operations/n5k_vpc_ops.html

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

hi again

ok - now I try to understand what you are telling me - the secondary will suspends its vpc ports - this means that the the primary will continue to forward packets and the other nexus - the secondary - will suspend its vpc similary as if I run into a blocked port (spt) - so that I f.ex in my case here with a vpc between a local and remote set of nexus will only run on one link?

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Correct that is in case you have a failure of N5K. If you have a fabric extender that is dual homed to two nexus 5000 switches, then this will allow you to have active / active

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi Lucien,

I've some doubts about stability and reliability of Nexus 5000 firmware...

4.2(1)N2(1)  Release Date: 30/JUL/2010

5.0(2)N1(1)  Release Date: 18/OCT/2010

5.0(2)N2(1)  Release Date: 20/DEC/2010

5.0(3)N1(1)  Release Date: 03/MAR/2011 --> deferred

5.0(3)N1(1c) Release Date: 07/MAY/2011

5.0(3)N2(1)  Release Date: 13/JUN/2011

5.0(3)N2(2)  Release Date: 09/SEP/2011

After 4 update in 5 month and after a deferred version i've stopped upgrade on 5.0(3)N2(1) because i can stop my network so often...

Nexus firmware 5.0(3)N1(1c) automatically reboot our two Nexus 5010 every 12/15 days; support suggestion was  update to 5.0(3)N2(1) and the new version was up for 28 days... a great improvement !!!

And then I've read incomplete Release Notes or caveat resolved but not listed...

What is happening ?!?

I've installed 4 Nexus 5010 and 12 N2K-C2248TP and every firmware upgrade is a conflict with my management due to real cause and resolution for the new firmware update ...

I cannot continue with this level of service...

Any suggestion ?!?

Regards

Marco

Ask the Expert: Nexus 5000, 3000 and 2000 Series

I'd suggest to work with our TAC engineers and escalation team to exactly find the issues you were running into and you can have them also involve me so I can work with you on a get well plan. There were reasons for the deferral and we had introduced new platforms and new software features which required to have a few releases in 5.0.3 code. In general due to the high feature velocity, the latest version is the greatest, here for instance 5.0(3)N2(2), but I would suggest to first identify precisely the root cause of your current issue(s).

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi Lucien! What is the maximum number of hardware port-channels in Nexus 5548 and 5596? If I remember correctly it was 16 in 5010 and 5020. Is the hw port-channel allocation logic also the same as in 5010 and 5020?

Ask the Expert: Nexus 5000, 3000 and 2000 Series

I'm glad you asked, it's a higher number on the 5500 series generation of switches.

There is no limitation besides the number of ports here.

You can have up to 48 port-channels on 5548 and 96 on the 5596.

We have a document that shows limits on both platforms here, it's called configuration limits:

http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hi,

we have just put a new core switch up for a customer, currently one 5548 feeding L2 into several 2960S switches, as well as 4 FEX 2248 (with 2x10GE each) for Data Center access in two locations. The 5548 and two 2248 are in one data center, the other two 2248 are in the second DC. Early next year the customer will be adding a second 5548 for full redundancy (currently, the servers in the DCs are connected to both 2248's in DC, which doesn't help if the 5548 crashes or dies). I've seen docs about doing a dual 5548 configuration with the FEXes connected to two N5K's. What do I need to watch out for when extending the current configuration? How is the configuration of the N5k's handled in respect of the FEX ports? Manually I assume?

Ask the Expert: Nexus 5000, 3000 and 2000 Series

When you will extend the configuration, you will be able to dual home the FEX to the Nexus 5000s, each FEX being dual connected to both upstream Nexus 5000. This will be what we call a vPC design. Depending on the number of ports free on your 2960 / 2248s, you could first configure both Nexus 5000s in vPC, then one FEX on each side dual home in vPC at the time. Migrate the server ports to the other FEX or on the 2960, then when the FEX comes back online in ''vpc'' topology you can move servers to this FEX and upgrade the other one in each site. This would minimize the down time in the design topology change.

Here is a document about quick start on vPC, a design guide and a operations guide:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/configuration_guide_c07-543563.html

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572834-00_STDG_NX-OS_vPC_DG.pdf

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/operations/n5k_vpc_ops.html

New Member

Re: Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hello I have a question about vPCs ..relating to being visble to the third switch via port-channel.Correct me if am wrong and expand on this staement please " Allows a single device to use port channel across two upstream devices'. Am trying to grasp the main comcept of vPCs and how to verify them in a live environment on the involved devices assuming its spannoing through more than two Nexus switches

Ask the Expert: Nexus 5000, 3000 and 2000 Series

vPC always works for a pair of Nexus switches (7ks, 5ks, 3ks). The pair of switches will be configured as vPC and will then appear as a ''single'' logical switch from the point of view of the device you port-channel to them (one or more link to each).

Here is more information about the vPC design:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572834-00_STDG_NX-OS_vPC_DG.pdf

Some documentation about how it's configured can clarify the concept:

http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/configuration_guide_c07-543563.html

Here is also a webcast about 5k and vPC (the concept applies as well to N7k and N3k):

https://supportforums.cisco.com/videos/1228

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

HI Lucien,

   I have one problem , I have two N5k with single Nortel Core switch.My uplink is 1G ports.I was able to ping the outside the network.But when access data ,it wont work. If I remove the redundant link it is working fine.Now I have upgraded firmware from 5.0(2) to 5.0(3). Now everything is working fine.What could be the problem.Can you pls explain ?

Regards,

Ajith

Ask the Expert: Nexus 5000, 3000 and 2000 Series

I'm guessing you have Nexus 5548. On the 5500 series, we started supporting 1G speed from 5.0.3 version.

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Hello Lucien

We will buy a pair of Nexus 5000 to interconnect racks containing  Dell chassis. They each contain a pair of switches 3130X. In a rack  there are four chassis.

We would like to make a stack of 3130X inside the rack and connect it with the pair of Nexus using VPC.

The goal is to have a switch at the logical level (the stacks of 8  3130X) connected to a swicth (the pair of Nexus). These two technologies  can be combined?

Thanks

Ask the Expert: Nexus 5000, 3000 and 2000 Series

As long as it's compliant with dot1q and lacp there should be no problem.

New Member

Ask the Expert: Nexus 5000, 3000 and 2000 Series

hi lucien

even though it is not a vpc forum I'll try this question - how do I verify load-balancing between the nexus boxes in a vpc domain? I have a set of nx5020 which holds a vpc domain - this vpc is interconnected to another set of nexus boxes which also holds another vpc domain. These two vpc domains are interconnected with  2 10 gig links - and I hoped that I somewhere/somehow could verify load-balancing and see how many links participate in this vpc etc - but hmm - ? - well as far as I can see it is a bit cumbersome to verify which links participate into a vpc and the traffic- the boxes exchange a lot of info about the state - are there some commands here I don't know about or something on the road map?

best regards /ti

Ask the Expert: Nexus 5000, 3000 and 2000 Series

Ti, from a server or a switch attached in a vPC to both Nexus switches, this is then the principle of hashing: the device will hash to either one of it's link going to the Nexus switches. When a packet comes through the Nexus Switches going to this end device depending on which one received the packet it will send it on that link.

You can see the interface bw usage via the regular show commands.

Please take a look at the vPC section on this webcast:

https://supportforums.cisco.com/videos/1228

27270
Views
9
Helpful
69
Replies
CreatePlease to create content