02-20-2012 04:43 AM - edited 03-01-2019 07:04 AM
Hi,
I would like to make a design with 4 Nexus 5596UP. 2 of them equipped with Layer 3 Expansion Module so they can serve as core layer and the other 2 Nexus used as Layer 2 for aggregation server layer.
What I am ensure of is how the vpc and port-channel configuration should look like between the 4 nexus. What I was thinking is to run vpc between the 2 Nexus in the aggregation layer and between the 2 Nexus in the core layer. Than I was thinking of connecting each Nexus in the aggragtion layer to both Nexus in the core layer using port-channel and vice-versa.
I have made a drawing og the design I have described above:
Do you have any recommendations/comments or improments to this design?
Regards,
Laurent
02-29-2012 06:15 AM
Hi Laurent,
1- Place all of your links between core and aggregation into one port-channel (double side vPC).
2-Make sure the Domain id's are unique on core and agg switches.
2- Attach your FEX only to one agg switch, because if you daul homed them to both Agg switches , FCoE will not work ( its sees as exta hop).
4- Daul homed your servers instead to both FEX's.
Regards.
03-05-2012 10:22 AM
Hi Mohammed,
Thanks a lot for your reply.
Regarding point 1 do you have an example on how the configuration looks like with double side vPC apply to my topology?
Regards,
Laurent
03-05-2012 03:37 PM
Hi Lap
Configuration is much simple in that case:
For example on n5k1 and 2 create portchannel 10 and assign all links from 1 and 2 to 3 and 4 to it, and configure vpc 10.
On n5k 3 and 4 create portchannel 20 and assign all links from 3 and 4 to 1 and 2 to it and configure vpc 20.
on n5k1 and 2
on n5k1
interface port-channel 10
vpc 10
interface ethernet 1/1
description -=link to n5k-3=-
channel-group 10 mode active
interface ethernet 1/2
description -=link to n5k-4=-
channel-group 10 mode active
on n5k2
interface port-channel 10
vpc 10
interface ethernet 1/1
description -=link to n5k-2=-
channel-group 10 mode active
interface ethernet 1/2
description -=link to n5k-3=-
channel-group 10 mode active
on n5k-3
interface port-channel 20
vpc 20
interface ethernet 1/1
description -=link to n5k-1=-
channel-group 20 mode active
interface ethernet 1/2
description -=link to n5k-2=-
channel-group 20 mode active
on n5k-3
interface port-channel 20
vpc 20
interface ethernet 1/1
description -=link to n5k-1=-
channel-group 20 mode active
interface ethernet 1/2
description -=link to n5k-2=-
channel-group 20 mode active
HTH,
Alex
03-08-2012 01:24 PM
Hi Alex,
thanks a lot. I will make a try and let your know.
Regards,
Laurent
03-12-2012 07:36 AM
Hi Alex and Mohammed,
I have made a drawing of my setup:
So I have the following:
- VPC peer link: 2x10G between Nk5-1 and Nk5-2 in vPC domain id 12
- VPC peer link: 2x10G between Nk5-3 and Nk5-4 in vPC domain id 34
- 1x10G between Nk5-1 and Nk5-3 and 1x10G between Nk5-1 and Nk5-4 bundle in port-channel 1234 vPC 1234
- 1x0G between Nk5-2 and Nk5-3 and 1x10G between Nk5-2 and Nk5-4 bundle in port-channel 3412 vPC 3412
1) Is that a correct setup for double side vPC?
2) Would it make sense to add more 10G link between Nk5-1 and Nk5-3 and Nk5-1 and Nk5-4 in port-channel 1234 and
between Nk5-2 and Nk5-3 and between Nk5-2 and Nk5-4 in port-channel 3412? I am asking that because I have 4 extra SFP-H10GB-CUM cable.
Best regards,
Laurent
03-12-2012 08:01 AM
Hi Laurent,
wrote,
1x10G between Nk5-1 and Nk5-3 and 1x10G between Nk5-1 and Nk5-4 bundle in port-channel 1234 vPC 1234
- 1x0G between Nk5-2 and Nk5-3 and 1x10G between Nk5-2 and Nk5-4 bundle in port-channel 3412 vPC 3412
1-These all links will be in one port -channel not two(1234 and 3412). so, crate po1234 on 5k2 and 5k4 as well and place all 4 links into po1234.
2-Yes , it will be good to have 20 gig between instead of 10g if you have spare ports.
Regards,
Mo.
pls rate this.
03-12-2012 08:14 AM
additionally for 1), you can call the 'vpc' same number vpc 1234 on both so it's clearer in your mind, when you will add more vpc's later this will help you.
When you look into this type of design, the peer-link capacity must be equal or greater than the aggregate of the uplinks, so if you add more uplinks, remember to add more peer-links. Often we recommend > 20G capacity.
The peer-link is used as essentially backplane extension and when we allow it to be configured using a small BW profile we can have problems in these cases (e.g. throttling multicast traffic to certain ports down from the 40G arriving from the core to 2G available to the servers.) In the right design these issues are minimized but in other cases they will impact deployments and therefore it's important to understand when you use a peer-link < uplink capacity it 'will' result in impaired network performance for some common design cases.
03-12-2012 08:33 AM
Hi Lucien and Mohammed,
Thanks a lot for your great advices.
Just to make sure I have it all right.
- vPC peer link right now has 20 G capacity if you look at my previous post
1) When I bundle all the ports between N5k in port-channel 1234 vPC 1234 that would represent 40 G (4*10G) between N5k as the 4 links will be in port-channel no?
2) Mohammed, you mean to add 4*10 G between N5k so the port-channel 1234 would represent 80 G (8*10G) is that correct? Then Lucien how many link I should add to the peer-link ?
Best regards,
Laurent
03-12-2012 08:40 AM
Yes your Peer-Link is 20 and you have 40GE uplinks, so you need a bigger
peer-link: 40GE or more to be safe for all possible deployments use
cases (and stick to our best practices), but in any case at least 30GE,
so if you 2x10GE fail on one side they can take the whole peer-link
20GE and you will have some additional control traffic on the peer-link,
hence needing another 10GE.
If you run some multicast you may need another one.
Remember the rule of growing your peer-link when you grow your uplinks
(not downlinks).
03-13-2012 01:07 AM
Hi Lucien,
As a picture speaks a thousand words I attach the final picture of the design taking in account your advices:
Please let me know if it´s correct also regarding vPC doamain id.
Thanks.
Best regards,
Laurent
03-13-2012 03:42 AM
I agree, some confusion still on the picture, so trying to upload mine to show you :-) Please note that the size of the peer-link should be 4 ( green links ) - I had drawn 2 there for space constraints.
03-13-2012 08:18 AM
Hi Lucien,
Thank you very much for taking time to help. Now I got it.
I am planing to run HSRP for different VLANs between N5K-1 and N5K-2. Then N5K-1 and N5K-2 will iBGP peer to the local ISP in order to get MPLS prefixes from the diferent locations. Is there any limitation best practice regarding HSRP and BGP on Nexus 5596UP for this design setup?
Ports used for iBGP peering will be configured as routed port instead of VLAN interface on both Nexus. It is ok?
Regards,
Laurent
01-02-2018 10:58 AM
To use this network as a 3 tier design how would you run a routing protocol between the Nexus's?
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: