Nexus - VPC

Network Pro · ‎01-25-2012

Hello,

when you vpc two nexus (peer-link), then all the vlans gets tied into this, isnt it ? how would you form a routing adjacenty between the two nexus using a single vlan ?

this is cisco best pratice but how do you do this ? i have follwed the cisco website and created a vpc between two nexus but can understand what they mean by the below ?

It’s OK to use the vPC peer-link to form a routing adjacency between the two Nexus 7000′s. Use a VLAN dedicated to the routing adjacency and only forward this VLAN on the peer-link, not on the vPC member ports.

Thanks

Network Pro · ‎01-25-2012

any ideas pls ?

rsimoni · ‎01-25-2012

Hi,

you can have routing peering between nexus across vpc peer-link but the vlan your are peering for MUST be pruned by the vPC channels; i.e it must be active and forwarding on the vpc peer-link only and not existing on other vpc channels.

This is supported but the best practise would be to have a dedicated L3 link between the 2 nexus for routing peering.

What you should not do is to have another dedicated L2 non-vpc link (with peering on SVI's) between the nexus. Altough it is officially supported (or at least not proibithed) it can cause unexpected problems.

Riccardo

Network Pro · ‎01-25-2012

ok thanks for this, should the l3 link be routed (i.e advertised in ospf or any routing protocol) so that other devices can go through this link ?

rsimoni · ‎01-25-2012

no requirement in that sense. since it is a L3 routing vlan/link between the 2 nexus for many topologies there is no need to advertise it; but the fact whether you do it or not is not important

Network Pro · ‎01-25-2012

i tried vpc our end user switches and ran into a problem of few users accessing the internet and few cant. looks like the traffic was going through vpc and was dropped. i have attached the topplogy and the the config of the nexus and wan switch.

so i have referred this in a previous post also. Jeye (cisco employee) thinks that adding L3 link between both nexus would sort out the issue ? would htis be sufficicent or does a l3 link needed between wan switch and nexus 2 ? or anything other ideas ?

Thanks

rsimoni · ‎01-25-2012

well... this post was on something else... also if you ask for help try to be as specific as possible.

what is the point in attaching diagram with poor documentation and entire configurations without elobaroting what people should look at?

you are peering ospf between the wan switch and the 2 nexus box? on which subnet/vlan?

in general symptoms like yours sound like the typical problem of the vpc rule which can be solved by configuring vpc peer-gateway, but in your case i would not know as it is not clear what is not working and how your routing peering is exactly configured.

Network Pro · ‎01-26-2012

Ricarrdo, sorry for not being clear.

Both nexus have SVI and so as the WAN switch. The wan SW has a l2 connection back to nexus 2 (not ospf) . Nexus 1 and Nexus 2 are connected through vpc. (sh vpc - shows all vlans).

sh vpc

1 Po2 up 1,5,19,26-27,29-30,67,73-74,77,88,93-95,97,101,126

-127,129,193,197,200,202,205,210,212,300-307

config of the vpc of nexus 1

interface Ethernet10/8

description vpc peerlink member

switchport mode trunk

channel-group 2 mode active

no shutdown

same on Nexus 2

i have attached a modififed diagram with hsrp. please let me know if you need any further info.

Also i tried vpc on a non hsrp subnet but didnt work so i gues peer-gateway wouldnt fix this problem as peer-gateway fixes hsrp issues. think it could be the l3 link between the nexus. what would be your througts on thsi ?

rsimoni · ‎01-26-2012

what about routing between nexus 1 and wan switch?

how is it achieved?

Network Pro · ‎01-26-2012

this setup is quote strange as previous network engineer has set this up. but as far as i can see that there are are SVI on both nexus for all vlans and for some vlans there are SVI on WAN switch. so for the ones it has SVI it uses l2 to get to nexus 1. for the ones that nexus 1 does not have SVI then it uses ospf to get there.

so i would say mainly if wan switch wants to reach nexus 1 then it would go through nexus 2 vpc to nexus 1

rsimoni · ‎01-26-2012

well... you are in trouble then as I see that nexus 2 is connected at L2 with the wan switch on vlans carried in the vpc peer-link. So you will see drops for traffic crossing the vpc link from nexus1 to nexus2.

you need to make the link between the wan switch and nexus 2 a L3 link and enable routing on that link. Don't leave the port a L2 trunk peering on a SVI on the nexus. It is better if you make that interface a L3 port with an IP address assigned to it. Then you turn OSPF on and route traffic coming from the below switches to the wan switch.

Riccardo

Network Pro · ‎01-26-2012

Thanks for this.

a few queries regarding this

a. if i make a l3 routed link between the nexus 1 and nexus 2 will the above sort out the the problem with l2 connection to WAN switch (running ospf)

b. if i remove the unneeded SVI from both nexus and run only ospf for hte networks that are connected to both nexus and remove the un needed SVI from wan switch and run ospf for the netwroks that are connected to WAN switch - will this sort out the problem ?

c. so do i need a l3 connection between both nexus and l3 between nexus 2 and wan switch or just one ?

d. as per what you have said above, if i make the link between wan switch and nexus 2 a l3 link - do i need to advertise this in ospf ? if i advertise in ospf will this cause any routing loops (if i change from l2 to l3 link )

e. finally does the vpc need to carry all vlans or can i make it specific to a sperate vlan so that only this vlan carries all control information - in this way i can keep the exisitng setup and create a l3 link between both nexus - will this sort out the issue ?

just so many things going around on my head with this vpc

sorry for the trouble

Thanks

rsimoni · ‎01-26-2012

a. if i make a l3 routed link between the nexus 1 and nexus 2 will the above sort out the the problem with l2 connection to WAN switch (running ospf)

I don't think so, why would you do that?

b. if i remove the unneeded SVI from both nexus and run only ospf for hte networks that are connected to both nexus and remove the un needed SVI from wan switch and run ospf for the netwroks that are connected to WAN switch - will this sort out the problem ?

not sure what you mean by this.

c. so do i need a l3 connection between both nexus and l3 between nexus 2 and wan switch or just one ?

Ideally you want to have L3 peerings between wan switch and both Nexus (1 from switch to nexus 2 and 1 from switch to nexus 1) using dedicated L3 ports. Other ways are possible but this is the best option.

d. as per what you have said above, if i make the link between wan switch and nexus 2 a l3 link - do i need to advertise this in ospf ? if i advertise in ospf will this cause any routing loops (if i change from l2 to l3 link )

the point is to have a L3 ROUTING peering (ospf is the option you were referring to.. it is ok). so yes you need to enable a protocol.

e. finally does the vpc need to carry all vlans or can i make it specific to a sperate vlan so that only this vlan carries all control information - in this way i can keep the exisitng setup and create a l3 link between both nexus - will this sort out the issue ?

you don't carry vlan on l3 links...

Network Pro · ‎01-26-2012

let me put this way and clear my doubts

a. does the vpc need to carry all vlans or should it be limited just to 1 vlan (by allowing only 1 vlan on the vpc trunk connected between two nexus ) ?

b. if the above is true, can i have a layer 3 routed link between both nexus and a layer 2 link between just one nexus (nexus 2) and wan switch ( this is waht i am trying to achive as in this figure) - if this is possible then it would be easy for to implement as downtime needed would be minimal and do i remove the common SVI on the nexus and wan switch (for example : WAN switch would have an svi for int vlan 100 whereas vlan 100 actually sits on the nexus (users are connected to vlan 100 and advetised in ospf - but in the present setup there is a int vl 100 on wan switch and advertised in ospf and vl 100 in nexus and advertised in ospf - i understand this is a design flaw but hte previous engineer has setup this like this )

rsimoni · ‎01-26-2012

the scenario in the picture is allowed as long as the peering occurs on non-vpc vlans.

as you see there is a vpc peer-link which carries all the vlans of the various vpcs, and also another l2 link between the nexus carrying a NON-VPC vlan for the peering.