cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3002
Views
0
Helpful
21
Replies

vpc - nexus

Network Pro
Level 1
Level 1

Hi,

we have the following setup.

ISP  <------- Firewall <----------- Switch

                                          (Vlan100)

                                                  |

                                                  |

                                           Nexus 2----------- Nexus 1

                                                  |         (VPC)     |

                                                  |                       | 

                                                User Switch stack - 3 cisco 2950

                                                           (Vlan 101)

Initally we just had one connection from the switch to the nexus. I configured vpc, thereby giving a secondary connection to each nexus. both nexus are connected using vpc. the end user switch is a stack of 3 2950 cisco switches. for some reason only for a few people, the internet does not seem to work (maybe a switch in the stack) - but when you remove the secondary connection the one of the nexus the internet seems to work for everyone. any thought s?

21 Replies 21

nkarpysh
Cisco Employee
Cisco Employee

It seems that your WAN is connected to Nexus 2 only. Am I right? If so Nexus will need to pass traffic coming to Nexus 1 to VSL link and it then shoudl come bacj through it. In certain cases Nexus can drop traffic on VSL link due to loop avoidance mechanism. You can use peer-gateway feature on your setup. It will make Nexus 2 to send traffic to access on behalf on Nexus 1 not sending that through VSL link.

Nik

HTH,
Niko

Thanks Nikolay for the reply.

yes there is only one link from wan router connected to nexus 2. We have indivigual static routes on each nexus each pointing to the internet firewall. i understand that the edge switch can load balance (vpc) between the links to nexus  1 and nexus 2. so as per your note, if the traffic is passing from the edge switch to nexus 1 (say on load balancing), then nexus 1 should be able to route back to the firewall since it has a static route. (but the fact that few users can access the internet and rest cant makes me think what you say is possible as the few users can be connected to the switch in the stack that uses the nexus 2 and the users that cant connect can be using the nexus 1 route via vpc load balancing)

still if i am wrong and the only way to sort out this is using peer gateway, how do i go about configuring peer-gateway and it which router should it be connected ?

on a side note if i configure vpc to the wan switch (at the moment only switch is connected to nexus 2) will this problem be sorted ?


Thanks

Basically Peer-gateway is a feature which is configured with a single command under the VPC domain:

http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/5_x/nx-os/interfaces/configuration/guide/if_vPC.html#wp1812734

Though still not sure if that can help here. SO if we think of the traffic path I guess packet from 3750 coming e.g. to Nexus 1 in VLAN 101 first. Then Nexus 1 doing intervlan routing between VLAN 101 and 100 based on routing table and further switch traffic to Upper (on diagram) switch.

Interesting thing is how return traffic is coming back, Firewall shoudl send it to particular MAC of one of the Nexus switches within Vlan 100, depending on load-balancing (if present) or better routing. If there is a chance that FW sending return traffic to the hosts to the MAC of Nexus 1 than there is a chance we hit the problem which can be solved by peer-gateway. This feature is fixing the issue when Nexus2 receiving packet sent to Nexus 1 MAC which further should be sent through VPC. These packets can be dropped by VS link due to loop avoidance mechanism.

So hard to say without knowledge of the packet flow. So peer-gateway feature can be tried if your network allowing this change.

Nik

HTH,
Niko

Nikolay any thougts on the above?

also the user switch vlans are running hsrp.

I have been going through cisco website and about peer-gateway.

I think we are in a sitation of single homed device or orphan port - where the WAN switch is just connected to the nexus 2.

Also the edge switch have hsrp. If the packet from edge switch takes the nexus 2 path then it goes to the wan switch and the problem is sorted. Say a packet that comes from the edge switch take the nexus 1 path (port channel load balancing on edge switch) then nexus 1 cant route through the peer-link (as per the rule of vpc) and thus the packet gets dropped, isnt it ?

So if we had peer gateway on both nexus, will this problem be sorted ? will the edge switch send the packets to the nexus 2 as the the wan switch is connected to the nexus 2 ? (also i beleive we have to disable ip redirects during peer-gateway implementation ) - am i correct ?

I have attached the diagram of what our present setup is ?


Thanks

any thoughts on the above ? as this is in production network and affecting

i understand that the vpc peer link is used if the secondary connection to a switch fails and therby it uses the vpc peer link to get to that switch....what will happen if tehre is a device connected to nexus 1 and the edge switch routes packets to nexus 2 (due to port channel load balancing ) - will this go through the peer link ?

my network setup is similar in a way as the wan switch is connected the nexus 2 which is like connecting a device to a nexus ?

Of course this is affecting production traffic. Remember how HSRP works under vPC? - Both HSRP active and standby routers are forwarding traffic.

In order to privide outbound connectivity for traffic hitting Nexus-1 with vPC and HSRP, you will need a dedicated L3 link between Nexus-2 and Nexus-1. This way traffic hitting Nexus-1's SVI will be able to use the L3 link to send traffic toward the single uplink of your WAN switch.

HTH,

jerry

If access switch will send packet to Nexus 1 - it will for sure go through peer-link as the next hop MAC learnt from it. It will not yet dropped by peer-link as the packet is not leaving VPC. But the return packet coming from server to Nexus 2. If by any chance that will have destination MAC of Nexus 1 and destination ip of PC located behind VPC - then it will be dropped on peer-link.

With a peer-link this issue will be sorted as Nexus 2 receiving packet with destination MAC of Nexus 1 will not send that on peer-link but will send through VPC on behalf of Nexus 1. So this is smth to try.

Nik

HTH,
Niko

thanks both for your time however i am confused now with the concept of peer-link. when will the packets travel through the peer-link and when will it be dropped.

think in the above diagram i missed out the switch. I have attached a new diagram. the 2 nexus's and l3 wan switch are running ospf. the 2 nexus's are conencted using vpc. and the l3 wan switch is conncted to nexus 2 via l2 link. now how do i acheive the above. will peer-gateway problem fix this ? and what would be the solution to a L3 switch that is connected to a orphan port (here in my topology)

(also we need to vpc wan switch at a later stage after vpc the edge switch - what would be your recommandation ?) 

any help appreciated !

Thanks

Please take a look at the following link:

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html#wp1058482

basically, vPC peer-link is used to sync L2 information between both N7Ks. In terms of what is going through the peer-link, traffic destined to single attach host/switch (a.k.a. orphan device) will be allowed to go through the peer-link.

Like I said before, you are missing a L3 link between the N7Ks, please take a look at the below link which explain how HSRP works under vPC. In a very short sentance, both HSRP router (active and standby) will forward traffic outbound. If the traffic is on the Nexus-B, without the L3 link, traffic will be black holed.

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html#wp1045736

What do you mean by vPC WAN? Do you mean another L3 device running OSPF? If this is the case, you should use L3 Equal Cost MultiPath (ECMP). Please do not use vPC since running routing protocol over vPC is not supported.

Regards,

jerry

so jerry, you think that the traffic comes to nexus 1 (in my case - pls refer topology diagram) and gets blackholed, isnt it. but what i am thinking is since the traffic comes to nexus 1 and it sees the device (L3 switch) is connected to nexus 2 directly (a.k.a orphan port), wont it pass traffic through peer-link (peer-link cant drop the traffic as the traffic would flow thorugh a non vpc port - in my case l3 switch a.k,.a orphan port) - please correct me if i am missing a point in this ?

And i meant we need to vpc the l3 switch (wan switch) - i read on the cisco website, saying any l3 device needs l3 link to vpc and cant be vpc over l2 links - but this ist he second stage of my implemenataion and not worried about it too much at this time. i am just worried about the edge switch vpc as its affecting our production network.

Yes, it will be black holed (assuming your network 10.1.1.x is passive interface, and it should be passive). If you look at your config closely, when you turn on peer-gateway, it will turn on no ip redirect automatically. Traffic will not send over the Nexus-A if it hits Nexus-B (HSRP interaction with vPC) unless you have L3 between Nexus-A and B.

BTW, the L3 switch in Nexus-A is not consider orphan port, it is a L3 interface. When we said orphan, it is refers to L2 interfaces.

I don't think peer OSPF between the 10.1.1.x network is a supported design.

HTH,

jerry

hi jerry,

the orphan port you mentioned is a l2 port. (l3 wan switch connected to the nexus 2 (Nexus B) in my case is connected using a l2 link so it should be considered as a orphan port isnt it ? so in theory it should hit the WAN switch (even though it runs ospf) but is connected to the nexus using a l2 link

and edge switch is l2 as well and connecteed through l2 link

Your L3 switch has an IP address of 10.5.5.x/24. What is on the Nexus side, is that on the interface or SVI? If it is SVI, is that extended over peer-link to the other Nexus?

Regards,

jerry

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco