cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
68936
Views
56
Helpful
6
Comments
Matthew Blanshard
Cisco Employee
Cisco Employee

What is peer gateway?

Peer gateway is a feature which was developed to support network devices which use non-standard layer 2 packet forwarding in a vPC environment.  This was first discovered by NetApp using the fastpath feature but other devices have started using this method as well over the years. 

How exactly does this non-standard layer 2 forwarding work?

Here’s how a typical ping works from host to host in a vPC environment where the hosts are in different vlans:

Topology

topology.png

In this scenario when HostA pings HostB the packet flow is as follows:

HostA Echo Request (pre-routing)

1.png

Based upon port-channel load balancing it will hash to one N7k1 or N7k2.  For the purposes of this we will assume it hashes to N7k1.  N7k1 will then route the frame.  Here’s the frame HostB will receive:

HostA Echo Request (post-routing)

2.png

HostB then responds with the following:

HostB Echo Reply (pre-routing)

3.png

Again this will be subject to hashing.  For the purposes of this we will assume it hashes to N7k2.  N7k2 will route the frame and the frame HostA receives will look like this:

HostB Echo Reply (post-routing)

4.png

This is how it works with a properly behaving set of hosts and works as expected.

In the scenario where we need peer gateway the scenario looks like this:

HostA Echo Request (pre-routing)

1.png

Again we will assume that the packet hashes to N7k1.  N7k1 does the routing and HostB receives this:

HostA Echo Request (post-routing)

2.png

Now here’s where it gets different.  HostB replies with the following frame:

HostB Echo Reply (pre-routing)

5.png

As you can see HostB has just flipped the source and destination MAC address.  Again this frame is subject to port-channel hashing.  If it hashes to N7k1 then everything is great and no issues.  If it hashes to N7k2 then the packet has to cross the peer link, and a special bit is set and it can’t leave N7k1 on a any port that is a member of a vPC.  This functionality is how loops are prevented on a VPC.   

So how does Peer Gateway work?

What peer gateway does is allow the nexus switches to route frames which are destined to the mac address of their peer device.  In this way it works the same as HSRP in a vPC environment where both nexus switches forward the frames destined to either nexus’s physical mac addresses.  When enable it you will get output like this:

N7k-1# show mac address-table vlan 10

Legend:

        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC

        age - seconds since last seen,+ - primary entry using vPC Peer-Link

   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID

---------+-----------------+--------+---------+------+----+------------------

G 10       0000.0c07.ac0a    static       -       F    F  sup-eth1(R)

G 10       0024.986f.bac1    static       -       F    F  sup-eth1(R)

G 10       0024.986f.bac2    static       -       F    F  vPC Peer-Link(R)

Note the G flag even though the mac address is learned on the peer link.  This means it will be treated like the HSRP virtual addresses and this switch will forward packets destined to that mac.  The only exception is if a packet is destined to both the physical mac of the peer and the physical ip address.  Under that circumstance the packet will be tunneled across the peer link.

Should I enable peer gateway?

If you don’t have any devices which behave in this fashion then you should not enable peer gateway.  It should also be noted that using peer gateway to get around the limitation of no peering routing protocols across vPC’s is not an officially supported design and can result in performance issues.

Comments
cypherscuall
Level 1
Level 1

This is a really nice explanation about this option, but I still have one question, you said:

" If it hashes to N7k2 then the packet has to cross the peer link, and a special bit is set and it can’t leave N7k1 on a any port that is a member of a vPC"

Where is that bit at?

Thanks.

fcz88503636
Community Member
the G bit(Gateway bit)
sergeyvtb
Level 1
Level 1

Cisco should not set a bit but vpc number. If packet come from one vpc it should have possibility to go to another vpc.

sergeyvtb
Level 1
Level 1

Great article!

We have this problem with f5 big ip and netapp. After enabling peer-gateway problem solved.

Thank you!

shulipal
Cisco Employee
Cisco Employee

Hi Matt,

 

Really good explination. I have few questions on the same.


If the Packet initally reaches to N7K2 rather than N7K during the ping request, in that case traffic will be fowarded to N7K1 and it crosses to PL then it will set the VSL bit.
After the packet reaches the N7K1 will that packet will forwarded to destination? if yes I have couple of questions here,

1. During the ping reply the packet has to reach back to N7K1 ? if yes then again it will get discarded due to VPC loop avaoidance.

2. If the packet chooses interface connecting to N7K2 will that get dropped at the Ingress?


Thank You!

Daniel Castillo
Cisco Employee
Cisco Employee

There is a typo on ethernet header for HostA Echo Request (pre-routing)

I think the DST_MAC should be the Virtual MAC ending in ac:0a and not ac:14 as we read in the blue box.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: