Nexus 5k with L3

[elauriault-ubisoft]

Hi,

Could you explain the following paragraph located at the following URL :

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/operations/interfaces/602_n1_1/N5K_Interfaces_Ops_Guide/L3_w_vpc_5500platform.html#wp1014116

"In some circumstances, you might consider having a separate link between the two vPC switches, either to carry non-vPC VLAN traffic or to form Layer 3 routing protocol peering. *While this design is supported on the Cisco Nexus 7000 Series switch, it does not work on the Cisco Nexus 5000 Series switch*. With the Cisco Nexus 5000 Series switch, we recommend that you use a vPC peer link for Layer 3 peering to carry both vPC and non-vPC VLAN traffic."

I have deployments of Nexus 5k with L3 cards that have separate links For L3 and non vpc vlans instead of having them on the VPC peer link and they seem to work fine. Is this an error in  the documentation or could I run into problems?

Thanks,

Eric Lauriault, CCIE 27521

[klei]

Hi Eric,

  On N5K, the design is not tested and it will have issue with layer 3 multicast.  That's why it is not a supported design on Nexus 5K.

Thanks,

-KL

[elauriault-ubisoft]

Hi Klei,

What types of problems can we expect with multicast? I already have pim-sm running on the same interfaces I have ospf adjacencies between the 2 switches. For the moment, we haven't seen anything odd but maybe we're not looking in the right place.

Thanks,

Eric Lauriault, CCIE 27521

[klei]

Hi Eric,

  Potential of loops with multicast traffic.

Thanks,

-KL

[elauriault-ubisoft]

Hi Kley,

Are those loops on the L3 links or the peer link? As far as I know, on the peer link, once a packet comming from a vpc crosses it, it cannot go out another vpc on the other swith.

I would appreciate diagrams and technical details to reproduce this potential problem.

Regards,

Eric Lauriault, CCIE 27521

[klei · Accepted solution]

Hi Eric,

  In the URL that you have indicated http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/operations/n5k_L3_w_vpc_5500platform.html

Take a look at Figure 3-10, and now add another layer 3 link between N5K-1 and N5K-2.

1. Multicast is send over to N5K-1

2. Multicast routing on N5K-1.  One copy is send to L3 interface and other copy is send over to peer-link in reserved vlan (in the vpc bin-vrf default vlan xxxx).

3. The multicast packet recieved on the Layer 3 interface will get replicate to peer-link again.

Thanks,

-KL

[elauriault-ubisoft]

Hi Klei,

Thanks for you answer.

If I understand correctly, keeping L2 and L3 on the peer link would be beneficial for multicast as it would force the use of the peer link's loop prevention mechanism. Since the peer link's multicast replication behaviour prevents the L3 node from doing an RPF check as the traffic appears local, If we had pim-sm enabled on a L3 link outside of the peer link, we would encounter loops that would last as long as TTL > 0.

1) Can you confirm that this potential issue goes away by disabling pim-sm on the L3 link outside the peer link?

2) Why is this design supported with N7K? Do they perform additional verification to prevent loops?

Thanks,

Eric Lauriault, CCIE 27521

[klei]

Hi Eric,

     1. Yes

     2. Implemenation differeence between different platforms.

Thanks,

-KL

[elauriault-ubisoft]

Hi Klei,

Could you give me more information as to how does the implementation of the 7K platform perform loop prevention in this scenario?

Thanks,

Eric

[klei]

Hi Eric,

  Please contact your Cisco sale person to arrange a meeting with the right contact about N7K. I am not the right person for N7K.

Thanks,

-KL

[elauriault-ubisoft]

Hi Klein,

Thank you for your answers.

Eric Lauriault, CCIE 27521