Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

Webcast- Automate Nexus9k

Community Member

MP-BGP , IS-IS, COOP

Hello,

Can someone explain to me what are doing those 3 technologies ?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Silver

The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Hi mgual57@gmail.com

 

Let me start with a visual picture.  Imagine a simple 2leaf/2spine topology with HostA attached to to Leaf1 and with HostB attached to to Leaf2

Leaf1 has a VTEP address of 10.0.1.101

Leaf2 has a VTEP address of 10.0.1.102

Spine1 has a VTEP address of 10.0.1.201

Spine2 has a VTEP address of 10.0.1.202

 

HostA has a MAC address of A and an IP address of 192.168.1.1 and is attached to port 1/5 on Leaf1

HostB has a MAC address of B and an IP address of 192.168.1.2 and is attached to port 1/6 on Leaf2

 

 

Enter IS-IS

The leaves and spines will exchange IS-IS routing updates with each other so that Leaf1 sees that it has two equally good paths to reach Leaf2, and Leaf2 sees that it has two equally good paths to reach Leaf1

For now, that's all we need to know about IS-IS - IS-IS is the routing protocol used by the VTEPs to learn how to reach the other VTEPs.

Now think about the hosts.

This is where COOP comes in.

When Leaf1 learns about HostA because say HostA sent an ARP request seeking the MAC address of 192.168.1.2 (which you know is HostB, but that's not relevant at the moment)

Leaf1 looks at that ARP request, and just like a normal switch, learns that MAC A is present on port 1/5.  But the leaf is a bit more clever than that, and looks INSIDE the payload of the ARP packet and learns that Host1 also has an IP address of 192.168.1.1 and records all this information in its Local Station Table

AND THEN reports this information to one of the spine switches (chosen at random) using the Council Of Oracles Protocol (COOP).  The spine switch that was chosen then relays this information to all the other spines so that every spine has a complete record of every end point in the system.

The spines record the information learned via the COOP in the Global Proxy Table, and this information is used to resolve unknown destination MAC/IP addresses when traffic is sent to the Proxy address.

Note that all of this happens without anything to do with BGP.

But to round off the COOP story, we would assume that at some stage Leaf2 will also learn HostB's MAC and IP and also inform one of the spines at random of this information using the COOP.

So COOP is used solely for the purpose of distributing endpoint information to Spine switches. As far as I know, Spine switches never use COOP to distribute end host information to leaf switches.

So where does BGP fit in?

BGP is not needed until an external router is connected.  So now imagine that Leaf2 has had a router connected and has learned some routes from that external router for a particular VRF for a particular Tenant. 

How can Leaf2 pass this information on to Leaf1 where HostA is trying to send packets to one of these external networks?  For Leaf2 to be able to pass routing information on to Leaf1 and keep that information exclusive to the same VRF, we need a routing protocol that is capable of exchanging routing information for multiple VRFs across an underlay network.

Which is exactly what MP-BGP was invented for - to carry routing information across MPLS underlay networks.  In the case of ACI, BGP is configured by choosing an Autonomous System number and nominating one of the spine switches to be a route reflector.  MP-BGP is self configuring, you don't need to do anything to make it work!

Hope this helps.

Reference: A Version of this post with illustrations and examples can be found here.

 

 

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

Everyone's tags (5)
10 REPLIES
Community Member

Hi @mgual57@gmail.com  

Hi [@mgual57@gmail.com]  

MP-BGP, when enabled by configuring an ASN and enabling your spines as route reflectors,  handles distributing the (host and external) routing table to the devices in the fabric (i.e leafs) and to external devices via L3outs.

Think of IS-IS as the IGP within the fabric that is building the routing tables.

Think of COOP (Council of Oracle Protocol) as the protocol that that helps to build the tables that identify endpoints...what helps build the "arp table" across the entire fabric, not just on a sigle device.

This is how I like to think of them...there is alot more "under the hood" but one of the nice things about ACI is that it does alot of that for you!

Community Member

Thanks you !So if I

Thanks you !

So if I understand well , IS-IS is used for routing between leaves and spines and VTEp , MP-BGP is used for routing to external devices but leaves learn Endpoint's mac and BGP send it to spines. BGP also store information about the IP address of VTEP. And finally COOP is used for mac discovery of the host?

Community Member

One more question : When is

One more question : When is COOP used exactly , cause MP-BGP can learn host mac address so I dont really understand the utility of COOP

Highlighted
Silver

The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

Hi mgual57@gmail.com

 

Let me start with a visual picture.  Imagine a simple 2leaf/2spine topology with HostA attached to to Leaf1 and with HostB attached to to Leaf2

Leaf1 has a VTEP address of 10.0.1.101

Leaf2 has a VTEP address of 10.0.1.102

Spine1 has a VTEP address of 10.0.1.201

Spine2 has a VTEP address of 10.0.1.202

 

HostA has a MAC address of A and an IP address of 192.168.1.1 and is attached to port 1/5 on Leaf1

HostB has a MAC address of B and an IP address of 192.168.1.2 and is attached to port 1/6 on Leaf2

 

 

Enter IS-IS

The leaves and spines will exchange IS-IS routing updates with each other so that Leaf1 sees that it has two equally good paths to reach Leaf2, and Leaf2 sees that it has two equally good paths to reach Leaf1

For now, that's all we need to know about IS-IS - IS-IS is the routing protocol used by the VTEPs to learn how to reach the other VTEPs.

Now think about the hosts.

This is where COOP comes in.

When Leaf1 learns about HostA because say HostA sent an ARP request seeking the MAC address of 192.168.1.2 (which you know is HostB, but that's not relevant at the moment)

Leaf1 looks at that ARP request, and just like a normal switch, learns that MAC A is present on port 1/5.  But the leaf is a bit more clever than that, and looks INSIDE the payload of the ARP packet and learns that Host1 also has an IP address of 192.168.1.1 and records all this information in its Local Station Table

AND THEN reports this information to one of the spine switches (chosen at random) using the Council Of Oracles Protocol (COOP).  The spine switch that was chosen then relays this information to all the other spines so that every spine has a complete record of every end point in the system.

The spines record the information learned via the COOP in the Global Proxy Table, and this information is used to resolve unknown destination MAC/IP addresses when traffic is sent to the Proxy address.

Note that all of this happens without anything to do with BGP.

But to round off the COOP story, we would assume that at some stage Leaf2 will also learn HostB's MAC and IP and also inform one of the spines at random of this information using the COOP.

So COOP is used solely for the purpose of distributing endpoint information to Spine switches. As far as I know, Spine switches never use COOP to distribute end host information to leaf switches.

So where does BGP fit in?

BGP is not needed until an external router is connected.  So now imagine that Leaf2 has had a router connected and has learned some routes from that external router for a particular VRF for a particular Tenant. 

How can Leaf2 pass this information on to Leaf1 where HostA is trying to send packets to one of these external networks?  For Leaf2 to be able to pass routing information on to Leaf1 and keep that information exclusive to the same VRF, we need a routing protocol that is capable of exchanging routing information for multiple VRFs across an underlay network.

Which is exactly what MP-BGP was invented for - to carry routing information across MPLS underlay networks.  In the case of ACI, BGP is configured by choosing an Autonomous System number and nominating one of the spine switches to be a route reflector.  MP-BGP is self configuring, you don't need to do anything to make it work!

Hope this helps.

Reference: A Version of this post with illustrations and examples can be found here.

 

 

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

Everyone's tags (5)
Community Member

Just one last question , I

Just one last question , I thought MP-BGP can be used to learn host mac address but it seems like I was wrong or I misunderstood your explaination?

Silver

Ahh - now you are getting

Ahh - now you are getting into the realm of MP-BGP EVPN (what a mouthful) which is used between pods in a multi-pod environment and between sites in a multi-site environment, but I don't have those details in my head, I'd have to read http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/guide-c07-734107.html as a starting point

So for now I'll leave this - perhaps you should ask this as a new question to attract other respondents who by know are skipping this thread.

CW

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

Cisco Employee

Re: Just one last question , I

Indeed your understanding is correct and this can be done using BGP as well but not in ACI

Silver

Re: Hi mgual57@gmail.com

This is a fantastic, clear, and super concise write up of the ACI fabric in a nutshell. Thank you for writing this.

Re: The roles of IS-IS, COOP, BGP and MP-BGP in Cisco ACI

As a teacher, I appreciate the clear, concise story you told to explain these concepts. If you are not a teacher, please consider it. You will improve the lives of many.

Community Member

Ok i got this , thanks you so

Ok i got this , thanks you so much. Very clear explaination 

4145
Views
71
Helpful
10
Replies
CreatePlease to create content