Full Bi-Sectional Bandwidth

[visitor68]

All uplinks are forwarding because of vPC. Would one consider this architecture to offer "full bi-sectional bandwidth"? It seems that that term is interpreted in different ways. To me, it does because STP is not blocking any uplinks. So, if I were to part the network into two equal segments, all the bandwidth serving each segment is available.

HOWEVER, one may interpret "bi-sectional bandwidth" as being the bandwidth that connects two equal segments of the network at its worst case scenario. FROM WIKIPEDIA:

If the network is segmented into two equal parts, this is the bandwidth between the two parts. Typically, this refers to the worst-case segmentation, but being of  equal parts is critical to the definition, as it refers to an actual bisection of the network.

That bi-section between the two parts is the 40G crosslink, which imposes oversubscription. In fact, there is an 8:1 overubscription for traffic that traverses that crosslink. On the other hand, does oversubscription even impact the definition of "FULL bi-sectional bandiwdth" - or are the two things really isolated from each other? So, how do you folks see it?

[visitor68]

C'mon...where are all the heavy hitters on here?

[Leo Laohoo]

Bisectional Bandwidth. And Why L2MP and Trill/RBridges Is Important?

Cisco's interpretation of TRILL is FabricPath.

[visitor68]

I read that paper already....you gave me a non-answer...sorry

[nkarpysh]

Hi,

For me it will depend on typ of traffic. If the main traffic is the one travelling between the Rack servers than the bandwidth connecting the two segments is not the issue as each Nexus will be able to switch traffic by itself not sending it to other peer.

But in Case of e.g. L3 links (to the core - not in VPC) connected only to one of the Nexus switches, the other one will need to send traffic coming from servers to the peer-link which will be indeed oversubscribed.

Nik

[visitor68]

Nikolay, you make an excellent observation.

If the servers are non-ESX, then the MAC address of the active NIC (assuming active/standby teaming) will be present on both core switches because of vPC. In other words, both uplinks of the access switch will forward traffic from the server to each of the core switches. With that being the case, each switch will have knowledge of the server's MAC address and know out which downlink to send traffic destined for that server. Therefore, a core switch should never have to see the need to forward traffic across the crosslink to the other switch to reach any particular server.

Do you agree with that so far?

Now, what happens in an ESX environment? Imagine a particular VM's NIC (vNIC) pinned to one of the server uplinks to one of the access switches....again, because of vPC both core switches should have knowledge of that vNIC address - so, again, no need to go across the crosslink, right?

Nikolay, please share your thoughts.

Thank you

[nkarpysh · Accepted solution]

visitor68 написал(а):
Nikolay, you make an excellent observation.
If the servers are non-ESX, then the MAC address of the active NIC (assuming active/standby teaming) will be present on both core switches because of vPC. In other words, both uplinks of the access switch will forward traffic from the server to each of the core switches. With that being the case, each switch will have knowledge of the server's MAC address and know out which downlink to send traffic destined for that server. Therefore, a core switch should never have to see the need to forward traffic across the crosslink to the other switch to reach any particular server.

Correct

Now, what happens in an ESX environment? Imagine a particular VM's NIC (vNIC) pinned to one of the server uplinks to one of the access switches....again, because of vPC both core switches should have knowledge of that vNIC address - so, again, no need to go across the crosslink, right?

Yeap VPC peers synch their MAC tables. Even if single flow is coming only on one leg - the other side knows about that MAC and can send traffic directed to tha MAC by itself without using the peer-link. For L2  traffic is all links are up we use peer-link only for control traffic (STP, HSRP, etc). But if smth breaks - e.g. one link failed - then there is a chance of the traffic to flow through peer-link. E.G. traffic received on working two-legs VPC on Nexus 1 and should go out VPC2 which has only one leg (orphan port) on the Nexus 2 - in this case it will flow through peer-link.

Nik

[visitor68]

Thank you, Nikolay!

[nkarpysh]

Nop Probs, Glad if it helped.

Nik

[visitor68]

Nik:

Let me throw this one at you....

new scenario...

Imagine I have two Nexus 7Ks connected in a vPC, BUT this time there are no access switches. Instead, the servers are dual homed directly to the core. The servers are running LACP NIC teaming because vPC enables it. So, it would be the same scenario here, correct? All servers will have active/active connections to BOTH core switches and so each switch will learn the MAC addresses of every server - moreover, vPC synchronizes the MAC address tables anyway. So, again, there is NO need to traverse the crosslink, correct?