Etherchannel bandwidth

yadavsandip · ‎02-22-2012

Hi,

if i configure two 1-gig interfaces for etherchannel, after bundling two interface does etherchannel interface become (1+1) 2 Gig in actual.

i heard Etherchannel interface becomes 2gig logically (as per this case) but in actual during data flow time it is unable to use full 2gig bandwidth.

What is true?

nkarpysh · ‎02-22-2012

Hi Sandeep,

Etherchannel can use up to 2 GIG. Though there are many things to consider to achieve that

1. those 2 Gigs will not be an average rate but indeed a pick rate which can also be lowered down by the platform Tx buffers

2. Based on platform Etherchannel may use different logic select the link out of which packet will be sent (load-balancing). It can be per destination/source/or both load-sharing or per-packet load-sharing.

So based on this load-sharing logic and your traffic pattern you can get situation when one link will be loaded more heavily thus you will not be able to reach 2 Gigs.

So in general Port-channel will use both links, but you need to consider traffic pattern, load-sharing logic for the platform/port-channel and some other aspects to make Po to use those links more efficient.

Nik

HTH,
Niko

hobbe · ‎02-23-2012

Hi

The total bandwith in a 2 gig etherchannel will be 2 gig.

However that said that does not mean that a session going over a 2 gig etherchannel will get 2 gig, it will not.

if the 2 gig etherchannel consists of two 1 gig connections, the max speed in any session will be 1 gig.

The Etherchannel is balanced due to a hashing algorithm that will place the packets either on line 1 or 2 of the etherchannel.

how is this determined then ? well in different switches there are different algorithms, but for the 3750/3560 it works something like this.

the algorithm for the etherchannel will dependant on how you configure it take into consideration.

Source IP, Destination IP, Source Mac, Destination Mac or a combination of them.

so if fx you have one server that everyone loads their data from then it might not be a good idea to have destination as a hashing mekanism. That would mean that all the connections to that server would go over a specific link and the other link would be empty.

If you on the other hand had two servers that is load balanced, then it might be a very good idea to do just that.

on the other hand it could be that the two servers due to the balancing mekanism would be set up to use the same link in the etherchannel.

you can test wich link a specific packet will take with the command

"test etherchannel load-balance interface port-channel x" and then play around with the ip address and mac address parameters of this command to understand wich of the links would handle wich packets in the configuration that is done today.

As a general point if you have done you homework well and you have loads of smaller sessions then it would balance this out so that you will have most of the bandwith for the 2 gig link usable.

but if you have large sessions with larg data transfers then it might be very unbalanced.

Good luck

HTH

Joseph W. Doherty · ‎02-23-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Short version (of what the other posters have posted):

Single flow will only use one link. I.e. single flow would not be able to obtain more than 1 link's bandwidth.

Multiple flows can use more than one link. Assuming random distribution (which is what you try to achieve), two links will provide about 50% more bandwidth. (For two flows, half the time, there will be one flow per link, half the time, two flows same link.)

nkarpysh · ‎02-23-2012

So imagine you have trffic coming on 10G interface and going out of your 2G Port-channel.

If there is a single flow coming from TE interface which is taking all those 10G then it will be limited just to 1 Gig on Port-channel due to per destination/source load-balancing algorythm. So in this case Po would be limited to 1 Gig.

If you have many diferent flows coming from 10G interface - there is a good chance of port-channel to redistribute those almost equaly between 1G links and the total utilization will be close to 2 G.

So again - all depends on the traffic flow and the load-sharing algorythm used based on paltform and config.

Hope this helps,

Nik

HTH,
Niko

Joseph W. Doherty · ‎02-24-2012

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

If you have many diferent flows coming from 10G interface - there is a good chance of port-channel to redistribute those almost equaly between 1G links and the total utilization will be close to 2 G.

Key item is "many different flows" leads to "good chance"; this also assumes a random like distribution.

The two important points to remember are single flow will not be able to obtain more than a single link's bandwidth and link selection is not influenced by existing path load. I.e. Etherchannel does not bundle bandwidth the same as MLPPP nor direct flow link selection based on current load as might OER/PfR.

For two flows, for example, there's a 75% probability that both links will be used, and a 25% probability the same link will be used. For three flows, there's a 87.2% probability that both links will be used, and a 12.5% probability the same link will be used.

My mention of 150% would be applicable to two flows, worst case of multiple flows. Nik is, I think, trying to emphasis with more flows average utilization should increase and approach 100%; he's correct. What I wanted to emphasis was with just a couple of, or few, flows average flow utilization can be much lower than an assumption you'll always see X times the number of links.

PS:

Also with most current Cisco hashing algorithm divisors, if you use a non-binary number of links, per link utilization will also be skewed. I.e. 3 links will have a problem providing 3x the bandwidth vs. 2 or 4 links providing up to 2x and 4x, respectively.

yadavsandip · ‎02-24-2012

Hi,

Thanks all of U.