Load balancing in link aggregation

Unanswered Question
Mar 5th, 2009

Is it possible to use load balancing per packet inside link aggregation ?

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (3 ratings)
Loading.
Jon Marshall Thu, 03/05/2009 - 12:46

Petru

When you say link aggregation are you referring to etherchannel ?

If so to my knowledge it is done per packet but the same src/dst pair will always use the same link within the channel because of the algorithm used.

Jon

badalam_nt Thu, 03/05/2009 - 13:31

What I'm interested to know is if an end-to-end communication between 2 PCs can benefit of the increased link capacity.

Ex: PC1 <-> SW1 <-> SW2 <-> PC2

Link PC1 - SW1 is 1Gb/s

Link PC2 - SW2 is 1Gb/s

2 100Mb/s links between SW1 <-> SW2, combined in etherchannel

And I fear that load balancing cannot be done per packet, but only per destination, so all packets between PC1 and PC2 will pass through the same physical link.

Hence for a single end-to-end communication, aggregation does not add any benefit.

Per packet would mean that for a same destination first packet will go on link1, second packet on link2 etc.

Jon Marshall Thu, 03/05/2009 - 13:46

Petru

You are correct in that communication between the same 2 PC's will use the same link within the etherchannel. So you don't get any additional throughput for this communication.

But it is still load-balancing per packet ie. the etherchannel does not have a concept of a flow in the same way a router would.

So each packet is analysed in isolation but the algorithm used will always choose the same link if the input if the same.

Jon

badalam_nt Fri, 03/06/2009 - 03:51

Thanks Jon.

Just one more question, as I miss the difference compared to router behavior (sorry for limited knowledge and weird questions - I'm in the learning process):

So we can't use Process switching here to load balance the packets between PC1 and PC2 on the 2 links from the etherchannel?

Where can I find the algorithm used in such a case with etherchannel link?

In a router we can load balance the packets from the same src-dst pair if we use CEF or Process Switching AND also use load balancing per packet method.

So that's why I was wondering why not possible to do the same in etherchannel case?

Jon Marshall Fri, 03/06/2009 - 03:59

Petru

"sorry for limited knowledge and weird questions - I'm in the learning process"

No need to apologize, NetPro is here for people to learn as well as fix problems.

With a router bear in mind that 2 links are seen as 2 separate links. This is not the case with etherchannel. Logically it is just one link. It may help rather than me going on to have a look at this document which explains how Cisco load-balance across etherchannel links. Have a read and then if you have further questions please come back -

http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a0080094714.shtml

Jon

badalam_nt Fri, 03/06/2009 - 08:29

To quote:

"EtherChannel frame distribution uses a Cisco-proprietary hashing algorithm. The algorithm is deterministic; if you use the same addresses and session information, you always hash to the same port in the channel."

I understand that I could have a TCP connection between PC1 and PC2 on link1 and another TCP connection between PC1 and PC2 on link2, if the result of the hashing algorithm will provide 2 different values.

Am I right?

Jon Marshall Fri, 03/06/2009 - 08:31

It depends on which load-balancing method you are using. Most switches support a combination of src/dst mac and src/dst IP address. These would not change between PC1 & PC2 so they would use the same link.

Jon

Giuseppe Larosa Fri, 03/06/2009 - 08:55

Hello Jon,

my understanding is that the load balancing is flow based:

what can change are the criteria to decide what link is used but given a flow it will use always the same path and this is not per frame load balancing

It is flow based for efficiency reasons: the switch needs a simple rule to decide out what link to send a frame/packet.

Hope to help

Giuseppe

Jon Marshall Fri, 03/06/2009 - 09:01

Giuseppe

From the doc i posted -

"The algorithm is deterministic; if you use the same addresses and session information, you always hash to the same port in the channel. This method prevents out-of-order packet delivery."

The above suggests that each packet is run through the algorithm and that it is then matched to a link within the etherchannel. So it analyses each packet in isolation but obviously with the same src/dst the result will be the same link to send it across.

But happy to be proved wrong. Do you have documentation that states it is flow based.

Jon

Giuseppe Larosa Fri, 03/06/2009 - 09:24

Hello Jon,

actually you mean that a detailed forwarding entry is not built but each frame is processed on the fly through the hash algorythm.

This is an important detail of the forwarding process in the anatomy of the switch that can imply a little additional latency when sending the frame out a bundle (I've never tried to test this and it should be difficult to measure).

This approach saves resources on TCAM tables.

From a user perspective it is still a "flow based" load balancing

I just wanted to note that talking of per packet load balancing could be misleading from a user traffic point of view.

Hope to help

Giuseppe

badalam_nt Fri, 03/06/2009 - 12:55

Jon, and if I'm taking into consideration TCP source and dst ports, then hashing could provide different values.

I agree with what you quote from that doc regarding MAC and IP addresses, but there there they didn't mention anything about ports, whereas in that Cisco doc the ports could be used for balancing algorithm.

adamclarkuk_2 Fri, 03/06/2009 - 13:10

Hi

If you look at the link I posted earlier you will see the command to select the algorithm you want :-

Issue the port-channel load-balance {src-mac | dst-mac | src-dst-mac | src-ip | dst-ip | src-dst-ip | src-port | dst-port | src-dst-port} global configuration command in order to configure load balancing

Jump down to the Cisco IOS section

hxxp://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a0080094...

badalam_nt Fri, 03/06/2009 - 13:21

Adam, thanks for that link.

It is actually based on that link that I draw the conclusion that the communication between PC1 and PC2 can be load balanced over the 2 links from the etherchannel.

But the messages from all you is that it is not like that. So I'm puzzled, wondering actually what do I miss?

As TCP ports are part of the algorithm I find normally that the communication between PC1 and PC2, supposing multiple TCP connections are open and active, could be load balanced between the 2 links.

Is there a flaw in my logic?

PS: And one out-of-topic question, saw it countless times on all forums. What is the reason the links are given with hxxp instead of normally with http ?

adamclarkuk_2 Fri, 03/06/2009 - 13:35

Answer to P.S.

The forum screws up the entry

If you are LB based on src or dst port then that would suggest that one HOST could use multiple links based on port numbers.

Have a play with this cool little command :-

test etherchannel load-balance interface port-channel number {ip | l4port | mac} [source_ip_add | source_mac_add | source_l4_port] [dest_ip_add | dest_mac_add | dest_l4_port] command.

P.S.

Don't forget to rate any posts you have found useful ;-)

badalam_nt Fri, 03/06/2009 - 13:41

Just popped-up a question: how a switch knows about Layer4 TCP ports ???

Is only 6500 series switches that know about L4 ports ? Or any other switch can do this?

PS: Yes, I'm always rating the posts, but normally at the end when I consider to stop the conversations. So it should come soon.

adamclarkuk_2 Fri, 03/06/2009 - 13:43

6500/3550/3560/3750/3750-e + others

MLS ( Multi layer switching ) gives these bad boys this ability.

[Correction]

I was talking about L4 in general, not specific the etherchannel, sorry :-s

P.S.

Don't forget to rate any posts you have found useful ;-)

Jon Marshall Fri, 03/06/2009 - 13:46

Adam

According to the link i posted earlier only the 6500/4500 platforms can load-balance on layer 4 information.

Jon

badalam_nt Fri, 03/06/2009 - 13:50

And to finalize the discussion: please could you confirm me black on white that a PC1 to PC2 communication (my example given before) can benefit from the etherchannel?

Some posts above the conclusion was that it cannot benefit at all.

Now I see that actually if using Layer4 ports for configuring the balancing algorithm the PC1 to PC2 end-to-end communication could double its bandwidth compared non-etherchannel situation with 1 link only.

Sorry, I need this confirmation clearly stated for gaining confidence.

Jon Marshall Fri, 03/06/2009 - 13:56

Adam

Think we both put a bit of effort in :-), and we both got rated for it.

Jon

Jon Marshall Fri, 03/06/2009 - 13:53

Petru

As i said if the switch support L4 load-balancing ie. the 4500 & 6500 then yes you could have multiple conversations among the same 2 PC's using different links in the etherchannel bundle as long as these conversations were on different ports.

If the switch only supports mac/IP then you will always use the same link no matter how many conversations.

Jon

Jon Marshall Fri, 03/06/2009 - 13:47

Petru

If the switch can load-balance on L4 info such as the 6500/4500 then yes you could actually have more than conversation between the same 2 PC's being sent over different links in the etherchannel.

Jon

Jon Marshall Fri, 03/06/2009 - 14:00

Giuseppe

"actually you mean that a detailed forwarding entry is not built but each frame is processed on the fly through the hash algorythm."

Exactly, which means it is not really flow based load-balacing at all ie. per packet load-balancing on a L3 device means that the device makes a forwarding decision based on each packet. This is exactly what a switch does with etherchannel. So to say it is flow based is i think a bit misleading.

The other problem with saying etherchannel is flow based is what constitutes a flow because different switches can use different criteria.

So we get the confusing state where a switch that can only load-balance on mac/IP will always use the same link for the same 2 endpoints but a switch that can load-balance on L4 ports can use multiple links within the etherchannel for the same 2 endpoints.

Jon

Actions

This Discussion