EtherChannel configuration between two switches

VADIM ZHUKOV · ‎06-28-2010

I want to increase bandwidth on trunk between two (for now) switches (specifically 3750 and 6500). OK, I configure Etherchannel between them on two ports making bandwidth up to 2 GB. Now, to finish configuration I need to specify load-balancing method on both ends. Here I have three questions:

1. Since these are both switches (not switch and server) and traffic goes in either direction what load-balancing method do I use? Source and Destination, both? How would Source\Destination work? Would it load balance at all in common case?

2. Assuming I have one core switch with most of my servers there and another one is access switch mostly with workstations I guess in such a case I can configure access switch with load-balancing by source so traffic to one server from multiple workstations will be balanced between two etherchannel members based on sourc IP or MAC. But then what do I configure on Core switch - load balancing by destination (that is workstations) or also by source (would not that be then the server?)? Simply put should the the same command (say 'port-channel load-balance src-mac') be issued on both sides of the channel or if on Switch1 I issued 'port-channel load-balance src-mac' command I need to match it on another switch with 'port-channel load-balnce dst-mac'?

3. Since normally one MAC corresponds to particular IP whats the difference of configuring load balancing based on IP or on MAC?

I think thats somewhat confusing matter that I could not find satisfying explanation about anywhere in CISCO documentation. Any info on that would be very appreciated. Need to get this done, got the commands but cant quite understand the consequences.

Thanks

Ganesh Hariharan · ‎06-29-2010

I want to increase bandwidth on
trunk between two (for now) switches (specifically 3750 and 6500). OK,
I configure Etherchannel between them on two ports making bandwidth up
to 2 GB. Now, to finish configuration I need to specify load-balancing
method on both ends. Here I have three questions:

1.
Since these are both switches (not switch and server) and traffic goes
in either direction what load-balancing method do I use? Source and
Destination, both? How would Source\Destination work? Would it load
balance at all in common case?

2.
Assuming I have one core switch with most of my servers there and
another one is access switch mostly with workstations I guess in such a
case I can configure access switch with load-balancing by source so
traffic to one server from multiple workstations will be balanced
between two etherchannel members based on sourc IP or MAC. But then
what do I configure on Core switch - load balancing by destination
(that is workstations) or also by source (would not that be then the
server?)? Simply put should the the same command (say 'port-channel
load-balance src-mac') be issued on both sides of the channel or if on
Switch1 I issued 'port-channel load-balance src-mac' command I need to
match it on another switch with 'port-channel load-balnce dst-mac'?

3. Since normally one MAC corresponds to particular IP whats the difference of configuring load balancing based on IP or on MAC?

I
think thats somewhat confusing matter that I could not find satisfying
explanation about anywhere in CISCO documentation. Any info on that
would be very appreciated. Need to get this done, got the commands but
cant quite understand the consequences.

Thanks

Hi,

Let us discuss with simple example and the packet flow to understand the actual load balancing method to be deployed in network

PC1 --- R1---SW1---------SW2---R2---PC2

Firstly we need to briefly consider how packets are forwarded from LAN1 to LAN2

PC1: 192.168.1.2 in LAN1 is pinging PC2: 192.168.2.2 in LAN2

1) PC1 recognize that PC2 is in a different subnet (packets need to be sent to the gateway) so it creates a packet with the following information:

Src IP: PC1 IP address – Dst IP: PC3 IP address – Src MAC: PC1 MAC – Dst MAC: R1(GW) .

2) R1 receives the packet finds a match in its routing table (dst: 192.168.2.2) pointing to the next hop (R2) and forwards the packet with the following information:

Src address: PC1 IP – Dst address: PC3 IP – Src MAC: R1 MAC – Dst MAC: R2 Mac.

3) R2 receives the packet looks up for the destination in its routing table and forwards the packet to PC3 using the following infromation:

Src address: PC1 IP – Dst address: PC3 IP – Src MAC: R2 MAC – Dst MAC: PC3 Mac.

The part that matters us here is the forwarding from R1 to R2 passing through the Etherchannel link between switch1 and switch2; the frame always has R1 and R2 MAC addresses as source and destination depending on the direction of the traffic.

Following are the different types of EtherChannel Load balancing methods available in switches:-

dst-ip—     Load distribution is based on the destination-host IP address.
dst-mac—    Load distribution based on the destination-host MAC address of the incoming packet.
src-dst-ip— Load distribution is based on the source-and-destination host-IP address.
src-dst-mac—Load distribution is based on the source-and-destination host-MAC address.
src-ip—     Load distribution is based on the source-host IP address.
src-mac—    Load distribution is based on the source-MAC address of the incoming packet.

In our case its clear that packets sourced from LAN1 destined to LAN2 will have source IPs of PCs belonging to LAN1 and destinations of PCs belonging to LAN2 but will always have R1 MAC address as a source MAC address and R2 as a destination MAC address when passing the switching section of our topology.

This analysis means that load distribution methods based MAC addresses will not provide the best distribution of traffic over the Etherchannel.

We need to consider load distribution based on IP addresses, but which one really gives us the best even distribution over the Ehterchannel links?

we need to consider the traffic pattern between the two LANs; lets see the scenarios below:

Scenario1: Traffic is random from any workstation to another which means that any user from LAN1 may be communicating with any user from LAN2. Any IP based load distribution method will do the work.

Scenario2: If a server exist (i.e. mail server) in any of the two subnets and many PCs are always sending or receving traffic to or from this server then we are left with one option that provides the most evenly distribution which is the src-dst-ip—Load distribution.

Hope to Help !!

Ganesh.H

Remember to rate the helpful post

VADIM ZHUKOV · ‎06-29-2010

Thanks, Ganesh, very good explanation, that helps. I do see now that if in your example I used LB by MAC, either source or destination, I would not have LB at all - the traffic would go only through one port.

Still not sure though about commands entered on both sides of the etherchannel for LB.

Also, still not clear how it would work for LB by both destination and source - when packets are coming from some three IP to the Etherchannel configured for both dest and src and they are destined to two servers how are they going to be handled? If balanced properly then should not we always configure LB for both src and dst? Why would it be a problem (at least then no need to think what configure on both sides of the Etherchannel - just do the same).

Ganesh Hariharan · ‎06-30-2010

Thanks, Ganesh, very good explanation, that helps. I do see now that if in your example I used LB by MAC, either source or destination, I would not have LB at all - the traffic would go only through one port.

Still not sure though about commands entered on both sides of the etherchannel for LB.

Also, still not clear how it would work for LB by both destination and source - when packets are coming from some three IP to the Etherchannel configured for both dest and src and they are destined to two servers how are they going to be handled? If balanced properly then should not we always configure LB for both src and dst? Why would it be a problem (at least then no need to think what configure on both sides of the Etherchannel - just do the same).

In order to configure load balancing for ethere channel in switches just configure with the below mentioned steps:-

Router# configure terminal
Router(config)# port-channel load-balance src-dst-ip

Check out the below link on etherchannel load balancing

http://www.testequipmentconnection.com/specs/Cisco_WS-X6548V-GE-TX.PDF

Hope to Help !!

Ganesh.H

Remember to rate the helpful post

bmcginn · ‎06-29-2010

Hi there,

This document may help.

http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a0080094714.shtml

1. Since these are both switches (not switch and server) and traffic goes in either direction what load-balancing method do I use? Source and Destination, both? How would Source\Destination work? Would it load balance at all in common case?

I would use source and destination IP. However I don't know your network. You would be in a better place to figure that out. If you have lots of different IP address talking to several servers, then load balancing using the source and destination IP will be ok probably. Another thing to think about is if there are only a few IP addresses talking to your servers then the hashing algorithm may put them all down the same physical link.

2. Assuming I have one core switch with most of my servers there and another one is access switch mostly with workstations I guess in such a case I can configure access switch with load-balancing by source so traffic to one server from multiple workstations will be balanced between two etherchannel members based on sourc IP or MAC. But then what do I configure on Core switch - load balancing by destination (that is workstations) or also by source (would not that be then the server?)? Simply put should the the same command (say 'port-channel load-balance src-mac') be issued on both sides of the channel or if on Switch1 I issued 'port-channel load-balance src-mac' command I need to match it on another switch with 'port-channel load-balnce dst-mac'?

I'm not sure about that, sorry can't help much there. I think that document may help to understand it though.

3. Since normally one MAC corresponds to particular IP whats the difference of configuring load balancing based on IP or on MAC?

The MAC will correspond to the layer 2 device that the transmission is coming from or going to.. the single broadcast domain specifically. For example, say 200 PCs send traffic to another network; the router's MAC will be seen on that other network as the MAC for the 200 PCs from the sending network. Load balancing on IP can offer greater distribution accross the physical links.

Hope that helps a little, let me know how you go.

Brad

VADIM ZHUKOV · ‎06-29-2010

Thanks, bmcqinn,

I looked up the article and it does explain the main concept of Etherchanel but as often the case the more clear points and those not too important are explained in depth while the more 'in debth' points are not explained at all.

I think I got the main idea and mostly have an answer to questions '1' and '3' but the question '2' is still open.

I guess I will go with LB by both src and dstn on both ends, this way it certainly would work.

But still what if I need for some reason do only LB by IP source (with workstations coming as sources to core switch with servers) - so access switch will have LB by source and what will core switch has? I dont care much for LB from server side - I guess they will respond back on the same individual port from which workstation calls came from so will be balanced anyway (or would they?) but I still need to enter LB command on core switch and should it be same as on access switch? Then it will also be balanced by source IP (servers?). Or it should be LB by destination (opposite to what is on access switch)? Or it should not really depend on whatever is on the other side as long as there is something configured?