cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
858
Views
5
Helpful
20
Replies

3750 Etherchannle load-balnace Issue

chintan-shah
Level 3
Level 3

Hi,

I have 3750 aggregation switches as aggregation layer to connect 2 access router to Core router. 3750 runs e etherchannel from to Core with 2 1 gig link

I have src-mac based loadbalanced method used in 3750 and what I see that 3750 use only one link part of ethrchannel so during test when i have total traffic comming more than 1gig , i see drops....

Here are details :

Mac address : 000d.edac.8900 - Access Router 1

0006.d61b3c1a - Access Router r2

0015.c75d.d42c- Core router

sw1.LAB-3750G#show etherchannel summary | beg Group

Group Port-channel Protocol Ports

------+-------------+-----------+-----------------------------------------------

1 Po1(SU) - Gi2/0/1(P) Gi2/0/4(P)

sw1.LAB-3750G#

sw1.LAB-3750G#test etherchannel load-balance interface port-channel 1 mac 000d.edac.8900 0015.c75d.d42c

Would select Gi2/0/1 of Po1

sw1.LAB-3750G# test etherchannel load-balance interface port-channel 1 mac 0006.d61b.3c1a 0015.c75d.d42c

Would select Gi2/0/1 of Po1

As per above test, I see that 3750 select Gi2/0/1 for two source mac-address and that's reason i have only one link utilized..

Does some body knows the algorithm used by 3750 for above two source-mac address and how did it select always gi2/0/1 not other link for 2nd stream of traffic ??

Unforutnately I can't do src-dst-ip based loadbalnace becuase incomming traffic is MPLS labelled and existing 3750 doens't have capbility to understand MPLS so it takes as non-ip traffic..

Thanks in advance,

Chintan

20 Replies 20

Giuseppe Larosa
Hall of Fame
Hall of Fame

Hello Chintan,

the algorithm is so simple as:

exor of last bit (righmost bit) of MAC SA and MAC DA

so you have in both cases :

0 exor 0 = 0 and always the same link chosen

In general:

n bits where 2^n is M the number of links if M is even or 2^n if M is odd with 2^n = 2 *(M+1)

n=3 if M=3

n=2 if M=4

even changing to source MAC only will not change the choice

you could try to modify the mac address in one of the two routers if you can

Hope to help

Giuseppe

Hi Giuseppe,

Thanks for alogirthm.

So what you suggest, since destination mac-address is always same ( CR router mac-address) ??

I think I can change one of the access router mac-address with last digit to 1 so that 3750 choose both link each for one stream...

But in practically, I may have 4 links part of etherchannle and dst-mac address will be always same and X-OR operation may fall in to using only either of 2 link right ?

Regards,

Chintan

Hi agian,

I have load balnaced method only src-based mac address so my question is still dst-mac address play role. why does switch do XOR operation than using right most n bits

sw1.LAB-3750G#show etherchannel load-balance

EtherChannel Load-Balancing Configuration:

src-mac

EtherChannel Load-Balancing Addresses Used Per-Protocol:

Non-IP: Source MAC address

IPv4: Source MAC address

IPv6: Source MAC address

sw1.LAB-3750G#

Regards,

Chintan

Hello Chintan,

in this case DA MAC address shouldn't play any role.

the rightmost n bits of the MAC SA says which link you are going to use

but if rightmost bits are equal the same link is chosen

Hope to help

Giuseppe

Hi Giuseppe,

Now I changed src-mac for one of the access router with rightmost bit 1 but i still see on link is being used.

ar1.LAB-7609#show int Ge-WAN 2/1 | in Hardware

Hardware is GigabitEthernet Interface, address is 000d.edac.8901 (bia 000d.edac.8900)

sar1.LAB-7609#

8901 - changed the mac-address but it looks like aggregation 3750 switch still using 8900 as src how do I make sure that packet from router goes with src-mac 8901.

sw1.LAB-3750G#show mac-address-table dynamic | in Gi2/0/2

51 000d.edac.8900 DYNAMIC Gi2/0/2

51 000d.edac.8901 DYNAMIC Gi2/0/2

sw1.LAB-3750G#

Regards,

Chintan

Hello Chintan,

after 300 seconds the old entry should be removed from CAM table on C3750

So now it shouldn't appear

Hope to help

Giuseppe

Hi Giuseppe,

After 300 secs old entry disapperas and i only see new mac-address but when I sent ping from router , i agin see old and new both mac-address and I feel 3750 still using old mac for loadbalance...

sw1.LAB-3750G#show mac-address-table | in Gi2/0/2

Mac Address Table

-------------------------------------------

Vlan Mac Address Type Ports

---- ----------- -------- -----

51 000d.edac.8901 DYNAMIC Gi2/0/2

### AFTER sending PING ofr 5 packets.

sw1.LAB-3750G#show mac-address-table | in Gi2/0/2

Mac Address Table

-------------------------------------------

Vlan Mac Address Type Ports

---- ----------- -------- -----

51 000d.edac.8900 DYNAMIC Gi2/0/2

51 000d.edac.8901 DYNAMIC Gi2/0/2

53 0015.c75d.d42c DYNAMIC Po1

Total Mac Addresses for this criterion: 13

sw1.LAB-3750G#

Any cluse ?

Regards,

Chintan

Hello Chintan,

you told you are using MPLS.

Verify on the router sending the ping requests if they are sent out inside mpls frames

do this with

sh mpls forwarding

The c3750 can only see MAC addresses as source in router's frames so the only possibility is that the router is using two different source addresses for different protocols (IP and MPLS for example).

Try to shut and then unshut the router interface to give it a chance to use the new MAC address in all cases.

Hope to help

Giuseppe

chintan-shah
Level 3
Level 3

Hello Giuseppe,

Yes, We are using MPLS.

I see that on router( access) it use new sourc-mac address for destination prefix/lable

sar1.LAB-7609#show mls cef adjacency mpls 16017

Index: 360448 smac: 000d.edac.8901, dmac: 0015.c75d.d42c

mtu: 1530, vlan: 4079, dindex: 0x0, l3rw_vld: 1

packets: 0, bytes: 0

Index: 491520 smac: 000d.edac.8901, dmac: 0015.c75d.d42c

mtu: 1530, vlan: 4079, dindex: 0x0, l3rw_vld: 1

packets: 0, bytes: 0

But some how 3750 still use 8900 old mac-address and it learns on mac-add table...

Regards,

Chintan

Hi Giesuppe,

I believer your theroy at some extent of using diff src-mac for diff protocol . In my case for IP and MPLS LSP.

I attach test results, let me know your thought...

Hello Chintan,

your results are interesting and confirms my guess: by the way it is a smart idea that of the MAC address filter on the C3750.

first of all IS-IS is neither IP and neither MPLS it has its own PDU over ethernet.

So here you have three protocols and the change of mac-address is effective for IP and IS-IS and not for MPLS.

The interesting part is that this is a 7609 very powerful and very complex where MLS and MPLS have to coexist.

So the commands that really count should be:

show mpls forwarding 192.85.12.1 detail

notice that already in another thread under service providers/MPLS was noticed that the two commands:

sh mpls forwarding

sh mls cef

were providing different outputs and the observed behaviour was like the one showed in sh mpls forwarding (it was an issue about load-balancing).

However, you have already tried to remove the mpls from the interface.

I would try, but only if you are in a lab setup, something more invasive:

the following options the second to be used if the first doesn't fix and so on.

a)try to shut down the ge-wan 2/1 and then re-enable it.

b) disable mpls at node level, then reenable it

and you like and you can

save config with mpls active and modified mac-address.

you can try to restart the module 2

c) #hw-module module 2 reset

d) even to reload the whole system

if none of the suggested actions solves you have hit a bug that could be stated as: MPLS code uses always BIA instead of current interface MAC address.

I see also you are using the old style commands tag instead of mpls

What IOS release is in your C7609 ?

You can try to look for known bugs with bug toolkit.

Let us know if you have been able to solve this issue.

Hope to help

Giuseppe

Hi Giuseppe,

I tried option C and d but didn't help. I already tried shut and not shut of GE-WAN2/1interface too.

I could not understand option b)

disable mpls at node level, then reenable it

and you like and you can

save config with mpls active and modified mac-address.

you can try to restart the module 2

IOS:

disk0:s72033-adventerprisek9_wan-mz.122-18.SXF12a.bin

s72033-pk9sv-mz-eng-sp-122-17d.SXB11A-v3

Regards,

Chintan

Hi Giuseppe,

My all test I did with 12.2(18)SXF12a.

I have upgraded 7600 with s72033-pk9sv-mz-eng-sp-122-17d.SXB11A-v3 ( we use in network).

It works fine...

sw1.LAB-3750G#show mac-address-table | in Gi2/0/2

51 000d.edac.8901 DYNAMIC Gi2/0/2

sw1.LAB-3750G#

Regards,

Chintan

Hi Giuseppe,

Also it seems that algorithem mentioned by you don't look correct.

I have 4 link

Dst mac address right most 2 digit : 10

I use source-mac address such way that each time i have right most 2 digit diff i.e 11,01,10,00 so i should see each time diff link but for 10 and 00 i see same link...

Group Port-channel Protocol Ports

------+-------------+-----------+-----------------------------------------------

1 Po1(SU) - Gi3/0/5(P) Gi3/0/6(P) Gi3/0/8(P)

Gi3/0/9(P)

sw1#

sw1#$d-balance interface port-channel 1 mac 0011.2133.773f 0019.30f1.6b06

Would select Gi3/0/8 of Po1

sw1#$d-balance interface port-channel 1 mac 0011.2133.7731 0019.30f1.6b06

Would select Gi3/0/5 of Po1

sw1#$d-balance interface port-channel 1 mac 0011.2133.7736 0019.30f1.6b06

Would select Gi3/0/6 of Po1

sw1#$d-balance interface port-channel 1 mac 0011.2133.7730 0019.30f1.6b06

Would select Gi3/0/6 of Po1 ===>I was expecting Gi3/0/9

Am I doing soemthing wrong ?

Regards,

Chintan

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco