Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

LACP Link Aggregation not using full bandwith

Hi all!

I have configured a LACP port-channel on my Cisco 4506 on one side and a Win 2k8R2 Fujitsu Server (Intel NIC) on the other. I have three interfaces on the server that join the etherchannel without any problems. Looking at the sh int po 9, I get a strange output:

HF-SW-01-02#sh int port-channel 9

Port-channel9 is up, line protocol is up (connected)

  Hardware is EtherChannel, address is 68ef.bdaa.78d8 (bia 68ef.bdaa.78d8)

  Description: BackUp Server 1

  MTU 1500 bytes, BW 3000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is N/A

  input flow-control is on, output flow-control is unsupported

  Members in this channel: Gi3/41 Gi3/42 Gi3/43

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input never, output never, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 3000 bits/sec, 6 packets/sec

     23017370 packets input, 34014684782 bytes, 0 no buffer

     Received 3140 broadcasts (1265 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 input packets with dribble condition detected

     6365464 packets output, 2159208832 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier

     0 output buffer failures, 0 output buffers swapped out

The etherchannel consists of three 1GBit/s interfaces, but it shows only 1000Mb/s in the sh int command. When copying files to two seperate servers on the same switch, I only ge an accumulated bandwith of 1Gb/s, while I would excpect 1Gb/s per Server (I tried copying betrween the servers and the got about 1Gb/s in each direction).

Here the relevant config sniplets:

interface Port-channel9

description BackUp Server 1

switchport

switchport access vlan 15

switchport mode access

bandwidth 3000000

end

interface GigabitEthernet3/41

description 4-1-1 - hf-srv-bu-001 Team CS 1 - 10.1.15.18

switchport access vlan 15

switchport mode access

channel-protocol lacp

channel-group 9 mode active

spanning-tree portfast

spanning-tree bpduguard enable

end

!

interface GigabitEthernet3/42

description 4-1-2 - hf-srv-bu-001 Team CS 1 - 10.1.15.18

switchport access vlan 15

switchport mode access

channel-protocol lacp

channel-group 9 mode active

spanning-tree portfast

spanning-tree bpduguard enable

end

!

interface GigabitEthernet3/43

description 4-1-3 - hf-srv-bu-001 Team CS 1 - 10.1.15.18

switchport access vlan 15

switchport mode access

channel-protocol lacp

channel-group 9 mode active

spanning-tree portfast

spanning-tree bpduguard enable

end

HF-SW-01-02#sh etherchannel 9 summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 7
Number of aggregators:           7

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
9      Po9(SU)         LACP      Gi3/41(P)   Gi3/42(P)   Gi3/43(P)


HF-SW-01-02#sh etherchannel 9 detail
Group state = L2
Ports: 3   Maxports = 8
Port-channels: 1 Max Port-channels = 1
Protocol:   LACP
Minimum Links: 0
                Ports in the group:
                -------------------
Port: Gi3/41
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 9           Mode = Active          Gcchange = -
Port-channel  = Po9         GC   =   -             Pseudo port-channel = Po9
Port index    = 0           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi3/41    SA      bndl      32768         0x9       0x9     0x6F        0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi3/41    SA      255       0019.99ca.a2b7   4s    0x0    0x11   0xF     0x3D

Age of the port in the current state: 0d:00h:22m:10s

Port: Gi3/42
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 9           Mode = Active          Gcchange = -
Port-channel  = Po9         GC   =   -             Pseudo port-channel = Po9
Port index    = 1           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi3/42    SA      bndl      32768         0x9       0x9     0x70        0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi3/42    SA      255       0019.99ca.a2b7   7s    0x0    0x11   0xD     0x3D

Age of the port in the current state: 0d:00h:22m:09s

Port: Gi3/43
------------

Port state    = Up Mstr Assoc In-Bndl
Channel group = 9           Mode = Active          Gcchange = -
Port-channel  = Po9         GC   =   -             Pseudo port-channel = Po9
Port index    = 2           Load = 0x00            Protocol =   LACP

Flags:  S - Device is sending Slow LACPDUs   F - Device is sending fast LACPDUs.
        A - Device is in active mode.        P - Device is in passive mode.

Local information:
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi3/43    SA      bndl      32768         0x9       0x9     0x71        0x3D

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi3/43    SA      255       0019.99ca.a2b7   7s    0x0    0x11   0xE     0x3D

Age of the port in the current state: 0d:00h:22m:11s

                Port-channels in the group:
                ---------------------------

Port-channel: Po9    (Primary Aggregator)

------------

Age of the Port-channel   = 0d:02h:39m:10s
Logical slot/port   = 11/9          Number of ports = 3
Port state          = Port-channel Ag-Inuse
Protocol            =   LACP
Port security       = Disabled

Ports in the Port-channel:

Index   Load   Port     EC state        No of bits
------+------+------+------------------+-----------
  0     00     Gi3/41   Active             0
  1     00     Gi3/42   Active             0
  2     00     Gi3/43   Active             0

Time since last port bundled:    0d:00h:22m:11s    Gi3/43
Time since last port Un-bundled: 0d:00h:22m:17s    Gi3/43

Have I configured something incorrectly? I would like to be able to use three seperate sessions of about 1Gb/s each simultaneously.

A few words reagrding our setup:
We have two 4506e connected with an 8 x 1GB/s etherchannel. The server mentioned above (back up server) is connected to both switches with 3 links of 1GB/s each, also there are two smaller servers, each connected with 1GB/s to one of the switches.

I would like to be able to copy a file from both smaller servers to the back up server simultaneously with a combined speed of 2GB/s (or at least close to it). I am able to get speeds of 1GB/s when copying one file simultaneously.

I would greately appreciate any insight on this matter...

Cheers

Phil

2 REPLIES
VIP Purple

LACP Link Aggregation not using full bandwith

Did you change the Load-balancing-Algorithm? I remember that the 4500 uses a default of src-dest-ip, which should be ok in your scenario. Check that with "sh etherchannel load-balance"

The Problem ist that with only a small amount of systems you have a high propability of ending up on the same link.

You can control that with the command "test etherchannel load-balance interface port-channel ...". There you provide the IP-addresses of your servers and see which link would be picked.


--
Don't stop after you've improved your network! Improve the world by lending money to the working poor: http://www.kiva.org/invitedby/karsteni
New Member

Re: LACP Link Aggregation not using full bandwith

Karsten,

thank you for your reply.

I tried the default load balancing mehtod and have just tried to issue your command. Unfortunately, my 4506e (IP Base) does not offer to test any etherchannel related config.

I have tried switching to a SRC & DST Port load balancing method (port-channl load-balance src-dst-port), but unfortunately without any success....

Cheers

Phil

      

[EDIT:

I don't know how relevant this is, but I just looked at the show etherchannel load-balance command. I noticed, it uses the MAC for non IP traffic. The Etherchannel is configured as a L2 channel, but I am using IPv4 for the data transfer... Not sure, which load balancing method the 4506e would use...

HF-SW-01-02#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
        src-dst-port

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
  IPv4: Source XOR Destination TCP/UDP (layer-4) port number
  IPv6: Source XOR Destination IP address

]

[EDIT2:

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.18 10.1.15.10

Map port for Ip 10.1.15.18, 10.1.15.10 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.18 10.1.15.12

Map port for Ip 10.1.15.18, 10.1.15.12 is Gi3/41(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.19 10.1.15.12

Map port for Ip 10.1.15.19, 10.1.15.12 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.10 10.1.15.18

Map port for Ip 10.1.15.10, 10.1.15.18 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.10 10.1.15.19

Map port for Ip 10.1.15.10, 10.1.15.19 is Gi3/43(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.12 10.1.15.18

Map port for Ip 10.1.15.12, 10.1.15.18 is Gi3/41(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

HF-SW-01-02#$rm software etherchannel po 9 map ip 10.1.15.12 10.1.15.19

Map port for Ip 10.1.15.12, 10.1.15.19 is Gi3/42(Po9)

NOTE: Software forwarded traffic will use Gi3/42(Po9)

When issueing the same command with a random IP + Port combination, the results are similar... Is there a way to optimize the load balancing behavoiur?

]

Nachricht geändert durch Philipp Bieber

3504
Views
0
Helpful
2
Replies