Questions on LACP and performance

john.hardman · ‎02-16-2012

I have (2) 3750E switches in a stack. I have a Windows 2K8R2SP1 server with (4) 1Gbps ports configured in a port-group. I have a FalconStor iSCSI SAN similarly configured with (4) 1Gpbs ports in a different port group. I have MRTG running. The SAN and server are both pre-production with basically no loads, the stack is in production but is not getting hit hard at all.

So I creadted a iSCSI target on the SAN and did some sustatined file copy operations and watched MRTG over the course of about 30mins before stopping the copy opertation (from local DAS on the server to iSCSI attached disk). My port channels show a max speed of ~118Mbps and ~120Mbps and the interfaces in those port groups show anywhere from 1000bps to ~100Mbps... the ports will have one or two with ~100Mbps, one with ~20-30Mbps and one with esentually nothing.

So why would I see such low traffic rates? If I were going point to point with a port-group to a single port I could see never getting over the single port speed, but there is 4Gbps of pipe here on both sides, and its not even using 10% of 1Gbps. No errors found anywhere in the mix.

Here are the relivant configs...

port-channel load-balance src-dst-ip

!

interface Port-channel20

description 60 iSCSI to VSS-1

switchport trunk encapsulation dot1q

switchport trunk native vlan 60

switchport trunk allowed vlan 60

switchport mode trunk

(this is the SAN)

!

interface Port-channel40

switchport trunk encapsulation dot1q

switchport trunk native vlan 60

switchport trunk allowed vlan 60

switchport mode trunk

flowcontrol receive on

(this is the server)

!

(typical port in the 20 group)

interface GigabitEthernet1/0/16

description VSS-1 iSCSI bond for 60

switchport trunk encapsulation dot1q

switchport trunk native vlan 60

switchport trunk allowed vlan 60

switchport mode trunk

speed 1000

duplex full

lacp port-priority 100

channel-group 20 mode passive

!

(typical port in the 40 group)

interface GigabitEthernet2/0/3

switchport trunk encapsulation dot1q

switchport trunk native vlan 60

switchport trunk allowed vlan 60

switchport mode trunk

speed 1000

duplex full

flowcontrol receive on

channel-group 40 mode on

glen.grant · ‎02-16-2012

If you are just transferring between 2 addresses that conversation will only flow down a single port within that port channel , thats the way port channels work . As you get more inputs from different addresses then the port channels will be more evened out due to the way the switch hashes the traffic from different sources down each port in the port channel . A single given conversation will only go down a single port .

john.hardman · ‎02-16-2012

Thanks for the response Glen.

I suspected that is the way it had to work, I was uncertain as everything I read on PtP IP on channels refered to a single host with only one port connecting to a multiport channel.

The outstanding part though is the bandwidth. I did some testing with iperf. The client is on a UCS blade connected to the 3750E stack by 10Gbps and the same server. The best it would do is about 500Mbps.