Configuring 4507R for LACP and Trunking?

paul_hutchings · ‎04-04-2008

We're trying to configure our Cisco 4507 (Supervisor Engine IV) to allow a new Dell server with a pair of Broadcom 5708 GigE NIC's to aggregate its NIC's to give us a 2gbps link to the switch.

So far we seem to have got the team and LACP up and enabled, but the adaptor that the Broadcom Admin Util creates for the team is only showing a 1gbps connection where I would have expected it to show as 2gbps.

The individual NICs show as connected at 1gbps.

We're not Cisco experts so are struggling on how to get the 2 NICs to aggregate.

On the server side we've done nothing other than create a team using 802.3ad LINk Aggregation using LACP.

This is what I think the relevent output from "sho conf" is, more available if needed.

version 12.2

boot system flash bootflash:cat4000-i9s-mz.122-18.EW1.bin

!

interface Port-channel2

description "Main Server link"

switchport

switchport mode access

!

interface GigabitEthernet4/4

switchport mode access

spanning-tree portfast

channel-group 2 mode active

channel-protocol lacp

!

interface GigabitEthernet4/20

switchport mode access

spanning-tree portfast

channel-group 2 mode active

channel-protocol lacp

!

I'd really appreciate any pointers here, as I have the feeling we're a couple of commands away but as usual it's knowing what they are :-)

glen.grant · ‎04-04-2008

Switch side looks correct , verify with a "show ether detail" command and make sure the channel is created and running . For a given transfer or connection to your server it will never be more than a gig , a single conversation always rides down the same link . It does not load share across both links . Ehterchannel gives you more aggregate bandwidth in which to transfer all traffic but any single conversation is limited to a 1 gig link .

paul_hutchings · ‎04-04-2008

Hi Glen, thanks for the reply.

This is a new server and the first time we've attempted switch side trunking/LACP so it's quite nice to know we appear to have got it right first time.

Have to admit I wasn't aware that the conversations couldn't physically span more than one link as I thought that was the reason for the switch-side config - the switch essentially presents a 2gbps channel to/from the server.

Either way that shouldn't be an issue so long as we're getting the full benefits of having the 2 links, and that's what's confusing me - I'm simply not sure whether or not the "Team 1" network connection that the Broadcom software creates should appear as a 1gbps or 2gbps link speed in Control Panel/Networking.

Group: 2

----------

Group state = L2

Ports: 2 Maxports = 8

Port-channels: 1 Max Port-channels = 1

Protocol: LACP

Ports in the group:

-------------------

Port: Gi4/4

------------

Port state = Up Mstr In-Bndl

Channel group = 2 Mode = Active Gcchange = -

Port-channel = Po2 GC = - Pseudo port-channel = Po2

Port index = 1 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.

A - Device is in active mode. P - Device is in passive mode.

Local information:

LACP port Admin Oper Port Port

Port Flags State Priority Key Key Number State

Gi4/4 SA bndl 32768 0x2 0x2 0xA1 0x3D

Partner's information:

LACP port Oper Port Port

Port Flags Priority Dev ID Age Key Number State

Gi4/4 SA 1 001e.4f15.5fa7 13s 0x1 0x9 0x3D

Age of the port in the current state: 00d:06h:11m:31s

Port: Gi4/20

------------

Port state = Up Mstr In-Bndl

Channel group = 2 Mode = Active Gcchange = -

Port-channel = Po2 GC = - Pseudo port-channel = Po2

Port index = 0 Load = 0x00 Protocol = LACP

Flags: S - Device is sending Slow LACPDUs F - Device is sending fast LACPDUs.

A - Device is in active mode. P - Device is in passive mode.

Local information:

LACP port Admin Oper Port Port

Port Flags State Priority Key Key Number State

Gi4/20 SA bndl 32768 0x2 0x2 0xA0 0x3D

Partner's information:

LACP port Oper Port Port

Port Flags Priority Dev ID Age Key Number State

Gi4/20 SA 1 001e.4f15.5fa7 14s 0x1 0xA 0x3D

Age of the port in the current state: 00d:06h:11m:31s

Port-channels in the group:

---------------------------

Port-channel: Po2 (Primary Aggregator)

------------

Age of the Port-channel = 04d:06h:52m:30s

Logical slot/port = 11/2 Number of ports = 2

Port state = Port-channel Ag-Inuse

Protocol = LACP

Ports in the Port-channel:

Index Load Port EC state No of bits

------+------+------+------------------+-----------

1 00 Gi4/4 Active 0

0 00 Gi4/20 Active 0

Time since last port bundled: 00d:06h:11m:34s Gi4/20

Time since last port Un-bundled: 00d:06h:12m:12s Gi4/20

Tony Ortiz · ‎11-27-2012

Hey bud,

Don't forget to review the global setting of your switch for "load-balance". By default, it's set to "src-dst-ip". You may want to read up on all the different options shown below:

DR-LANSTACK(config)#port-channel load-balance ?

dst-ip Dst IP Addr

dst-mac Dst Mac Addr

dst-port Dst TCP/UDP Port

src-dst-ip Src XOR Dst IP Addr

src-dst-mac Src XOR Dst Mac Addr

src-dst-port Src XOR Dst TCP/UDP Port

src-ip Src IP Addr

src-mac Src Mac Addr

src-port Src TCP/UDP Port

I believe you brought up an interesting point to think about concerning how a single channel takes the communication once established in a link. If you have SRC-DST-IP, that is the case. However, if you change it to SRC-DST-MAC, the packets are split and truly load-balanced and thus taking advantage of all the links at the same time. The DRAWBACK, based on what I read, is that if a packet that comes after another gets there first, I believe packets need to be retransmitted, or some overhead on the CPU is placed because the switch has to re-order them somehow.

I don't at all claim to know everything I'm talking about. Just that we as an organization chose NOT to utilize SRC-DST-MAC because we were using stacked switches. Now that we're upgrading to CORE switches, I may reconsider.

The command above is a global one, by the way, affecting ALL of the port-channels on your switch. FYI.