NIC Teaming

[joemarr_brodart]

I have several servers that we are about to deploy Oracle RAC to. Each of these servers have 6 NICs. We would like to take 4 of the NICs and dedicate them to the RAC interconnect in bounded pairs. I would like to bound 2 NICs (for ~200MB of throughput) and connect them to 6509E and configured the other 2 NICs and connect them to another 6509E for failover purposes.

I would suspect that I would use LACP on the switch side, but what would I need to do on the server side? Im using Broadcom NICs, and was hoping to use a kernel mode driver (vs. the Broadcom software). Is this a common practice?

Also, are there any switch configuration settings for tweaking performance (other than jumbo frame support)?

[tomek0001]

I think you might have to use the Broadcom drivers to configure Etherchannel, don't believe there is native support for that.

As for performance, you can look into doing QoS on those ports if the servers are more important than others. You should also look at the different load balancing algorithms for etherchannel to fully utilize your links. For example:

sw(config)#port-channel load-balance (dst-ip | dst-mac | src-dst-ip | src-dst-mac | src-ip | src-mac)

destination ip

destination mac

source and destination ip

please rate is this helps

[joemarr_brodart]

Thanks for the reply, I know I can do 802.3ad in the kernal. What I'm trying to understand is if what is the best way to configure the ports. Do I create 1 bonded channel on the server (so I can put the 1 IP on the bonded channel) and then split the 4 ports so their are 2 on each switch?

[jasosan22]

I am looking at teaming on a couple of servers. One is a Dell and One is an HP both have broadcom drivers. I saw the etherchannel configuration guide. I had a question on what exactually the load balance commands do:

Configure the EtherChannel load balancing. This configuration is applicable for all EtherChannels configured on this switch.

Switch(config)#port-channel load-balance ?

dst-ip Dst IP Addr

dst-mac Dst Mac Addr

src-dst-ip Src XOR Dst IP Addr

src-dst-mac Src XOR Dst Mac Addr

src-ip Src IP Addr

src-mac Src Mac Addr

Switch(config)#port-channel load-balance src-mac

====================================

You feel like eloboratin on these load balance algorithms? I couldn't find any explaination on why to use src-mac or dst-mac.

[Ryan Carretta]

LACP is always a best-practice, as it will prevent a misconfiguration from hosing up your network. A channel negotiation protocol (LACP, f.ex.) just helps to ensure both participating devices are on the same page so they can act accordingly.

As far as the hashing algorithm. Imagine you have traffic traversing over an etherchannel that is a routed interface. All the traffic comes from the same source MAC, and is probably destined for the same destination MAC (assuming a full-duplex point-to-point connection). In this case, if you were to set the algorithm to be by src-mac or dst-mac, you'd see that all the traffic hashed to just one link in the etherchannel and the other ports would not be utilized unless the link being utilized were to fail.

On the other hand, there are situations where IP hashing is not desirable, as when all the traffic is coming from/going to the same IP address, as in a virtual server cluster, for example.

It really just depends on what the traffic profile is that the etherchannel will be carrying.

-Ryan

[jasosan22]

I was going to configure the two ports as per the etherchannel guide. This is simply layer 2 etherchannel. The issue is that there is a server with a SAN Connected to it. There are some users(about 3) that are on the same subnet that like to ftp very large image files. (8 gig each) There is also a SAN replication that goes out of WAN. I will eventually create a "back end" network for the replication but there is a financial limitation as to how much I can spend on the infrastructure. My question. If there is no router interface stamping everything with the same source. How should I configure the load balancing (for everything on the local segment) so that users on the local segment can equally load balance. Thank you in advance. Here is the config (3560/2950)

Switch(config-if)#spanning-tree portfast

Switch(config-if)#channel-group 1 mode active

Switch(config)#port-channel load-balance src-mac