Huge Etherchannel Problems

[gavin.mckee]

Hi,

I am trying to configure etherchannel between a 3650G and a 6509 (Hybrid mode).

Ports gi0/1 and gi0/24 on the 3560G should be in the channel. I add them with the command:

channel-group 1 mode desirable non-silent

So this adds both the ports and the channel comes up fine.

I check the CatOS 6509 for channel information: show channel

This shouws me that the channel has been formed.

After about 10 minutes the ports 3/3,3/17 fo into errdisable. When I check show errdisable I can see that etherchannel misconfiguration is the problem.

I can't get to the root of the problem.

I had to remove the etherchannel configuration from the 3560 in order to get the network to stabilize.

What is confusing me is the fact that the channel comes up and works for approx 10 mins then goes down causing a spanning tree problem.

I have attached the config files:

itgedcat1-241106-ETHERCHANNEL_PROBLEMS.txt (Cat 6509 CatOs) Config

ps-switch-dub_ETHERCHANNEL_PROBLEMS.txt (C3560G Config)

Any help would be well rated and much appreciated.

Thanks

Gavin

[ankbhasi]

Hi Gavin,

Can you also set the duplex settings on CAT6k for port 3/3 and 3/17 as full which you have configured on your 3550 for port gig0/1 and gig0/24?

Also can you configure the port channel with mode "ON" on both the sides for 3550 as well as Cat6509?

Regards,

Ankur

[gavin.mckee]

Hi Ankur,

I think you posted the other day when I was having trouble with spanning tree. It was the etherchannel misconfig that was causing this.

Why would setting the mode to on help?

If you look at the duplex and speed settings for 3/3, 3/17 they are FULL and 1000, just like the 3560G gi0/1 and gi0/24.

I thought if the channel comes up in the first instance everything should be fine.

Thanks for helping here.

[amit-singh]

Hi Gavin,

I see that config looks fine on both the switches. Except that on 6500 the duplex is not hardcoded as full on the ports 3/3,3/17, which is there on the 3560 side. Your all the port configs should match when you are setting up an ether-channel.

Can you please paste the following outputs.

Show interface gig 0/1 and gig 0/24 from 3560

Show port 3/3 and 3/17 from 6500.

Also logs from btoh the switches.

-amit singh

[gavin.mckee]

Here is the show interface for the relevant ports on the 3560G and Show port 3/3 and 3/17 on the Cat.

[amit-singh]

Hi Gavin,

As per the outputs attached, Duplex is not hardcoded on port 3/3 and 3/17.

set port speed 3/3,3/8,3/17,3/30,3/43 1000

set port duplex 3/1-2,3/4-6,3/10-13,3/15-16,3/20,3/25,3/27-28,3/32-34,3/39,3/41,3/45,3/48 full

-amit singh

[gavin.mckee]

would this cause problems with the channel even after is have become established?

How would I tell that this is causing the problem. Is there a channel debug for the 6509. I ran debug etherchannel all on the 3560 and could not see why the channel was going down.

Thanks

[ankbhasi]

Hi Gavin,

Sometime negotiation does not work fine as we have hardcoded the duplex on one side and on aother side it is auto negotiate so the channel will come up first and once it sees a negotiation problem it can go into errdisable state.

Duplex and speed and all settings for etherchannel ports should match ob both the sides of etherchannel.

Also theer are no debugs available on CATOS for etherchannel. Als the reason i recommeded for mode "ON" is to see if negotiation of PAGP packtes not causing the link to go into errdisable state and once we configure "ON" it will unconditionally form an etherchannel.

Try setting hardcoded duplex settings om cat6509 and see the results.

Ankur

[amit-singh]

In addition to Ankur's post one of the configuration guideline when configuring the ether-channel is to " Configure all ports in an EtherChannel to operate at the same speed and duplex mode". So in this the channel came up and when the port tried to check the speed/duplex status, ether-channel protocol saw the inconsistency and it went down on you.

When we are setting up the port-channel to " ON" we are not letting the Pagp to negotite on the channel formation. With this I would still suggest to have all the ports either hardcoded for speed/duplex or leave them on autonegotiate.

-amit singh

[gavin.mckee]

Thanks guys for all suggestions.

I'll try the following:

double check that ports 3/3,3/17 are:

set port duplex 3/3 full

set port duplex 3/17 full

set port speed 3/3 1000

set port spedd 3/17 1000

I will also set the channel mode to on

Then on the 3560G

interface gi0/1

duplex full

speed 1000

channel-group 1 mode on

interface gi0/24

duplex full

speed 1000

channel-group 1 mode on

The way I see it this should allow the channel to work!

By the way I am using CAT5E Straight through cable to connect the 3650G to the 6509. This has always worked grand. Should I change these cables?

Thanks

Gavin

[amit-singh]

Gavin,

Sounds like that's we need on this. I would suggested to use a factory fitted cross-over cable not the self-make.

Let us know your findings on this and we will take it from there.

-amit singh

[glen.grant]

Make sure on your gig links that they are set as auto on both ends , it is seeing these mismatches and pulling down the channel . We saw this in some testing we did , where it would stay up for awhile and then pull the channel down if something is not right. Your original config should work fine with the desirable non-silent config , we have it all over between the same 2 types of gear and it works fine . In cisco docs I have read reccomends using PAGP or LACP instead of forcing it on because if you don't configure the channel correct it can cause spanning tree loops . Below is a warning out of a cisco doc on using the "on" mode.

Caution You should exercise care when setting the mode to on (manual configuration). All ports configured in the on mode are bundled in the same group and are forced to have similar characteristics. If the group is misconfigured, packet loss or STP loops might occur.