stacking ports are unstable in 3750 cisco switch

Unanswered Question
Jul 24th, 2008
User Badges:

Dear All,


We have noticed on 3750 switch that the stacking ports have been unstable,

they are flapping server times a day,i couldnt able to find out any thing exeptthe below explanation

Error Message STACKMGR-4-STACK_LINK_CHANGE: Stack Port [chars] Switch [dec] has

changed to state [chars].



--------------------------------------------------------------------------------


Note This message applies only to the Catalyst 3750-E switches.



--------------------------------------------------------------------------------


Explanation The specified stack port status has changed state to up or down. The first [chars] is the stack port (1 or 2),

[dec] is the switch number, and the second [chars] is the new state (up or down).


Recommended Action No action is required.


Logs of the Switch are below

-------------------------------

ul 24 15:35:02.028 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN

Jul 24 15:35:02.892 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state UP

Jul 24 15:35:03.035 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state UP

Jul 24 15:37:30.089 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN

Jul 24 15:37:30.123 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state DOWN

Jul 24 15:37:31.096 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state UP

Jul 24 15:37:31.129 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state UP

Jul 24 15:46:41.542 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state DOWN

Jul 24 15:46:42.549 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state UP

Jul 24 15:50:18.392 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state DOWN

Jul 24 15:50:21.420 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 2 Switch 1 has changed to state UP

Jul 24 15:51:23.044 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN

Jul 24 15:51:24.051 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state UP

Jul 24 15:56:33.144 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN

Jul 24 15:56:34.151 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state UP

Jul 24 15:59:43.478 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN

Jul 24 15:59:44.484 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state UP

Jul 24 16:05:06.748 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state DOWN

Jul 24 16:05:07.755 UTC: %STACKMGR-4-STACK_LINK_CHANGE: Stack Port 1 Switch 2 has changed to state UP


Could some one help out on this


Thanks and regards


Sharan


  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Mark Yeates Thu, 07/24/2008 - 19:56
User Badges:
  • Gold, 750 points or more

Sharan,


Have you tried different stackwise cables or ports of possible? I have had troubles in the past with the cables not being securely tightened in the back of the switch also.


HTH,

Mark

Sebastian Helmer Thu, 07/24/2008 - 22:33
User Badges:
  • Silver, 250 points or more

Yes, I would first check the hardware too.

My question, did you stach switch 1 port 1 -> switch 2 port 1, this is maybe why you got problems, beause it is wrong I think.

(Stupid question, but it's possible... ;-) )


PS: you can give us the output from "show switch detail"

maybe we can see a problem.


Sebastian

UCBNOCWAN Thu, 07/24/2008 - 23:06
User Badges:

Dear All,


Thanks very much for your valuble inputs

please find the show switch details


XXXX-2100-G-3750-1#sh switch detail

Switch/Stack Mac Address : 0091.aad8.2d00

H/W Current

Switch# Role Mac Address Priority Version State

----------------------------------------------------------

1 Member 002f.c987.7300 1 0 Ready

2 Member 0032.1b8a.2200 1 0 Ready

*3 Master 0091.aad8.2c00 1 0 Ready




Stack Port Status Neighbors

Switch# Port 1 Port 2 Port 1 Port 2

--------------------------------------------------------

1 Ok Ok 3 2

2 Ok Ok 1 3

3 Ok Ok 2 1


XXX-2100-G-3750-1#

XXX-2100-G-3750-1#sh switch neighbors

Switch # Port 1 Port 2

-------- ------ ------

1 3 2

2 1 None

3 2 1



XXXX-2100-G-3750-1#sh switch stack-ports

Switch # Port 1 Port 2

-------- ------ ------

1 Ok Ok

2 Ok Ok

3 Ok Ok




Sebastian Helmer Thu, 07/24/2008 - 23:21
User Badges:
  • Silver, 250 points or more

Hi Sharan,

thanks for the output.


The "None" in your output you see on the button ist a mistake. It does not really match to your error messages, but can be in relation to that. I have stacks but never this problem bevor. I think the Stack cable or port is damaged.


I can check that in my lab, but only next tuesday. Maybe some others have more ideas and experience. If you dont't get reply until there, I will try check that in lab.


XXX-2100-G-3750-1#

XXX-2100-G-3750-1#sh switch neighbors

Switch # Port 1 Port 2

-------- ------ ------

1 3 2

2 1 None

3 2 1

UCBNOCWAN Tue, 07/29/2008 - 22:46
User Badges:

Dear All


Any update in this issue


Thanks & regards


Sharan

Sebastian Helmer Tue, 07/29/2008 - 23:10
User Badges:
  • Silver, 250 points or more

Sorry, I was on the road yesterday, i try to imitate it today.


Sebastian

Willem de Groot Wed, 07/30/2008 - 05:50
User Badges:

Hi Sharan,


you are not the only one with this problem :-(

I see the same log entries.


New Stack, IOS 12.2(44)SE2


My Stack has only 2 of once 3 members!


Have there been another member in your stack, which was removed?


Willem

UCBNOCWAN Wed, 07/30/2008 - 21:45
User Badges:

HI all


I have 5 stack switches with the same IOS,Version 12.2(44)SE2. but this problem exist only in 2 stack,i am not sure it is related to IOS


thanks

sharan


Sebastian Helmer Thu, 07/31/2008 - 12:22
User Badges:
  • Silver, 250 points or more

First Sorry for my late replay, lt of work...


I check it today with a Stack and i haven't the problem or can see andy point which could be that.


Second I check some troubleshooting guides, config guides and else. No idea.

Google didn't help too..


I'M really Sorry :(


Maybe this helps, my last Idea

http://www.cisco.com/univercd/cc/td/doc/product/lan/cat3750/12235se/scg/swgold.pdf


I will stay on that problem, if i have new, it old you of course


best regards

Sebastian

UCBNOCWAN Thu, 07/31/2008 - 22:37
User Badges:

Dear All,


Any sugestions from any one


regards

Sharan

Danilo Dy Fri, 08/01/2008 - 08:14
User Badges:
  • Blue, 1500 points or more

Hi,


Thus your stackwise cable in a full ring?


Try configuring the stack-ring speed to full.


Regards,

Dandy

UCBNOCWAN Mon, 08/04/2008 - 01:49
User Badges:

Hi,

please find the out put

G-3750-1#sh switch stack-ring speed


Stack Ring Speed : 32G

Stack Ring Configuration: Full

Stack Ring Protocol : StackWise

#




Could you please tell me how to go abt from here


regards

Sharan

Willem de Groot Thu, 08/28/2008 - 21:45
User Badges:

Hi Sharan,


as i wrote before, i have the same problem.

Down/Ups every 30 Sec.

The same as in your case mostly the same link (sw2 Port 1 to SW1 port 2).

So i decided to change this Cable -> YES it changed it got even worse! Now i have Down/ups every 10 Sec.

After "reload slot 2" (I have only 2 switches) it came to Down/Up times from 1-2 Sec.

After a "reload" (i think this will reload the hole stack) the Down/Ups show up every 20-30 sec somtimes even longer. (Same switch is master again)


Sharan (and others) can you do a ping from a windows client, not connected at the switch to a device connected to this 3750?

I notice here a TTL of 63 or even 59 (different clients)!!!

Pinging a device on the same vlan on a other switch (3524) a have a TTL of 254.

a Tracert only shows 2 Hops.


Can sombody explain this last behavior?


What are the TTLs on a "working" stack?


Thanks

Willem

Willem de Groot Wed, 09/03/2008 - 10:05
User Badges:

Hi Sharan,


Do you still have this problem?

My problem seems to go away if a plug in the 3rd., provisioned switch.

(the Stack was setup originaly with 3 Switches, in Production I only used 2 of them)


if I did a show run, I saw/see


switch 1 provision ws-c3750e-24pd

switch 2 provision ws-c3750e-24pd

switch 3 provision ws-c3750e-24pd


But I only had 2 Swiches in my stack and got the same messages as you have.

Now I Pluged in the 3rd. switch, runnung now for 25 Minutes with the same cables, without a down/up.


doing a sh switch detail:

chspn175#sh switch detail

Switch/Stack Mac Address : 0022.5529.1680

H/W Current

Switch# Role Mac Address Priority Version State

----------------------------------------------------------

*1 Master 0022.5529.1680 10 1 Ready

2 Member 0022.5529.0900 1 1 Ready

3 Member 0022.5529.0980 1 1 Ready




Stack Port Status Neighbors

Switch# Port 1 Port 2 Port 1 Port 2

--------------------------------------------------------

1 Ok Ok 2 3

2 Ok Ok 3 1

3 Ok Ok 1 2




Can you Post the same parts of your config (show run, the switch part and show switch detail).


As soon as i go back to 2 switches I will have the same problem again ;-(


Thanks

Willem


HI all.

Did anyone get a resolution to this problem?

I have a customer with a stack of 6 3750's. We often have problems with stack up/down messages in the log, usually between switches 1&2 but sometimes with other stack members. I've replaced individual switches, all stackwise cables, upgraded the IOS - still we have the same problem.


I'm wondering if there is an IOS problem or a batch of faulty hardware out there.

glen.grant Fri, 01/09/2009 - 04:30
User Badges:
  • Purple, 4500 points or more

First thing to check is to make sure they are tight , we have found they can look tight and seem tobe tight to the hand but you have to use a screwdriver to tighten them down the way they should be . Then check to see how they are cabled , we always go from port 1 switch 1 to port 2 switch , then port 1 switch 2 to port 2 switch 3 and then port 1 switch 3 back to switch 1 port 2 . I have never seen any stack bouncing messages to this point . That being said I think remember seeing some TSB somewhere that cisco had delivered some bad stack cables but I can't swear to it. As far as I'm concerned those connectors are shakey at best. They could have used a better design.In the old days Synoptics had better stack cables for there hubs than the design they used for these.


stormfidus Fri, 01/09/2009 - 05:14
User Badges:

Hi Brian


I have now changed about 8 switches with the flapping stack-ports. The TAC cases I've opened all came with the solution was to insert a new working switch between the 2 flapping switches, to see whether it was switch1 or 2 which was faulty, after that I changed the switch, and the flapping ports stopped right away. Before that solution I also tried to replace stack-cables with no difference and also upgraded the IOS serval times with no change.


So as I see it, the only solution is to isolate the switch which is flapping and replace it with a new one.. - hell of a work, but we dont like flapping stack ports!


-Claus

Actions

This Discussion