Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

Brand new 3750's Drop all Ports

I had an existing 3570 and ran short on ports, so I just bought/installed a pair of (brand new) 3750's in a stack and moved everything (Servers) to that stack.

Everything is fine for about 3 days, then at 2:40 am Thursday production servers drop off line for global users.  I come in and see that the power to the switches is fine but every single port has no link light on (both Switches).  Cycled power on the switches and no help, same condition.  This stack of switches uplinks to another server farm stack with a simple gbit ethernet port.

Figured I better move everything back to the old switch so I first moved the uplink cable back to the 3570 that I had decommisioned earlier.  Then moved about a half dozen cables (servers) back to the old switch and bang all the link lights came back on on the "failed" stack.  Since it was working agin, I quickly ran another uplink from the 3570 to the 3750 stack and all is good.

I opened a case with Cisco and unfortunately they were no help.  I was hoping for some advice on how to troubleshoot, where to find log files, anything that might help.

Doing a little reading I see that by default "Spanning-tree Etherchannel guard misconfig" is enabled.  I do not have any loops going on but I did have a boatload of traffic on one port due to the tape backup server running on that port (I just moved the backup server to another switch).  I disabled the above command for the time being.

I have verified there were no power issues, I have redundant UPS's, etc for all the gear.

There are zero errors in the ports statistics (since reboot). Temperature is fine...

Questions:

1. Is it possible the "Spanning-tree etherchannel guard guard misconfig" was seeing heavy traffic from the backup system and shut down ports? If that is the case why would if shut down ALL ports (no link lights).

2. Is there something I should do (port configuration wise) wen I attach a gbit ethernet port from the stack to another switch (ie Cisco 3570).  Reason I ask it that when the 3750 Stack is connected to the 3570 then the traffic lights are on almost constantly on the 3750 stack.  It looks real similar to a looped switch condition (visually).  But during the crazy link light flashing on the 3750 stack, the 3570 that the stack is connected to looks normal (link lights flicker a little...pause...flicker...pause...)

3. Any logs I can look through?  I did the show log commands and don't erally see anything in there.

Any other suggestions are appriciated.

1 ACCEPTED SOLUTION

Accepted Solutions
Silver

Re: Brand new 3750's Drop all Ports

I hope you never have this kind of outage again, but if you do gathering information at the time of crisis is critical to determine what's happing.  Following a reload it's difficult to discover the cause; unless a crash file was written.

In order to have good logs configure the following at installation:

logging monitor debugging
logging buffered debugging

If you have a syslog server point the switch at it; generally the switch will start sending insightful messages as it begins to fail.

logging trap debugging
logging a.b.c.d ### were a.b.c.d = IP address of syslog server

When the problem is noticed get on the console and see what the switch thinks is going on.  There may be messages scrolling across the screen.  Also try the following.


show log
show spanning-tree
show interface status



Chris

6 REPLIES
New Member

Re: Brand new 3750's Drop all Ports

1,

no. this is to diable ports that are configured in a bundle with wrong memebers.

2,

it is possible there was a spanning tree loop somewhere, but if you only had 1 cable between the two, and both had default out of the box cofig, the this wouldnt occur.

3,

show log is the only place to look for logs. try a "dir flash" and look for any files called crashinfo. this is what ios does when it crashes, puts info it that file.

send this to TAC, and they can help you then,

perhaps post the configs?

New Member

Re: Brand new 3750's Drop all Ports

thanks for the reply.  Config is below:

3750_Cluster#
3750_Cluster#show config
Using 5044 out of 524288 bytes
!
! Last configuration change at 14:11:19 LA Fri Oct 29 2010
! NVRAM config last updated at 14:11:44 LA Fri Oct 29 2010
!
version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname 3750_Cluster
!
boot-start-marker
boot-end-marker
!
enable secret 5 ***********************
enable password ***********
!
!
!
no aaa new-model
clock timezone LA -8
switch 1 provision ws-c3750x-48
switch 2 provision ws-c3750x-48
system mtu routing 1500
authentication mac-move permit
ip subnet-zero
!
!
!
!
crypto pki trustpoint HTTPS_SS_CERT_KEYPAIR
enrollment selfsigned
serial-number
revocation-check none
rsakeypair HTTPS_SS_CERT_KEYPAIR
!
!
crypto pki certificate chain HTTPS_SS_CERT_KEYPAIR
certificate self-signed 01 nvram:**********************
!
spanning-tree mode pvst
spanning-tree extend system-id
!
!
!
!
vlan internal allocation policy ascending
!
!
!
interface FastEthernet0
no ip address
shutdown
!
interface GigabitEthernet1/0/1
!
interface GigabitEthernet1/0/2
!
interface GigabitEthernet1/0/3
!
interface GigabitEthernet1/0/4
!
interface GigabitEthernet1/0/5
!
interface GigabitEthernet1/0/6
!
interface GigabitEthernet1/0/7
!
interface GigabitEthernet1/0/8
!
interface GigabitEthernet1/0/9
!
interface GigabitEthernet1/0/10
!
interface GigabitEthernet1/0/11
!
interface GigabitEthernet1/0/12
!
interface GigabitEthernet1/0/13
!
interface GigabitEthernet1/0/14
!
interface GigabitEthernet1/0/15
!
interface GigabitEthernet1/0/16
!
interface GigabitEthernet1/0/17
!
interface GigabitEthernet1/0/18
!
interface GigabitEthernet1/0/19
!
interface GigabitEthernet1/0/20
!
interface GigabitEthernet1/0/21
!
interface GigabitEthernet1/0/22
!
interface GigabitEthernet1/0/23
!
interface GigabitEthernet1/0/24
!
interface GigabitEthernet1/0/25
!
interface GigabitEthernet1/0/26
!
interface GigabitEthernet1/0/27
!
interface GigabitEthernet1/0/28
!
interface GigabitEthernet1/0/29
!
interface GigabitEthernet1/0/30
!
interface GigabitEthernet1/0/31
!
interface GigabitEthernet1/0/32
!
interface GigabitEthernet1/0/33
!
interface GigabitEthernet1/0/34
!
interface GigabitEthernet1/0/35
!
interface GigabitEthernet1/0/36
!
interface GigabitEthernet1/0/37
!
interface GigabitEthernet1/0/38
!
interface GigabitEthernet1/0/39
!
interface GigabitEthernet1/0/40
!
interface GigabitEthernet1/0/41
!
interface GigabitEthernet1/0/42
!
interface GigabitEthernet1/0/43
!
interface GigabitEthernet1/0/44
!
interface GigabitEthernet1/0/45
!
interface GigabitEthernet1/0/46
!
interface GigabitEthernet1/0/47
!
interface GigabitEthernet1/0/48
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/1
!
interface TenGigabitEthernet1/1/2
!
interface GigabitEthernet2/0/1
!
interface GigabitEthernet2/0/2
!
interface GigabitEthernet2/0/3
!
interface GigabitEthernet2/0/4
!
interface GigabitEthernet2/0/5
!
interface GigabitEthernet2/0/6
!
interface GigabitEthernet2/0/7
!
interface GigabitEthernet2/0/8
!
interface GigabitEthernet2/0/9
!
interface GigabitEthernet2/0/10
!
interface GigabitEthernet2/0/11
!
interface GigabitEthernet2/0/12
!
interface GigabitEthernet2/0/13
!
interface GigabitEthernet2/0/14
!
interface GigabitEthernet2/0/15
!
interface GigabitEthernet2/0/16
!
interface GigabitEthernet2/0/17
!
interface GigabitEthernet2/0/18
!
interface GigabitEthernet2/0/19
!
interface GigabitEthernet2/0/20
!
interface GigabitEthernet2/0/21
!
interface GigabitEthernet2/0/22
!
interface GigabitEthernet2/0/23
!
interface GigabitEthernet2/0/24
!
interface GigabitEthernet2/0/25
!
interface GigabitEthernet2/0/26
!
interface GigabitEthernet2/0/27
!
interface GigabitEthernet2/0/28
!
interface GigabitEthernet2/0/29
!
interface GigabitEthernet2/0/30
!
interface GigabitEthernet2/0/31
!
interface GigabitEthernet2/0/32
!
interface GigabitEthernet2/0/33
!
interface GigabitEthernet2/0/34
!
interface GigabitEthernet2/0/35
!
interface GigabitEthernet2/0/36
!
interface GigabitEthernet2/0/37
!
interface GigabitEthernet2/0/38
!
interface GigabitEthernet2/0/39
!
interface GigabitEthernet2/0/40
!
interface GigabitEthernet2/0/41
!
interface GigabitEthernet2/0/42
!
interface GigabitEthernet2/0/43
!
interface GigabitEthernet2/0/44
!
interface GigabitEthernet2/0/45
!
interface GigabitEthernet2/0/46
!
interface GigabitEthernet2/0/47
!
interface GigabitEthernet2/0/48
!
interface GigabitEthernet2/1/1
!
interface GigabitEthernet2/1/2
!
interface GigabitEthernet2/1/3
!
interface GigabitEthernet2/1/4
!
interface TenGigabitEthernet2/1/1
!
interface TenGigabitEthernet2/1/2
!
interface Vlan1
ip address 172.16.7.201 255.255.248.0
!
ip default-gateway 172.16.7.254
ip http server
ip http secure-server
ip sla enable reaction-alerts
snmp-server community *** RO
snmp-server location ***********
snmp-server contact *********************
!
!
line con 0
password **********
login
line vty 0 4
password **********
login
length 0
line vty 5 15
password ***********
login
!
end

3750_Cluster#

Silver

Re: Brand new 3750's Drop all Ports

I hope you never have this kind of outage again, but if you do gathering information at the time of crisis is critical to determine what's happing.  Following a reload it's difficult to discover the cause; unless a crash file was written.

In order to have good logs configure the following at installation:

logging monitor debugging
logging buffered debugging

If you have a syslog server point the switch at it; generally the switch will start sending insightful messages as it begins to fail.

logging trap debugging
logging a.b.c.d ### were a.b.c.d = IP address of syslog server

When the problem is noticed get on the console and see what the switch thinks is going on.  There may be messages scrolling across the screen.  Also try the following.


show log
show spanning-tree
show interface status



Chris

New Member

Re: Brand new 3750's Drop all Ports

Thanks for the tips regarding Logging.  I have no IT training whatsoever so I am winging all this.

I will setup a syslog server (right after I Google what that is)

Regards,

-Bryan

Silver

Re: Brand new 3750's Drop all Ports

Bryan,

I highly reccommend this book:

http://www.absolutefreebsd.com/

download freeBSD:

http://www.freebsd.org/

Find an old PC that's been discarded cause windoze needs 42 gigs of RAM and a 20 terabyte hard drive to be supported.

With these three things go through the book cover to cover and you'll learn a ton, including syslog server.


Chris

New Member

Re: Brand new 3750's Drop all Ports

I'm alreadying downloading the sw thanks.

I'm running DOS 3.3 with 640K ram. When I can afford it

I will get a 5 MB hard disk.

Regards

-Bryan

646
Views
0
Helpful
6
Replies
CreatePlease login to create content