4500x losing connectivity to SG300 switches issue

[davidquintanilla]

I have a new Catalyst 4500X-16 SFP+ Switch with ten GLC-SX-MMD SFP modules connected to ten Cisco SG300-10 switches with MGBSX1 SFP modules.  The 4500X is all default config except all I did was setup SSH so I can manage remotely.  I have GB connectivity between small SG300-10 switches and 4500X switch without any problems at all, until the SG300 reboots.  If power goes out or SG300 is restarted for any reason, it does not communicate with the 4500X switch any longer.  If I restart the 4500X, then connectivity is restored.  I don't see any port security setup or no ports are down...everything appears up, but I can't communicate until I restart the 4500X switch.

I'm not sure where to begin troubleshooting this one.  I am putting battery units on each SG300-10 switch, but that is just a temporary fix and I'd like to get to the core of the problem. 

4500X is running IOS-XE Version 3.05.00.E

Thanks in advance for any help you can provide.

Below is info on the 4500 switch...

Joist4500>show int status

Port      Name               Status       Vlan       Duplex  Speed Type
Te1/1                        connected    1            full a-1000 1000BaseSX
Te1/2                        connected    1            full a-1000 1000BaseSX
Te1/3                        connected    1            full a-1000 1000BaseSX
Te1/4                        connected    1            full a-1000 1000BaseSX
Te1/5                        notconnect   1            full   auto 1000BaseSX
Te1/6                        notconnect   1            full   auto No XCVR
Te1/7                        connected    1            full a-1000 1000BaseSX
Te1/8                        notconnect   1            full   auto No XCVR
Te1/9                        notconnect   1            full   auto No XCVR
Te1/10                       connected    1            full a-1000 1000BaseSX
Te1/11                       connected    1            full a-1000 1000BaseSX
Te1/12                       connected    1            full a-1000 1000BaseSX
Te1/13                       notconnect   1            full   auto No XCVR
Te1/14                       connected    1            full a-1000 1000BaseSX
Te1/15                       notconnect   1            full   auto 1000BaseSX
Te1/16                       notconnect   1            full   auto No XCVR
Joist4500>

Joist4500>show span

VLAN0001
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0025.641d.cfe4
             Cost        10
             Port        1 (TenGigabitEthernet1/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32769  (priority 32768 sys-id-ext 1)
             Address     58f3.9c8d.b988
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  15  sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/1               Root FWD 4         128.1    P2p
Te1/2               Desg FWD 4         128.2    P2p
Te1/3               Desg FWD 4         128.3    P2p
Te1/4               Desg FWD 4         128.4    P2p
Te1/7               Desg FWD 4         128.7    P2p
Te1/10              Desg FWD 4         128.10   P2p
Te1/11              Desg FWD 4         128.11   P2p

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------

Te1/12              Desg FWD 4         128.12   P2p
Te1/14              Desg FWD 4         128.14   P2p

Joist4500>

Joist4500#show spanning detail

 VLAN0001 is executing the ieee compatible Spanning Tree protocol
  Bridge Identifier has priority 32768, sysid 1, address 58f3.9c8d.b988
  Configured hello time 2, max age 20, forward delay 15
  Current root has priority 32768, address 0025.641d.cfe4
  Root port is 1 (TenGigabitEthernet1/1), cost of root path is 10
  Topology change flag not set, detected flag not set
  Number of topology changes 10 last change occurred 4d05h ago
          from TenGigabitEthernet1/4
  Times:  hold 1, topology change 35, notification 2
          hello 2, max age 20, forward delay 15
  Timers: hello 0, topology change 0, notification 0, aging 300

 Port 1 (TenGigabitEthernet1/1) of VLAN0001 is root forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.1.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 20bb.c0f2.6800
   Designated port id is 128.50, designated path cost 6
   Timers: message age 3, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 10, received 185137

 Port 2 (TenGigabitEthernet1/2) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.2.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.2, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185136, received 1

 Port 3 (TenGigabitEthernet1/3) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.3.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.3, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185134, received 1

 Port 4 (TenGigabitEthernet1/4) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.4.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.4, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185125, received 0

 Port 7 (TenGigabitEthernet1/7) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.7.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.7, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185133, received 0

 Port 10 (TenGigabitEthernet1/10) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.10.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.10, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185130, received 0

 Port 11 (TenGigabitEthernet1/11) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.11.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.11, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185129, received 0

 Port 12 (TenGigabitEthernet1/12) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.12.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.12, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185129, received 0

 Port 14 (TenGigabitEthernet1/14) of VLAN0001 is designated forwarding
   Port path cost 4, Port priority 128, Port Identifier 128.14.
   Designated root has priority 32768, address 0025.641d.cfe4
   Designated bridge has priority 32769, address 58f3.9c8d.b988
   Designated port id is 128.14, designated path cost 10
   Timers: message age 0, forward delay 0, hold 0
   Number of transitions to forwarding state: 1
   Link type is point-to-point by default
   BPDU: sent 185127, received 0

login as: dquintanilla
Sent username "dquintanilla"
dquintanilla@172.29.30.237's password:

Joist4500>show int status

Port      Name               Status       Vlan       Duplex  Speed Type
Te1/1                        connected    1            full a-1000 1000BaseSX
Te1/2                        connected    1            full a-1000 1000BaseSX
Te1/3                        connected    1            full a-1000 1000BaseSX
Te1/4                        connected    1            full a-1000 1000BaseSX
Te1/5                        notconnect   1            full   auto 1000BaseSX
Te1/6                        notconnect   1            full   auto No XCVR
Te1/7                        connected    1            full a-1000 1000BaseSX
Te1/8                        notconnect   1            full   auto No XCVR
Te1/9                        notconnect   1            full   auto No XCVR
Te1/10                       connected    1            full a-1000 1000BaseSX
Te1/11                       connected    1            full a-1000 1000BaseSX
Te1/12                       connected    1            full a-1000 1000BaseSX
Te1/13                       notconnect   1            full   auto No XCVR
Te1/14                       connected    1            full a-1000 1000BaseSX
Te1/15                       notconnect   1            full   auto 1000BaseSX
Te1/16                       notconnect   1            full   auto No XCVR
Joist4500>show span

VLAN0001
  Spanning tree enabled protocol ieee
  Root ID    Priority    32768
             Address     0025.641d.cfe4
             Cost        10
             Port        1 (TenGigabitEthernet1/1)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32769  (priority 32768 sys-id-ext 1)
             Address     58f3.9c8d.b988
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  15  sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Te1/1               Root FWD 4         128.1    P2p
Te1/2               Desg FWD 4         128.2    P2p
Te1/3               Desg FWD 4         128.3    P2p
Te1/4               Desg FWD 4         128.4    P2p
Te1/7               Desg FWD 4         128.7    P2p
Te1/10              Desg FWD 4         128.10   P2p
Te1/11              Desg FWD 4         128.11   P2p

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------

Te1/12              Desg FWD 4         128.12   P2p
Te1/14              Desg FWD 4         128.14   P2p

Below is info on one of the SG300-10 switches.

I have no reason to stay on this version of OS and will work on  upgrading version soon to see if this helps.  Thanks again.

[nspasov]

Can you post:

1. Port configurations from both switches

2. Spanning-tree configurations from both switches

3. Output from "show logging" on both switches

Also, the XE code has been somewhat problematic so you should really try to stay on the 3.4.4 version unless you have a specific reason to not be on that version

Thank you for rating helpful posts!

[davidquintanilla]

I updated my question with more information in case that helps.  Below is spanning summary on the 4500.

[nspasov]

Thank you for posting all of the detailed info. However, for some reason the screenshots are not showing up. Can you try attaching them again?

[davidquintanilla]

I attached the pics for the sg300-10 switches to the original post.

[nspasov]

Got it now. Sorry and I should have been more specific but can you post the out from the 4500-x:

show run | sec spanning-tree

sh run interface interface_connecting_to_sg300

[davidquintanilla]

There is no configuration under the interface.  When I do sh run int Te1/14 (port to an SG300), it just has :

show run int Te1/14

Building configuration...

Current configuration : 40 bytes

!

interface TenGigabitEthernet1/14

end

The | isn't working on the 4500 so can't run the other command you asked for.  When I do a sh run, below are the only entries I see related to spanning-tree.

!

spanning-tree mode pvst

spanning-tree extend system-id

!

[nspasov · Accepted solution]

Here are a few things that I can recommend:

1. I would definitely move to the 3.4.4 code (Previously mentioned)

2. As suggested by "devils_advocate" I would also consider making the 4500-X the core switch (if applicable)

3. The SG 300 is running rapid-spanning-tree while the 4500-X is running classic pvst. I should try to run the same type of stp between the two switches and throughout your network

4. The SG 300 is running spanning-tree path long while that does not seem to be the case with the 4500-X. You definitely need this to match throughout the network. There are many writings that you can find about the long stp metric online

5. I looked in the screenshots from the SG 300 and the rest of the info that you have provided and it is not clear to me which port on the SG 300 is connecting to the 4500-X switch. I am interested to know what that configuration looks like. If it is a trunk, then you should also configure the 4500-X to be a trunk. If not then you still should do some basic interface configs on the 4500-X side. As of right now, the port is using its default configs so it will negotiate trunking and use default STP settings. 

6. If possible I would also like to see the log output from the 4500-X immediately after the SG300 reload and when the connectivity issue happens. So if possible clear the log buffer and then issue show logging after reloading the SG 300. Make sure that you temporary entered the following:

no logging console

logging buffered 4096 debugging

Thank you for rating helpful posts!

[Reuben Farrelly]

Also note that although you should be running the same (rapid) type of Spanning Tree on both, the SG300 does not support per-VLAN rapid Spanning Tree whereas the 4500-X does (along with most other Cisco switches).

[nspasov]

Thanks for info Reuben! I have never worked with the SG300 switches so I was not aware of its capabilities. 

[davidquintanilla]

Thanks for the answers to you and everyone who responded.  I'll change the 4500 to run Rapid pvst to match the 2960s I have on the network and SG300 switches.  That should get them all the same.  Hopefully I can upgrade the OS on the 4500 before the end of the week and also set the 4500 as root.  I'll update all once this is complete and post new configs.

[Reuben Farrelly]

We've had good experiences with 3.5.3E - seems the biggest problems have been finally fixed in this version.  Definitely would not recommend 3.5.0E though...

[devils_advocate]

I would start by setting the 4500x to be the root bridge for all the Vlans as it doesn't appear to be currently, looking at the details above.

You may well have some rather suboptimal routing going on.

[davidquintanilla]

I think I have all my switches running rapid pvst and the 4500x switch is the root (pic attached).  I only have vlan 1 on my network.  We'll add a voice vlan next year most likely but right now it is just the one vlan.  Thanks for the help.

[nspasov]

Keep us posted on the outcome! :)