Problem with LAN

[Aeroexpress]

Good day,

I've encountered strange problem and I'm trying to find solution few weeks. But nothing helps. We have remote office. The LAN consists of 3 switches cisco 2960 and router which is not under our administrative control. WAN link to this remote office terminates on one of the 2960's and then goes to rouoter in trunk.

Few times per hour (randomly) everything goes down and switches and everything that connected to it stops responding to the pings for 5-30 seconds. Then everything comes back and no messages in the syslogs of all devices (route also) that points to the problem.

I've tried to test provider's WAN link. Provider has created SVI 1.1.1.1/29 on his equipment in our routing center and SVI 1.1.1.2/29 on his equipement in the remote office. I've created SVI 1.1.1.4/29 on the switch in our remote office on which WAN link terminates. Then I've send approximately 1 million pings from SVI 1.1.1.4/29 to both SVIs on provider's equipment. And there were no lost pings. This says that WAN link and link from our 2960 to provider's equipment works fine.

Also during this outages cpu usage doesn't increase and second digit in the 5 second usage int the sh process cpu remains zero.

I've tried to upgrade firmware of 2960s, to change spanning tree mode to rstp, to monitor interface erros but nothing helped.

Have anybody also encountered something like this?

[ArchiTech89]

I'm not highly experienced, but a couple of points of clarification occur to me.

First, you say the WAN link terminates on a switch? That doesn't make sense to me, though again, I'm not a major pro. I would think the WAN link must terminate at a router. I don't know any other way. Is the WAN connected to a WAN port on the router, then the router connects to one of the 2960s? That would make more sense to me.

Second, when you say "everything" stops responding to pings, is that referring to pings from a central site coming over the WAN? Or are they pings locally? Do LAN-local pings still work (my hunch is that they don't...)?

Lastly, just from the description it almost sounds like a convergence issue -- that the network is trying to reconverge a "few times per hour." Again, I'm not the expert by any means, but could there perhaps be STP problems where election of the Root switch gets triggered more than once an hour? Perhaps you can try some spanning-tree deug commands or check counters for unusual statistics.

Sorry I can't be of more help...

[Aeroexpress]

Hello jeremyNLSO,

We cannot terminate WAN link directly to the router.Router is under administrative control of other organization and we would like to control this WAN link. Also router located in the other room than providers equipement. So we had to terminate WAN link as access port on the 2960, then this link goes in trunk to the router and returnes back as routed vlans.

I'm trying to send these ping from my office computer in central office so they goes through WAN link and also I've tried to send local pings: from SVI on one 2960 to another 2960's SVI, from 2960's SVI to router subinterface and so on. When ping from office computer dissapears I'm loosing SSH connection to 2960, but it restores after 3-5 sedonds. And I can't see any local loosage. But nevertheless it seems that WAN link works fine. Together with provider with tested it this way. On provider's router in the routing center of our company we've created SVI 1.1.1.1/29, on providers router in the remote office we've creaqted SVI 1.1.1.2/29 and on 2960 WAN link terminates on we've created SVI 1.1.1.4/29. Then i've issued 1000000 pings from 1.1.1.4 to both 1.1.1.1 and 1.1.1.2. And the were no loosage. This means for me that local link between our 2960 and provider's router works fine and WAN link from remote office to the routing center works fine too.

We're using RSTP in this LAN. But I've tried to monitor 'debug spanning-tree events' and nothing had changed.

Complexity of problem identification is that problem can apper few times per hour or one time per day. So I have to monitor different pings for the hole day.

Thank you for the help. I'll try to study RSTP topology.

[Rajeev Sharma]

Hey,

may you update the thread with following from edge 2960:

#show version

#show switch detail

#show span detail | i ieee|occurred|from

Regards,

RS.

[Aeroexpress]

Hello, rajeevsh,

I think I've found out what the problem was.

Distribution 2960 (let's name it 2960_2) was running PVST.

One edge (access) 2960 (let's name it 2960_1) was running RSTP.

One edge (access) 2960 (let's name it 2960_3) was running PVST. Also this switch had one port operating in half duplex which was constantly trying to provide PoE, however device on this interface is not PoE device.

So I made the following. Changed spanning-tree mode on all switches to PVST and switched off PoE on the interface with non-PoE device.

Problem dissappeared, but I have not fully understand fully the problem though trying to whatch spanning-tree debugs for few hours.

I hope you'll try to explain me the problem. It seems to me that problem was in the mismatch of STP modes.

Thank you.

[ArchiTech89]

Hello,

A question out of curiosity... How did you find out you had PVST and RSTP running simultaneously? Was it through the show spanning-tree command (actually, probably show spanning-tree summary right)? Or did you just do a show run and look for it? Default for Cisco switches is PVST (I'm pretty sure) so it might not have even shown up in the sh run command...

[Aeroexpress]

Hello,

I've checked sh spanning-tree vlan (number).

May be it is important. The switch performing RSTP was acting as stack of 2960s.

Do you have any ideas?

[Rajeev Sharma]

Hey,

The port working in half duplex mode may explain why you had spanning-tree issues in your network. Since that edge box was running PVST and any port going in forwarding is considered as topology change in this protocol so it must be reason for network issues.

Since upon reception of TCN the mac-address aging times is reduced a low value ans as this port was continuously trying to give power so its state kept changing and hence this part of network never recovered out of TCN and as you corrected the configuration it became normal.

For more information you may visit the following link:

http://www.cisco.com/c/en/us/support/docs/lan-switching/spanning-tree-protocol/12013-17.html#topology

HTH.

Regards,

RS.