High Latency issue to gateway switch

hcspl · ‎10-06-2015

Dear Team,

i have three nos of catalyst 2960 switch,i have connected these switches in trunk mode.after that i have connected my main uplink to one of my cisco 2960 switch ( the another end of uplink is connected to Nortel Layer three switch,whose ip is : 192.168.4.2)

Now the issue is that when i ping from my local machine which is connected to 2960 switch i am getting high latency sometimes get Request time out also.where as afte some time of interval it works smoothly for some time.after 2 3 min latency will get increased automatically.

below is the ping Response:

Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time=1ms TTL=64
Reply from 192.168.4.2: bytes=32 time=2ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Request timed out.
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time=80ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time=54ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64m 192.168.4.2: bytes=32 time=213ms TTL=64
Reply from 192.168.4.2: bytes=32 time=331ms TTL=64
Reply from 192.168.4.2: bytes=32 time=253ms TTL=64
Reply from 192.168.4.2: bytes=32 time=202ms TTL=64
Reply from 192.168.4.2: bytes=32 time=220ms TTL=64
Reply from 192.168.4.2: bytes=32 time=256ms TTL=64
Reply from 192.168.4.2: bytes=32 time=320ms TTL=64
Reply from 192.168.4.2: bytes=32 time=176ms TTL=64
Reply from 192.168.4.2: bytes=32 time=174ms TTL=64
Reply from 192.168.4.2: bytes=32 time=519ms TTL=64
Reply from 192.168.4.2: bytes=32 time=437ms TTL=64
Reply from 192.168.4.2: bytes=32 time=190ms TTL=64
Reply from 192.168.4.2: bytes=32 time<1ms TTL=64
Reply from 192.168.4.2: bytes=32 time=185ms TTL=64
Reply from 192.168.4.2: bytes=32 time=273ms TTL=64
Reply from 192.168.4.2: bytes=32 time=221ms TTL=64
Reply from 192.168.4.2: bytes=32 time=106ms TTL=64
Reply from 192.168.4.2: bytes=32 time=104ms TTL=64
Reply from 192.168.4.2: bytes=32 time=103ms TTL=64

please help me to sort out this issue.

below is my switch config:

!
!
!
!
spanning-tree mode rapid-pvst
spanning-tree extend system-id
no spanning-tree vlan 6
!
vlan internal allocation policy ascending
!
vlan 4,6
!
!
!
!
!
!
interface FastEthernet0/1
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/2
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/3
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/4
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/5
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/6
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/7
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/8
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/9
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/10
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/11
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/12
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/13
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/14
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/15
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/16
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/17
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/18
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/19
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/20
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/21
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/22
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/23
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface FastEthernet0/24
switchport access vlan 4
switchport mode access
storm-control broadcast level 20.00 10.00
!
interface GigabitEthernet0/1
switchport mode trunk
spanning-tree mst pre-standard
!
interface GigabitEthernet0/2
switchport mode trunk
!
interface Vlan1
no ip address
shutdown
!
interface Vlan4
ip address 192.168.5.7 255.255.254.0
!
interface Vlan6
no ip address
shutdown
!
ip default-gateway 192.168.4.2
ip http server
ip http secure-server
!

Awaiting for favorable answer.............

Attached here with Network Diagram

Steve Fuller · ‎10-07-2015

Hi,

This could be an issue with over-utilisation or errors on the links between any of the switches shown, or it could simply be that this is expected behaviour when pinging to the Nortel switch.

What is the core switch with the 192.168.4.1 address and do you see the same results if you ping that device?

What results do you see if you ping the Catalyst 2960 switches themselves? They each have an IP address in the same VLAN so traffic to them would go directly and not via the default gateway.

Which switch is your PC connected to? This is not shown in the diagram. Check each interface in the path between the PC and the Nortel switch and ensure you have no errors or that the link itself is not carrying large volumes of traffic.

One thing to consider here is that, depending upon where your PC is connected, the ICMP traffic will pass through one Catalyst switch and one GE link before reaching the Nortel switch, or as many as three Catalyst switches and three GE link before reaching the Nortel.

If you don’t see any errors, none of the links are congested, and ping response from other devices is OK, then ask yourself what is different about the Nortel. Does it have some form on control plane policing such that ICMP traffic to the switch itself might be dropped? We see this on some Cisco platforms such as the Nexus family switches.

Regards

hcspl · ‎10-07-2015

Dear Steve,

Thank for your Prompt Reply.

My machine is connected to 192.168.5.7 switch.

I am getting proper ping reply for 192.168.4.1 and 192.168.1.3

now the issue is that when i plugged my uplink to nortel latency will get increased for entire organisation network.and if i removed the uplink entire network works smoothly

I have attached a screenshot of ping response.

please help me to sort out this

Steve Fuller · ‎10-07-2015

Hi,

"My machine is connected to 192.168.5.7 switch. I am getting proper ping reply for 192.168.4.1 and 192.168.1.3"

That would seem to rule out any link errors or congestion on the links.

"now the issue is that when i plugged my uplink to nortel latency will get increased for entire organisation network.and if i removed the uplink entire network works smoothly"

I'm not clear as to the changes you're making. Are you saying that the uplink from the Catalyst switch is not normally connected to the Nortel switch, and in that configuration everything is working OK?

The problem occurs when you connect the Catalyst switch to the Nortel switch as indicated in the diagram you posted earlier, and you see latency is increased? Is the latency increased for all destinations, including the 192.168.4.1 and the 192.168.1.3 that work when the Nortel is not in path?

Can you indicate on the diagram you posted earlier what the topology is before and after the change you're making?

Regards

hcspl · ‎10-07-2015

is it possible to you to take my remote session.

else i have to explain you briefly

hcspl · ‎10-07-2015

Dear Steve,

when i plugged the Uplink the latency increased for only one destination i.e. 192.168.4.2.

Steve Fuller · ‎10-07-2015

It's still not clear to me what topology change you are making.

When you say "when i plugged the Uplink" I assume you mean you're connecting the link from the Catalyst 2960 with IP 192.168.5.7 to the Nortel switch. Please confirm I've understood this correctly.

If the above statement is correct, where is the uplink from the Catalyst 2960 connected before you connect to the Nortel switch?

You also state "latency increased for only one destination i.e. 192.168.4.2"

Does that mean with the Catalyst 2960 connected to the Nortel, you can ping from a PC connected to the Catalyst switch through the Nortel switch to the IP address 192.168.4.1 and the 192.168.1.3 ad receive normal response and latency?

Regard

hcspl · ‎10-07-2015

When you say "when i plugged the Uplink" I assume you mean you're connecting the link from the Catalyst 2960 with IP 192.168.5.7 to the Nortel switch. Please confirm I've understood this correctly.

yes.its correct.

see i have directly connected cisco 2960 to nortel switch through utp cable.

and your another statement is also right.i am getting proper ping response for 192.168.4.1 and 1.3

for 1.3 and 4.1 traffic is routed through 192.168.4.2 i.e.nortel switch

Steve Fuller · ‎10-07-2015

Then can you also confirm, when the catalyst 2960 is connected to the Nortel switch, the only problem you are experiencing is poor reponse for ping destined to the Nortel switch?

Any traffic routed through the Nortel switch to destinations beyond has no issues?

If that is the case, then the issue could be as I mentioned in my first response:

"If you don’t see any errors, none of the links are congested, and ping response from other devices is OK, then ask yourself what is different about the Nortel. Does it have some form on control plane policing such that ICMP traffic to the switch itself might be dropped? We see this on some Cisco platforms such as the Nexus family switches."

Regards

hcspl · ‎10-08-2015

Dear Steve,

Can you please let me know what configuration is to be checked on nortel.as i have limited access of nortel switch.

Steve Fuller · ‎10-08-2015

Sorry, but I have no knowledge of the Nortel switch. I'd suggest raising the question with the person that supports the Nortel switch or post the question to a Nortel forum.

Regards

Andre Neethling · ‎10-07-2015

This sounds like a spanning tree issue. Did you do any STP tuning in the network, or is everything still at it's default settings? Can you please provide the output of "show spanning-tree" on the 3 catalyst switches. What device is the core switch at 192.168.4.1. Where are your gateways for each VLAN residing?

hcspl · ‎10-07-2015

CAP2# show spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address bcc4.9361.3080
Cost 4
Port 26 (GigabitEthernet0/2)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address bcc4.9361.4180
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/2 Root FWD 4 128.26 P2p

VLAN0004
Spanning tree enabled protocol rstp
Root ID Priority 32772
Address bcc4.9361.3080
Cost 4
Port 26 (GigabitEthernet0/2)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32772 (priority 32768 sys-id-ext 4)
Address bcc4.9361.4180
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Fa0/1 Desg FWD 19 128.1 P2p
Fa0/2 Desg FWD 19 128.2 P2p
Fa0/3 Desg FWD 19 128.3 P2p
Fa0/5 Desg FWD 19 128.5 P2p
Fa0/6 Desg FWD 19 128.6 P2p
Fa0/7 Desg FWD 19 128.7 P2p
Fa0/8 Desg FWD 19 128.8 P2p
Fa0/11 Desg FWD 19 128.11 P2p
Fa0/13 Desg FWD 19 128.13 P2p
Fa0/14 Desg FWD 19 128.14 P2p
Fa0/15 Desg FWD 19 128.15 P2p
Fa0/16 Desg FWD 19 128.16 P2p
Gi0/1 Desg FWD 4 128.25 P2p
Gi0/2 Root FWD 4 128.26 P2p

hcspl · ‎10-07-2015

CAP1#sh sp
CAP1#sh spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address bcc4.9361.3080
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address bcc4.9361.3080
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- ------------------------------
Gi0/1 Desg FWD 4 128.25 P2p

VLAN0004
Spanning tree enabled protocol rstp
Root ID Priority 32772
Address bcc4.9361.3080
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32772 (priority 32768 sys-id-ext 4)
Address bcc4.9361.3080
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Aging Time 300 sec

Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- ------------------------------
Fa0/9 Desg FWD 19 128.9 P2p
Fa0/10 Desg FWD 19 128.10 P2p
Fa0/12 Desg FWD 19 128.12 P2p
Fa0/13 Desg FWD 19 128.13 P2p
Fa0/14 Desg FWD 19 128.14 P2p
Fa0/15 Desg FWD 19 128.15 P2p
Fa0/16 Desg FWD 19 128.16 P2p
Fa0/19 Desg FWD 19 128.19 P2p
Fa0/20 Desg FWD 19 128.20 P2p

Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- ------------------------------

Fa0/23 Desg FWD 19 128.23 P2p
Fa0/24 Desg FWD 19 128.24 P2p
Gi0/1 Desg FWD 4 128.25 P2p

Andre Neethling · ‎10-07-2015

It seems as if the switch "CAP1" is your STP root for VLAN4, but your gateway lives on the Nortel switch. Can you set the spanning tree priority on the Nortel switch to a lower value? Specifically for VLAN4. I am not aware of this switch configuration so I don't know how to check or configure it. Your network thinks that switch "CAP1" is the root of the network, so it sends all layer 2 traffic to "CAP1" then "CAP1" sends it to the default gateway at the Nortel switch. The traffic is not routed efficiently at layer 2 in your network. I would configure the Nortel switch to be the STP root bridge for VLAN4 if the VLAN4 gateway is on the Nortel Switch.