cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3310
Views
0
Helpful
11
Replies

Packet Latency on Catalyst 3560

imanco671
Level 1
Level 1

Hello Community,

I have installed a Catalyst 3560 as my internal router. I have created 4 vlans to route the traffic.

Here are the interface "show run"

vlan internal allocation policy ascending

!

interface FastEthernet0/1

switchport access vlan 2

switchport mode access

spanning-tree portfast

!

interface FastEthernet0/2

switchport access vlan 3

switchport mode access

spanning-tree portfast

!

interface FastEthernet0/3

switchport access vlan 4

switchport mode access

spanning-tree portfast

I have been getting notifications of large packet loss and latency for itnernal traffic going to servers. I have a simple setup of nagios who pings servers and will notify me of large packet loss or complete packet loss. I have implemented this L3 switch a few hours ago, I am currently running a constant ping to my servers, but I am not niticing packet loss right now.

This packet loss happened 4 time spans within the last hour of many large packet losses to all my servers within all subnets. It is now stopped.

I am not sure why this would be happening, the predessor of the internal router was just a normal linux box with 3 NICs on it. Nothing crazy going on since the business is closed.

Thanks in Advance!

2 Accepted Solutions

Accepted Solutions

Reza Sharifi
Hall of Fame
Hall of Fame

John,

The issue may not be your switch.  It might be the server or the NICs.  Besides the usual ping, you can also ping using larger packet size and see if you see packet drops.

example:

first clear the interface

clear interface gigabitEthernet x/x/x

then

ping 192.168.1.10 repeat 500 size 1500

Then check the interface for packet drops

sh ip int gi x/x/x/

HTH

View solution in original post

Hi John,

Your 3560 FE0/2 port stats are fine. I wouldn't change anything for now.

Are you still encountering the high latency and packet drop issue? I would suggest to perform SPAN and sniff with Whireshark if you do.

Sent from Cisco Technical Support iPhone App

View solution in original post

11 Replies 11

Reza Sharifi
Hall of Fame
Hall of Fame

John,

The issue may not be your switch.  It might be the server or the NICs.  Besides the usual ping, you can also ping using larger packet size and see if you see packet drops.

example:

first clear the interface

clear interface gigabitEthernet x/x/x

then

ping 192.168.1.10 repeat 500 size 1500

Then check the interface for packet drops

sh ip int gi x/x/x/

HTH

HTH,

I did not need to set a larger packet to see the issue. I have been watching it and noticed that all my pings to other subnets all went "out of wack" and started having response times up to 900 and even a couple time outs. This happened at the same time for all the constant pings to the other subnets.

I have a Constant. This constant is a ping to another server within the same subnet. That constant was not affected while all the other pings went nuts.

This never happened until I installed the L3.

I am seeing more and more sets of 10 packets that go from less than 1 up to 300-900 in a row. A few packets may drop too.

Thanks

John

hi john,

you may want to check the interface stats on the 3560 which connects to your servers. look under the txload and rxload. i suspect this maybe due to a link congestion. also, ensure 3560 FE ports and servers are hardcoded the same speed/duplex.

Hi John,

I have checked the ports and the txload and the rxload. Below is the output of my command: show interface FastEthernet 0/2

I have also inserted a picture of my Dell Powerconnect switch that is hooked up to my Catalyst. Is there something special I should do to connect the two. I noticed that my catalyst is using Full-Duplex 100Mb/s and my switch is using a 1000M, but shoudl that 1000M downgrade to 100M? My servers are mostly 100M too.

#show interface fastethernet 0/2

FastEthernet0/2 is up, line protocol is up (connected)

  Hardware is Fast Ethernet, address is 0007.0e76.0c84 (bia 0007.0e76.0c84)

  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,

     reliability 255/255, txload 90/255, rxload 40/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 100Mb/s, media type is 10/100BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:01, output 00:00:05, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 15735000 bits/sec, 4477 packets/sec

  5 minute output rate 35624000 bits/sec, 3555 packets/sec

     169356269 packets input, 76279088303 bytes, 0 no buffer

     Received 53247 broadcasts (0 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 36940 multicast, 0 pause input

     0 input packets with dribble condition detected

     132627013 packets output, 160530213511 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE output

     0 output buffer failures, 0 output buffers swapped out

Hi John,

Your 3560 FE0/2 port stats are fine. I wouldn't change anything for now.

Are you still encountering the high latency and packet drop issue? I would suggest to perform SPAN and sniff with Whireshark if you do.

Sent from Cisco Technical Support iPhone App

Hello John,

Yes I am still having issues.

Last night it was between 7:30 and 10:00. I get these Nagios packet loss notification emails and my blackberry blew up between those times. Then everything stopped. I noticed that the night before it happened at the same time.

Remember that all this happened after I installed this L3 switch which replaced a Linux Server with 3 NICs in it.

I am confused and dont know where to look. I do not know much about sniffing or wireshark.

John

Packet sniffing could be a lot of work for you if that's the case.

I would go back to what Reza had post. Did you find anything unusual with the server logs?

Sent from Cisco Technical Support iPhone App

Hi John,

I just experienced latency for a few minutes about 30 minutes ago. I have no idea what is going on. I was getting alerts from Nagios and users were experiencing latency to server.

I have checked event logs from servers and everything looks okay.

I am running wireshark on a server. Anything I should look for?

network acting up again. Large amounts of latency. Connections to servers freezing even

So things were getting so bad in my network that I had to revert to the old Linux server router.

I have a server constantly pinging all the subnets.

When on the L3 switch, most packets were <1ms, but a few packets would be <5ms - <50ms, it would be mixed.

But now that I reverted back to my old linux box, I see that all the packets are <1ms.

Things are normal now, but now I know that the L3 switch was the culprit.

Why would my L3 cause such issues? How can I diagnose it?

could my VLANS be causing this issue?

Below is my command output:

arisrouter>show vlan

VLAN Name                             Status    Ports

---- -------------------------------- --------- -------------------------------

1    default                          active    Fa0/5, Fa0/6, Fa0/7, Fa0/8

                                                Fa0/9, Fa0/10, Fa0/11, Fa0/12

                                                Gi0/1, Gi0/2

2    VLAN0002                         active    Fa0/1, Fa0/13, Fa0/14, Fa0/15

                                                Fa0/16

3    VLAN0003                         active    Fa0/2, Fa0/17, Fa0/18, Fa0/19

                                                Fa0/20

4    VLAN0004                         active    Fa0/3, Fa0/21, Fa0/22, Fa0/23

                                                Fa0/24

5    VLAN0005                         active    Fa0/4

1002 fddi-default                     act/unsup

1003 token-ring-default               act/unsup

1004 fddinet-default                  act/unsup

1005 trnet-default                    act/unsup

VLAN Type  SAID       MTU   Parent RingNo BridgeNo Stp  BrdgMode Trans1 Trans2

---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------

1    enet  100001     1500  -      -      -        -    -        0      0

2    enet  100002     1500  -      -      -        -    -        0      0

3    enet  100003     1500  -      -      -        -    -        0      0

VLAN Type  SAID       MTU   Parent RingNo BridgeNo Stp  BrdgMode Trans1 Trans2

---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------

4    enet  100004     1500  -      -      -        -    -        0      0

5    enet  100005     1500  -      -      -        -    -        0      0

1002 fddi  101002     1500  -      -      -        -    -        0      0

1003 tr    101003     1500  -      -      -        -    -        0      0

1004 fdnet 101004     1500  -      -      -        ieee -        0      0

1005 trnet 101005     1500  -      -      -        ibm  -        0      0

Remote SPAN VLANs

------------------------------------------------------------------------------

Primary Secondary Type              Ports

------- --------- ----------------- ------------------------------------------

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco