Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

WS-X6704-10GE packet loss

Hello,

I have a really strange problem. I'm trying to setup a 10G 802.1q trunk port between 2 buildings and I'm getting a strange packet loss issue. The connections will start fine and then just stop. Router 1 will send packets to router 2 that are received but the packets from router 1 will start disappearing on the way to router 2. I've even setup a tftp-server on both routers and copied files between them. The same results. The problem lies somewhere between the 2 routers.

This only occurs on the TE ports. The GE ports I've been using are fine. I've also put a light meter on the fiber and it checks out. It's single mode with LR transceivers. The run is about 800 feet. My equipment is OLD so direct TAC support is unavailable.

Hardware

Building 1:

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   16  16 port 1000mb GBIC ethernet           WS-X6416-GBIC     

  2   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

  3   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

  4   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

  5   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

  6   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

  7    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE    

  8    2  Supervisor Engine 720 (Hot)            WS-SUP720-BASE    

  9   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

10   48  48-port 10/100 mb RJ45                 WS-X6148-RJ-45    

11    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE     

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  0005.740c.59f8 to 0005.740c.5a07   2.0   5.4(2)       12.2(33)SXI9 Ok

  2  000c.ce27.3f18 to 000c.ce27.3f47   1.2   5.4(2)       12.2(33)SXI9 Ok

  3  0012.4304.5e90 to 0012.4304.5ebf   3.1   5.4(2)       12.2(33)SXI9 Ok

  4  0012.0182.2228 to 0012.0182.2257   3.1   5.4(2)       12.2(33)SXI9 Ok

  5  0015.c667.a944 to 0015.c667.a973   5.0   8.3(1)       12.2(33)SXI9 Ok

  6  000c.ce27.41b8 to 000c.ce27.41e7   1.2   5.4(2)       12.2(33)SXI9 Ok

  7  0013.7f0d.a5c0 to 0013.7f0d.a5c3   3.3   8.1(3)       12.2(33)SXI9 Ok

  8  0011.92e6.fa08 to 0011.92e6.fa0b   3.1   8.1(3)       12.2(33)SXI9 Ok

  9  0016.9d53.aeb0 to 0016.9d53.aedf   5.1   8.3(1)       12.2(33)SXI9 Ok

10  0012.016a.f6e8 to 0012.016a.f717   3.1   5.4(2)       12.2(33)SXI9 Ok

11  8843.e1aa.b978 to 8843.e1aa.b97b   3.1   12.2(14r)S5  12.2(33)SXI9 Ok

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  7  Policy Feature Card 3       WS-F6K-PFC3A                    2.5    Ok

  7  MSFC3 Daughterboard         WS-SUP720                       2.5    Ok

  8  Policy Feature Card 3       WS-F6K-PFC3A                    2.4    Ok

  8  MSFC3 Daughterboard         WS-SUP720                       2.3    Ok

11  Distributed Forwarding Card WS-F6700-DFC3A                  2.2    Ok

Building 2:

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   48  48 port 10/100 mb RJ21 Enhanced QoS    WS-X6248A-TEL     

  2   48  48 port 10/100 mb RJ21 Enhanced QoS    WS-X6248A-TEL     

  3   48  48 port 10/100 mb RJ21 Enhanced QoS    WS-X6248A-TEL     

  4   48  48 port 10/100 mb RJ-21 ethernet       WS-X6248-TEL      

  5    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE    

  6    2  Supervisor Engine 720 (Hot)            WS-SUP720-BASE    

  7    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE     

  8   16  16 port 1000mb GBIC ethernet           WS-X6416-GBIC     

  9   96  96-port 10/100 Mbps RJ45               WS-X6196-21AF     

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  0030.b63c.df90 to 0030.b63c.dfbf   1.0   5.1(1)CSX    8.5(0.46)RFW Ok

  2  0005.9a8a.9118 to 0005.9a8a.9147   2.0   5.4(2)       8.5(0.46)RFW Ok

  3  0001.9754.6d80 to 0001.9754.6daf   1.0   5.1(1)CSX    8.5(0.46)RFW Ok

  4  00d0.d331.f118 to 00d0.d331.f147   1.0   4.2(0.24)VAI 8.5(0.46)RFW Ok

  5  0011.21a1.587c to 0011.21a1.587f   3.2   8.1(3)       12.2(18)SXF1 Ok

  6  0013.7f0a.7004 to 0013.7f0a.7007   3.3   8.1(3)       12.2(18)SXF1 Ok

  7  0019.aaf1.dcf8 to 0019.aaf1.dcfb   2.5   12.2(18r)S1  12.2(18)SXF1 Ok

  8  0009.b794.1400 to 0009.b794.140f   2.3   5.4(2)       8.5(0.46)RFW Ok

  9  0019.306d.d796 to 0019.306d.d7f5   1.4   8.2(2)       8.5(0.46)RFW Ok

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  5  Policy Feature Card 3       WS-F6K-PFC3A                    2.4    Ok

  5  MSFC3 Daughterboard         WS-SUP720                       2.4    Ok

  6  Policy Feature Card 3       WS-F6K-PFC3A                    2.5    Ok

  6  MSFC3 Daughterboard         WS-SUP720                       2.5    Ok

  7  Distributed Forwarding Card WS-F6700-DFC3A                  2.2    Ok

  9  IEEE Voice Daughter Card    WS-F6K-FE48X2-AF                1.4    Ok

Below is the relavant configuration from both routers. I only pasted 1 copy as its identical on both except for slot differences on the ports. The only thing I do to cause the packet loss is move the fiber from the GE ports to the TE ports and everything falls apart. Moving it back to the GE ports puts everything back to normal.

spanning-tree mode rapid-pvst

spanning-tree portfast default

interface GigabitEthernet5/1

switchport

switchport trunk encapsulation dot1q

switchport mode trunk

mtu 9216

no ip address

mls qos trust dscp

end

interface TenGigabitEthernet7/3

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 1-999

switchport mode trunk

mtu 9216

no ip address

shutdown

mls qos trust dscp

end

I'm excluding vlan 1000 on the TE port since it has storage server traffic on it. The other vlans can handle some packet loss for short periods while I'm testing. The storage servers cannot.

The really strange thing about this is I get ZERO error counters.

Router 1:

show int

TenGigabitEthernet11/3 is down, line protocol is down (notconnect)

  Hardware is C6k 10000Mb 802.3, address is 8843.e1aa.b97a (bia 8843.e1aa.b97a)

  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 0/255, rxload 0/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 10Gb/s, media type is 10Gbase-LR

  input flow-control is off, output flow-control is off

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:39:30, output 00:39:30, output hang never

  Last clearing of "show interface" counters 11w1d

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     1756250870 packets input, 276893128198 bytes, 0 no buffer

     Received 4652610 broadcasts (2814099 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     2287076473 packets output, 1940759519804 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE output

     0 output buffer failures, 0 output buffers swapped out

Show fabric errors

Module errors:

slot    channel        crc      hbeat       sync   DDR sync

    7          0          0          0          0          0

    8          0          0          0          0          0

   11          0          0          0          0          0

   11          1          0          0          0          0

Fabric errors:

slot    channel       sync     buffer    timeout

    7          0          0          0          0

    8          0          0          0          0

   11          0          0          0          0

   11          1          0          0          0

Router 2:

Show int

TenGigabitEthernet7/3 is administratively down, line protocol is down (disabled)

  Hardware is C6k 10000Mb 802.3, address is 0019.aaf1.dcfa (bia 0019.aaf1.dcfa)

  MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,

     reliability 255/255, txload 0/255, rxload 0/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 10Gb/s

  input flow-control is off, output flow-control is off

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:37:27, output 00:36:55, output hang never

  Last clearing of "show interface" counters 12w2d

  Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 0 bits/sec, 0 packets/sec

  5 minute output rate 0 bits/sec, 0 packets/sec

     2287095723 packets input, 1940873532153 bytes, 0 no buffer

     Received 312018 broadcasts (298729 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 0 multicast, 0 pause input

     0 input packets with dribble condition detected

     1756240845 packets output, 276904112065 bytes, 0 underruns

     0 output errors, 0 collisions, 1 interface resets

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 PAUSE output

     0 output buffer failures, 0 output buffers swapped out

Show fabric errors

Module errors:

slot    channel        crc      hbeat       sync   DDR sync

    5          0          0          0          0          0

    6          0          0          0          0          0

    7          0          0          0          0          0

    7          1          0          0          0          0

Fabric errors:

slot    channel       sync     buffer    timeout

    5          0          0          0          0

    6          0          0          0          0

    7          0          0          0          0

    7          1          0          0          0

Hopefully someone can help me shed some light on this as its the strangest problem I've ever seen.

Thanks,

Dave

307
Views
0
Helpful
0
Replies
CreatePlease login to create content