Input errors on a DWDM link with good signal (-15.6) dBm

Josh Potter · ‎08-01-2013

I was hoping someone might be able to point me in the right direction. I just added an ECWDM solutoin with a 10 gig DWDM link which is X-2 on both sides. On my remote side I'm seeing input and CRC errors but with a good light level. I've changed out the amps, the X-2 taking errors, the DWDM mux so far. The near side isn't taking any errrors and the 2 CWDM circuits on this ring are not taking errors. Everything I read says this is a great signal for these optics. Any ideas?

Optical Optical

Temperature Voltage Tx Power Rx Power

Port (Celsius) (Volts) (dBm) (dBm)

--------- ----------- ------- -------- --------

Te1/6 25.8 N/A 1.7 -15.6

TenGigabitEthernet1/6 is up, line protocol is up (connected)

Hardware is Ten Gigabit Ethernet Port, address is e05f.b952.da45 (bia e05f.b952.da45)

Description: DWDM 59.79 (CRH) to cws2-nnd3-nno <Te6/5>

MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,

reliability 255/255, txload 1/255, rxload 1/255

Encapsulation ARPA, loopback not set

Keepalive set (10 sec)

Full-duplex, 10Gb/s, link type is auto, media type is 10G-DWDM-59.79

input flow-control is on, output flow-control is off

ARP type: ARPA, ARP Timeout 04:00:00

Last input 00:00:01, output never, output hang never

Last clearing of "show interface" counters 00:16:22

Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue: 0/40 (size/max)

5 minute input rate 6380000 bits/sec, 1643 packets/sec

5 minute output rate 0 bits/sec, 0 packets/sec

1603189 packets input, 780652652 bytes, 0 no buffer

Received 1600548 broadcasts (1524877 multicasts)

0 runts, 0 giants, 0 throttles

56 input errors, 56 CRC, 0 frame, 0 overrun, 0 ignored

0 input packets with dribble condition detected

49 packets output, 9456 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 babbles, 0 late collision, 0 deferred

0 lost carrier, 0 no carrier

0 output buffer failures, 0 output buffers swapped out

supoonia · ‎08-01-2013

Hi Josh,

DWDM is passive here right?

also you mentioned 2 other CDWDM circuit are running fine.

can we check if this third 10Gig which you added on MUX is properly balanced,

means on input of MUX all wavelength should have almost same power level.

so to start with, you can note down Tx of other two 10Gig from device interface and let us know.

so as we are getting CRC only on one side, so we should check power level between Tx------RxMUX at far end.

Wavelength of both interfaces have to be equal, can be verified from serial number of SFP/XFP.

check type of patchcord or cable.

also what is the distance between two nodes.

also what kind of device is this, can we get sh run only for interfaces from both ends and serial number of XFP (from sh inventory)

show controllers ethernet-controller tenGigabitEthernet 3/1/1

Josh Potter · ‎08-01-2013

Hi Sunil,

This setup is currently active with an amplifers but I have taken them out of the mix (this pushes the signal down to ~-22dBm. I'll check to see if the MUX is properly balanced.

I have a little better signal on my local side, but it's not seeing much traffic so that might be why I'm only seeing errors on one side. These are both the same DWDM and yes they're going into the right DWDM port on the MUX, I checked that. Attached is a picture of the simple ring setup I'm running.

NNO side (6500)

NAME: "Transceiver Te6/5", DESCR: "X2 Transceiver DWDM-59.79 Te6/5"

PID: DWDM-X2-59.79 , VID: V02 , SN: FLJ1621W007

Te6/5 32.1 N/A 97.4 -- 1.3 -12.3

WCH side (4900) (errrors here)

NAME: "TenGigabitEthernet1/6", DESCR: "DWDM-59.79"

PID: DWDM-X2-59.79 , VID: V02 , SN: FLJ1621W006

Te1/6 25.8 N/A 1.8 -16.1

supoonia · ‎08-01-2013

maybe try loopbacks with 10dB attenuator on interfaces to check if XFPs are good.

Josh Potter · ‎08-01-2013

I've already swithced out both XFP's. I've got some the same setup running on my desk (with 20db of attenuators to simulate a longer run) and I can't get them to take any errors, even with a worse signal.

My current plan is to scope and clean every link on this path, run an OTDR test to see if there are any odd relfections and if that doesn't work a local telecom has an opictal spectrum analyizer I'll have to borrow and test the specific wavelenghts. I'll be sure to report my findings.

supoonia · ‎08-01-2013

ok and if there is any tunable XFP with you, than you can try switching on another wavelength.

Leo Laohoo · ‎08-01-2013

Can you post the output to the command "sh counter interface Ten 1/6" and "sh idp interface Ten 1/6 det"?

Josh Potter · ‎08-01-2013

Right now these are the only DWDM optics I have on hand.

Leo, the 4900M doesn't support that command (just like the controller command sunil mentioned above), but I think this is similar.

csw1-comm-crh#show int te 1/6 counters all

Port InBytes InUcastPkts InMcastPkts InBcastPkts

Te1/6 27313185218 168240 51647298 2343690

Port OutBytes OutUcastPkts OutMcastPkts OutBcastPkts

Te1/6 337653 0 1727 0

Port InPkts 64 OutPkts 64 InPkts 65-127 OutPkts 65-127

Te1/6 1150 1152 6525179 0

Port InPkts 128-255 OutPkts 128-255 InPkts 256-511 OutPkts 256-511

Te1/6 3760097 0 35515121 575

Port InPkts 512-1023 OutPkts 512-1023

Te1/6 1184452 0

Port InPkts 1024-1518 OutPkts 1024-1518 InPkts 1519-1548 OutPkts 1519-1548

Te1/6 696964 0 6478202 0

Port InPkts 1549-9216 OutPkts 1549-9216

Te1/6 0 0

Port Tx-Bytes-Queue-1 Tx-Bytes-Queue-2 Tx-Bytes-Queue-3 Tx-Bytes-Queue-4

Te1/6 337653 0 0 0

Port Tx-Bytes-Queue-5 Tx-Bytes-Queue-6 Tx-Bytes-Queue-7 Tx-Bytes-Queue-8

Te1/6 0 0 0 0

Port Tx-Drops-Queue-1 Tx-Drops-Queue-2 Tx-Drops-Queue-3 Tx-Drops-Queue-4

Te1/6 0 0 0 0

Port Tx-Drops-Queue-5 Tx-Drops-Queue-6 Tx-Drops-Queue-7 Tx-Drops-Queue-8

Te1/6 0 0 0 0

Port Rx-No-Pkt-Buff RxPauseFrames TxPauseFrames PauseFramesDrop

Te1/6 0 0 0 0

Port UnsupOpcodePause

Te1/6 0

Port CrcAlign-Err Dropped-Bad-Pkts Collisions Symbol-Err

Te1/6 1926 0 0 708866

Port Undersize Oversize Fragments Jabbers

Te1/6 0 0 0 17

Port Single-Col Multi-Col Late-Col Excess-Col

Te1/6 0 0 0 0

Port Deferred-Col False-Car Carri-Sen Sequence-Err

Te1/6 0 0 0 296

Port RxIslTagFrames TxIslTagFrames RxDot1qTagFrames TxDot1qTagFrames

Te1/6 0 0 54157484 0

Leo Laohoo · ‎08-02-2013

Thanks for the output. It helps alot.

Line errors are in the bottom part of the output, starting with CRC/Alignment Errors. This means it's a RECEIVE error.

It's not a congestion error but a Layer 1 error. Be prepared to consider the following:

1. Change the fibre optic patch cable at both sides;

2. Observe;

3. Change the X2 at both sides;

4. Observe;

5. Move the X2 modules to a different ASIC; and

6. Observe.