Link failure WDM4Gxxxx Gbics

eggerste01 · ‎06-01-2010

in our company we have two locations for the Storage`s with a distance about 15km.

When we moved last week we saw that the connection speed of one Fabric ist just 2Gb - not the 4Gb we want.

We`r using on both locations and both fabrics MDS9124 FC Switches, CWDM-MUX8A, connection between the locations is dark fibre.

The uplink to the fc switch of one fabric works fine with the 4GB GBics CWDM4G1470 and 1490 wavelenght.

The second fabric uplink (it takes another, independent way to 2nd location) will only work with 2Gb CWDM-1510 and 1530 GBics.

If we install CWDM4G1470 and 1490 Gbics (on both locations) we just get a link failure on the FC Switch.

The GBics are OK - we changed them with the other Fabric one. We also changed all Single Mode FC cables before/after the MUX8A - without success

Also after changing buffer credits - no link.

Can anyone help?

Tom Randstrom · ‎06-01-2010

Do you have any optical test equipment (optical power meter and optical light source)? It would be good to know what the actual loss is through the CWDM mux link which fails with the 4G FC. It is possible the link has a higher loss than anticipated.

eggerste01 · ‎06-01-2010

No sorry - we do not have optical test equipment.

If i don't have a link, will be trying other BB_Credits than default a option? Are BB_credits just for optimizing ab existing link?

Tom Randstrom · ‎06-01-2010

Based upon the info I could find, at 4Gb, a rule of thumb is 1 buffer credit per 1/2 km of distance. Thus, your 15km link would require 30 buffer credits. I do not know what the default buffer credit is on the MDS 9124.

viyuan700 · ‎06-01-2010

I think the problem is not with your buffer credits but with Optical power you get for 4 G cards. see the link below and see what card you have and the optical parameters. Link budget at at 4G is 17.8 dB where as for 2G it is approx. 28dB(so you have more power so your link works).

Ask your fiber provider to give you fiber budget for the dark fiber or if it is yours then arrange a power meter. Check if the total loss is less than 17.8dB if it is more your link will not work for 4G. You need an optical amplification or you have to think of some other way.

http://cisco.biz/en/US/prod/collateral/ps4159/ps6409/ps4358/product_data_sheet09186a00801bc698.html

eggerste01 · ‎06-02-2010

THX for this hint Viyuan - this seems to be more realistic than buffer credits.

We will try this afternoon a direct link - if this also fails we know that there has to be an network failure.

I will report

eggerste01 · ‎06-03-2010

Direct Link works with the 4GB GBic´s

So one of the two MUX must have a failure.

THX

Stefan

viyuan700 · ‎06-04-2010

CWDm are very simple devices and I dont think that only one port will have problem as you told 2Gb is working (Possibililty are less if it works for other ports).

I think CWDM muxes have 3-4dB of loss and since 4G have less power so may be you are not getting enough power at the other end so 4G link not working . Add up all the losses in your link (Patch cable, Fiber, Mux etc ) and if the power budget is within the limits of 4G receiver see if problem with any patch cable.

Tom Randstrom · ‎06-04-2010

Stefan,

I do remember discussing an issue (not too long ago) with a user that was having with their CWDM muxes. It turned out that the CWDM muxes were incorrectly assembled (factory problem) so that the wavelengths on the faceplate were not correct (cross-fibered). The customer took their wavelength specific CWDM transceiver and had to test each port to see which one would pass the signal. Thus, when the customer finally connected the transceiver to the muxes, the wavelength on the transceivers did not match to wavelength labeled on the ports. I wonder if this is the problem you may be experiencing.

This solution should work with a 15km span.

16.7 dB link budget (+1 TX - -15.7dB RX) - 4dB fiber loss - 4.4dB mux loss (pair) - 1dB connector loss - 3dB EOL Safety Margin = ~4.3dB working margin

Hope this helps!

Tom

eggerste01 · ‎06-15-2010

Hi Tom,

last week we found out, that the optical panel from the provider has an defect on a connector.

The provider told us to clean each port of the MUX an the Uplink Panel and there we found it.

They changed the panel an everything works now on 4 gBit

THX

Stefan

Tom Randstrom · ‎06-15-2010

Stefan,

Thank you very much for posting the outcome. It helps us all learn from the situation.

Tom