×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

Fibre Link Length and Etherchannel

Unanswered Question
Feb 17th, 2014
User Badges:

Morning folks,


Had an interesting question put to us which I've not come across before.  Assuming I have a two channel Etherchannel consisting of multimode OM3 fibre,  One pair being 100 meters and the other being 300 meters would this cause any issues with the performance of the Etherchannel.  My thinking was that provided both pairs could negotiate the same speed and duplex settings the only issue would be that the longer distance may have potentially a slower transfer rate than the shorter length but I wasn't sure.


Can anybody point me in the direction of a suitable Cisco document or otherwise that may be able to advise on this?


TIA


Jamie

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Peter Paluch Mon, 02/17/2014 - 05:15
User Badges:
  • Cisco Employee,

Hi Jamie,


Do not confuse the bandwidth and the latency. They are related but not in a strict way that would determine one by the other. The link with the 300m fiber will have a higher latency but the amount of data it is able to carry within a second is the same as the 100m link - just the data will start "pouring out" on the other end sooner or later, depending on the length.


On Cisco switches, EtherChannel always steers entire frame flows over the physical link in the bundle. This means that an entire conversation (as defined by hashing the selected fields from the frames, check the show etherchannel load-balance command to check it) is forwarded through a given link. Therefore, because different frame flows are, in general, independent, so should be the choice of the link. The fact that one of them has a higher latency should not be significant.


My two cents...


Best regards,

Peter

Joseph W. Doherty Mon, 02/17/2014 - 05:29
User Badges:
  • Super Bronze, 10000 points or more

Disclaimer


The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.


Liability Disclaimer


In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.


Posting


As Peter noted, Etherchanneling keeps all flows on the same link, so as he also noted, the flow on the longer link would have some additional latency.  However, also keep in mind that for a delta of 200m, the fiber, itself, would only add about 1 microsecond of additional latency.  For most real-world applications, this additional latency is a non-issue.

Jim R Mon, 02/17/2014 - 05:37
User Badges:

Hi Peter,


Thanks for the reply,  I realised the overwhelming mistake in my orginal question as soon as I read the first line of your reply Getting my fibre channel and ethernet muddled up.  Teach me to try and do two things at once.  Thanks for clearing that up for me though, Much appreciated.


Jamie

Leo Laohoo Mon, 02/17/2014 - 13:52
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    The Hall of Fame designation is a lifetime achievement award based on significant overall achievements in the community. 

  • Cisco Designated VIP,

    2017 LAN, Wireless

One pair being 100 meters and the other being 300 meters would this cause any issues with the performance of the Etherchannel.

Some of our etherchannel links are like this.  No issues.


One physical link going one way and another physical link going another way.  This addresses physical path redundancy.  Quiet handy to deter an entire link being pulled by a fibre-trunk-hunting-buckhoe.

ziyaayan15 Sat, 05/21/2016 - 09:16
User Badges:

We have the same problem. The portchannel that has got 4 four channels(2 of them 2 meters, 2 of them 120 meters.) affects the throughput. This portchannel is uplink. When we transfer the data accross this portchannel on 4 four ports the throughput is 1GBps but if we shutdown the 120 meters port, the throughput is increasing and comes to 1.9GBps.

It'is ridicilous but real.

If some one interest, I will share more information.

Paul Chapman Sat, 05/21/2016 - 09:58
User Badges:
  • Silver, 250 points or more

Hi -

Need a little more information on your testing:

  • What fiber modules are being used?
  • What is the cabling type for each connection? (OM2 (62.5 or 50 micron), OM3, OS1/2)
  • Is the cabling type consistent from end to end? (ex. If using a 62.5 infrastructure cabling, are you using matching patch cables?)
  • Have you tested on the 120M cables with the 2M shutdown?
  • Have you tested each port in the channel individually?
  • Are you seeing port errors?
  • Are the switches Catalyst or Nexus?

PSC

ziyaayan15 Sat, 05/21/2016 - 10:33
User Badges:

Hi Paul,

The answers are on the below.

  • 10Gbase-SR
  • OM3
  • No
  • Yes. We tested only 120M cables and we saw the throughput is 1GBps. 
  • No.
  • We havent got any discards,errors etc..
  • The access switches are 5548s and the Core switches are 56128Ps. They connect each other with double site VPC.  The core switches are in different rooms. The distance between the rooms is about 120 meter. And we have 4 5548s and a pair in one room the other pair is in the other room.

I attached the topology.


Regards.



Attachment: 
Jim R Sat, 05/21/2016 - 11:02
User Badges:

Hi there,

Do you have any figures regarding loss on the larger 120m links? Ideally things like attenuation and total loss in db?

It may be that although your not getting any errors reported the total throughput may be limited by the above somewhat.  We certainly had that. The above also kind of assumes your tests are easily replicated and reliable and not giving you duff info regarding throughput.

Many thanks,

Jamie




Paul Chapman Sat, 05/21/2016 - 12:31
User Badges:
  • Silver, 250 points or more

Hi -

A few things come to mind here.

  • Nexus switches running 10Gb port-to-port use cut-through switching, so it's unlikely that you'd ever see packet loss on these ports.
  • Your configuration is rather unusual having a vPC pair split by 120M. Your diagram effectively shows that (1) of the 56128s is in the same room with the 5548s.
  • With the 120M connections are you using a single cable or are you going through a structured cable system?
  • Testing should be end-to-end, not switch-to-switch.  What are the results between the shown endpoints with various connections turned on and off?
  • When links are up and system is in the broken state, what is the result of "show port-channel load-balance forwarding-path interface" commands specifying information about the endpoints?

PSC

Actions

This Discussion