cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2930
Views
0
Helpful
9
Replies

Fibre Link Length and Etherchannel

Jim R
Level 1
Level 1

Morning folks,

Had an interesting question put to us which I've not come across before.  Assuming I have a two channel Etherchannel consisting of multimode OM3 fibre,  One pair being 100 meters and the other being 300 meters would this cause any issues with the performance of the Etherchannel.  My thinking was that provided both pairs could negotiate the same speed and duplex settings the only issue would be that the longer distance may have potentially a slower transfer rate than the shorter length but I wasn't sure.

Can anybody point me in the direction of a suitable Cisco document or otherwise that may be able to advise on this?

TIA

Jamie

9 Replies 9

Peter Paluch
Cisco Employee
Cisco Employee

Hi Jamie,

Do not confuse the bandwidth and the latency. They are related but not in a strict way that would determine one by the other. The link with the 300m fiber will have a higher latency but the amount of data it is able to carry within a second is the same as the 100m link - just the data will start "pouring out" on the other end sooner or later, depending on the length.

On Cisco switches, EtherChannel always steers entire frame flows over the physical link in the bundle. This means that an entire conversation (as defined by hashing the selected fields from the frames, check the show etherchannel load-balance command to check it) is forwarded through a given link. Therefore, because different frame flows are, in general, independent, so should be the choice of the link. The fact that one of them has a higher latency should not be significant.

My two cents...

Best regards,

Peter

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

As Peter noted, Etherchanneling keeps all flows on the same link, so as he also noted, the flow on the longer link would have some additional latency.  However, also keep in mind that for a delta of 200m, the fiber, itself, would only add about 1 microsecond of additional latency.  For most real-world applications, this additional latency is a non-issue.

Jim R
Level 1
Level 1

Hi Peter,

Thanks for the reply,  I realised the overwhelming mistake in my orginal question as soon as I read the first line of your reply Getting my fibre channel and ethernet muddled up.  Teach me to try and do two things at once.  Thanks for clearing that up for me though, Much appreciated.

Jamie

One pair being 100 meters and the other being 300 meters would this cause any issues with the performance of the Etherchannel.

Some of our etherchannel links are like this.  No issues.

One physical link going one way and another physical link going another way.  This addresses physical path redundancy.  Quiet handy to deter an entire link being pulled by a fibre-trunk-hunting-buckhoe.

ziyaayan15
Level 1
Level 1

We have the same problem. The portchannel that has got 4 four channels(2 of them 2 meters, 2 of them 120 meters.) affects the throughput. This portchannel is uplink. When we transfer the data accross this portchannel on 4 four ports the throughput is 1GBps but if we shutdown the 120 meters port, the throughput is increasing and comes to 1.9GBps.

It'is ridicilous but real.

If some one interest, I will share more information.

Hi -

Need a little more information on your testing:

  • What fiber modules are being used?
  • What is the cabling type for each connection? (OM2 (62.5 or 50 micron), OM3, OS1/2)
  • Is the cabling type consistent from end to end? (ex. If using a 62.5 infrastructure cabling, are you using matching patch cables?)
  • Have you tested on the 120M cables with the 2M shutdown?
  • Have you tested each port in the channel individually?
  • Are you seeing port errors?
  • Are the switches Catalyst or Nexus?

PSC

Hi Paul,

The answers are on the below.

  • 10Gbase-SR
  • OM3
  • No
  • Yes. We tested only 120M cables and we saw the throughput is 1GBps. 
  • No.
  • We havent got any discards,errors etc..
  • The access switches are 5548s and the Core switches are 56128Ps. They connect each other with double site VPC.  The core switches are in different rooms. The distance between the rooms is about 120 meter. And we have 4 5548s and a pair in one room the other pair is in the other room.

I attached the topology.

Regards.

Hi there,

Do you have any figures regarding loss on the larger 120m links? Ideally things like attenuation and total loss in db?

It may be that although your not getting any errors reported the total throughput may be limited by the above somewhat.  We certainly had that. The above also kind of assumes your tests are easily replicated and reliable and not giving you duff info regarding throughput.

Many thanks,

Jamie

Hi -

A few things come to mind here.

  • Nexus switches running 10Gb port-to-port use cut-through switching, so it's unlikely that you'd ever see packet loss on these ports.
  • Your configuration is rather unusual having a vPC pair split by 120M. Your diagram effectively shows that (1) of the 56128s is in the same room with the 5548s.
  • With the 120M connections are you using a single cable or are you going through a structured cable system?
  • Testing should be end-to-end, not switch-to-switch.  What are the results between the shown endpoints with various connections turned on and off?
  • When links are up and system is in the broken state, what is the result of "show port-channel load-balance forwarding-path interface" commands specifying information about the endpoints?

PSC

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco