cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2462
Views
52
Helpful
22
Replies

WAN Link issue

Ibrahim Jamil
Level 6
Level 6

Hi Experts

i have 1G Fiber link between 2 sites,as per MRTG output , the link is underutilized , but the strange is i m copying a size 100G from site 1 to site 2 , the download average just 16Megabyte per second

pls clarify

jamil

22 Replies 22

vmiller
Level 7
Level 7

You need to look at how the end stations have the tcp windowing configured

http://cisconet.com/traffic-analysis/throughput/104-tcp-throughput-calculation-formula.html

very good website for things like this.

Hi vmiller

Kindly, can u explain to me this issue?

thanks

TCP doesn't dump traffic on a link to fill the pipe.  It transmits a piece at a time and waits for the remote end to acknowledge receipt prior to sending more information.  This allows the protocol to ramp up it's transmit rate as long as things are going well, or throttle it down if it doesn't receive timely acknowledgements (such as if the link exhibits packet loss or is becoming congested).

Think of it as feeding a baby.  It would be tempting to think that the size of your link equates to the size of the baby's mouth, and we can just start dumping food in the mouth to keep it full.  However, that's a good way to ensure that little food makes it to baby's stomach.  Instead, we send a spoonful of food at a time (the amount of food kinda like the TCP window - the amount of data that the sender will transmit before waiting on an acknowledgement), allow the baby to chew, then open her mouth to show it is empty and ready for another bite (the acknowledgement).

Sometimes baby is going to hold her mouth closed (not ready - still chewing - latency/delay), or spit out the food (packet loss).  In this case we must scrape up the food and "re-send" it.  As you can see, depending on the packet loss, delay, speed of acknowledgements, the amount of time to transmit the baby (feed a jar of food) can vary based on link conditions.  Even if one baby's mouth is twice as large as another, if it's latency (chew time) is longer, it's going to be a longer feeding session than the baby with the smaller mouth.

vmiller's link provides a good explanation of the window size (size of a spoonful of food) and theoretical calculation of expected speed.  Note that a bigger spoon is not always better.

Hope that's not too silly of an explanation.

Well played sir. Much better than what I could have written.

It was a slow day. 

Guys,I Forgot to mentioned that both site are connected using l2 trunk

no matter. the endstations are still the ones that set transmit and receive windows.

Hi vmiller

since i have have 1 Gig Pipe , why i shouldn't  have an average of transfer rate  of 128 Mbps ,if use the formula of of dividing the 1024/8,pls advise

thanks

jamil

you would have to go thru the tcp tuning drill based on your workstation operating system.

http://www.psc.edu/networking/projects/tcptune/

there is some references to microsoft tech page for a how to ( I avoid workstation issues like the plague)

there could also be some application parameters that need adjusting.

Disclaimer

The      Author of this posting offers the information contained within this      posting without consideration and with the reader's understanding   that    there's no implied or expressed suitability or fitness for any    purpose.   Information provided is for informational purposes only  and   should not   be construed as rendering professional advice of any  kind.   Usage of  this  posting's information is solely at reader's own  risk.

Liability Disclaimer

In      no event shall Author be liable for any damages whatsoever    (including,   without limitation, damages for loss of use, data or    profit) arising  out  of the use or inability to use the posting's    information even if  Author  has been advised of the possibility of   such  damage.

Posting

Ibrahim Jamil wrote:

Hi vmiller

since i have have 1 Gig Pipe , why i shouldn't  have an average of transfer rate  of 128 Mbps ,if use the formula of of dividing the 1024/8,pls advise

thanks

jamil

Google BDP, "bandwidth delay product", LFN, "long fat network", and/or TCP RWIN.

PS:

There are other issue too, but the foregoing is usually the biggest issue.

Hi Joseph

i ping betwwen these 2 sites , the RTT is 2 ms

jamil

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Then you need a receive window of 244 KB, older TCP stacks often don't, by default, use TCP scaling; some also don't default even to 64 KB.

Or working the equation the other way, 16 MBbps (128 Mbps) would use a RWIN of 32 KB.

Hi Joseph , How can do it?

Disclaimer

The  Author of this posting offers the information contained within this  posting without consideration and with the reader's understanding that  there's no implied or expressed suitability or fitness for any purpose.  Information provided is for informational purposes only and should not  be construed as rendering professional advice of any kind. Usage of this  posting's information is solely at reader's own risk.

Liability Disclaimer

In  no event shall Author be liable for any damages whatsoever (including,  without limitation, damages for loss of use, data or profit) arising out  of the use or inability to use the posting's information even if Author  has been advised of the possibility of such damage.

Posting

First, confirm this is the issue. Best way would be sniff the traffic and look at the RWIN being advertised.  If it is too small, the easy, although expensive, way is to drop in an WAN accelerator on both ends.  Many will spoof the connection and do things like increase RWIN across the WAN link.  (They often do other tricks too.)

The more difficult way, possibly less expensive, determine how to adjust RWIN on the receiving host.  Then reconfigure the receiving host; often requires a host reload.  How the configuration should be modified varies per OS and OS version; often generally documented for many OSs on the Internet.

PS:

Later OSs shouldn't normally need manual configuration as their defaults are much better.

Later Windows hosts have different defaults whether workstation or server; there's also a patch for an earlier Windows servers to add later TCP stack features.

PPS:

Another approach is to use a transfer app that slices the file up so it can transfer multiple concurrent streams.  Such an app also works better dealing with packet loss; aggregate rate generally both ramps up faster and recovers fasters.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco