cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
26630
Views
39
Helpful
18
Replies

ip tcp window-size command on the router

anasubra_2
Level 1
Level 1

Hi All,

We have 2 MPLS PE nodes(Cisco 7600). The Server1 is connected on PE1 and Server 2 is connected on PE2.We have a latency of around 20 ms between these two PE nodes. When we tried to do data transfer from Server 1 to Server 2 by means of FTP,we could acheive some were around 20Mbps has peak utilization for that FTP session. Then we understood the BDP(Bandwidth delay product) comes into issue and the throughput depends on the latency. So inorder to increase the throughput,we try to increase the server 1 and server 2 window buffer sizes to more than 64Kbytes,but we couldn't see much differnce in the throughput.

Meanwhile we are tring to understand the ip tcp window-size command on the PE router and does this has any relation to the above scenario.Also what this command really does for the router and if it is not set,whats the default value on it.Any help would be really appreciated.

Also, when we increased the buffer sizes from 32k to 64k ,there are significant increase in the throughput but not when we increased above 64k.

Thanks

Regards

Anantha Subramanian Natarajan

18 Replies 18

royalblues
Level 10
Level 10

I dont think modifying the TCP window size on the router would increase the throughput.

The default tcp window size on the routers is 4128 bytes (without windows scaling)

The window sizes would be negotiated between the end systems.

To increase the window sizes to more than 64K, you need to enable window scaling. Also try a UDP transfer which will result in higher throughput

Narayan

Hi Narayan,

Thank you very much ...... UDP transfer is been tested and the peak utilization can be pushed upto nearly the pipe bandwidth as you were asking.

Assuming the window size is been negotaited between the end systems,what is the use of this router command and is it for TCP connections originated and destined to the router ?

Also ,I have an another question,is there a chance,the routers overwrites window size set by the servers ?

Once again thanks for your response

Regards

Anantha Subramanian Natarajan

i think configuring the TCP window sizes will affect the traffic destined to the router.

You can test it anyway by enabling windows scaling and doing the transfer. All you need to do is to define the TCP window size to eb higher than 65535.

Under normal circumstances the router is not intelligent enough to override the TCP window size for the transit traffic. This is actually what is done in the WAN optimisers which considerabley improves the throughput across a link by modifying the window size but it is never a requirement to adjust the windows sizes for all the network devices in between them .

HTH

Narayan

Hi Narayan,

Thank you very much for the reply. It seems the routers tcp window size doesn't give option more than 65535. Is there any command to be added on router to provide option more than 65535 ?

Thanks

Regards

Anantha Subramanian Natarajan

Anantha,

You shoudl be able to configure a window size greater than 65535. you cannot configure a scaling factor directly but when u use a value more than 65535 scaling is automatically enabled

http://www.cisco.com/en/US/docs/ios/12_2t/12_2t8/feature/guide/tcpwslfn.html

The ethereal trace mostly shows the normal window size + a scale factor in the options and not the newly calculated that you would get multiplying them together.

Narayan

Hi Narayan,

Thanks for your quick response. It seems the current code we are running 12.2(18)SXF4 doesn;t support TCP scaling. When I searched from the IOS feature navigator,it doesn;t highlight this code for this feature and also in the router when i do ip tcp window-size <0 - 65535 bytes>, it provides this range . If I miss something,kindly let me know .

Thanks

Regards

Anantha Subramanian Natarajan

yes mostly seems a IOS issue.

You can try using another IOS but i still do not feel its gonna increase any throughput for your transfers

cheers

Narayan

Hi Narayan,

Thanks

Regards

Anantha Subramanian Natarajan

Joseph W. Doherty
Hall of Fame
Hall of Fame

If the host's TCP receive buffer is less than BDP, sender will not be able to utilize path's maximum bandwidth. Sender will keep stopping tranmission while awaiting ACKs.

If host's TCP receive buffer is larger than BDP, you won't see any improvement in thoughput since receive buffer, when sized for BDP, should support 100% utilization, but you might see a reduction in performance if sender sends too much which results in packet drops within network devices along the path.

Often overlooked, network device queues may also need to be sized to support BDP, otherwise you may encounter early packet drops while flow is attempting to ramp up to maximum bandwidth.

You didn't note minimum bandwidth between servers, but if it were, for example, 100 Mbps with your noted 20 ms latency, the BDP is 250,000 bytes (assuming my math is correct). Assuming TCP packets are 1500 bytes (1460 MSS), a queue size of 171 packets might be needed on routers. Anything more or less, especially less, would/could preclude optimal performance.

Also don't forget TCP slow start, as the BDP increases, it takes TCP longer to ramp up. Further, any packet loss takes longer for TCP to recover its maximum flow rate.

Any, all, or some combination of the prior factors might account for why you didn't see an increase in performance when you tried a receive buffer larger than 64 KB. Also, remember scaled received buffers is a later TCP option; both hosts (sender and receiver) have to agree to use it.

On the issue of router's "ip tcp window-size", my understanding is that to set the router's TCP receive window when it's acting as a host. Little on the router would see an improvement when it's adjusted except perhaps sending a file to the router with something like FTP, although if writing to flash, the flash is likely to be the bottleneck.

PS:

Other than an optimization appliance to improve TCP performance when dealing with larger BDPs, perhaps the easiest method to improve performance is to send data via multiple concurrent TCP flows. Multiple flows deal much better with bandwidth ramp up and packet loss bandwidth recovery.

Hi Josephdoherty,

Awesome response,thank you very much ......One thing,which I couldn't understand is about tuning queue size required on routers,how that can be evaluated on the routers ?.

Thank you very much

Regards

Anantha Subramanian Natarajan

re: Tuning the router queue size (for worst case):

First compute the BDP across the network (end-to-end), not just to next hop. (Basically the same BDP you would compute for the receiving host).

Divide BDP by payload portion of expected packets (if standard Ethernet, 1460). That will provide your needed queue size to absorb maximum TCP transmission window.

NB: You'll only need to really tune the queue where there's a bandwidth reduction. Often this is at the WAN router WAN link egress interface. The larger the delta between the source sending bandwidth and the WAN bandwidth, tuning the WAN queue becomes more important. If the bandwidth delta is small, little tuning, or none, may be needed. (I don't have calculation to compute how much of the worst case queue allocation you'll need.)

To make this clearer, assume you had 10 Mbps WAN via sat. If the sender (on the LAN) was sending at 10 Mbps, you don't need to adjust WAN queue. If the sender (on the LAN) was sending at gig, packets will queue up at the WAN interface. Assuming sender's transmission window is BDP, router's queue will need to queue much/most of it because it arrives 100x faster than the router sends.

PS:

As my prior post shows, BDP and needed queue depth can get large, so if there are other flows sharing the link, try to insure everything isn't placed into one huge FIFO queue.

Joseph

Great explanation as always

Narayan

Narayan, thank you for the compliment.

Hi Joseph,

Thank you ....Actually in our case WAN link is 10 Gig circuit and the LAN is 1 Gig . So,I would assume ,queuing is not a problem right ?

Thanks

Regards

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco