configuring adjust-mss + ip mtu in the same tunnel, affects hsrp?

[ajcamachou]

Hi all,

Please let me know if I am right:

I am using a GRE-IPSEC tunnel configuration + EIGRP routing protocol and I would like to confirm if I am right with the following:

Due to I am configuring the IP MTU = 1400 in the Tunnel interface, I am avoiding additional fragmentation after GRE encapsulation because there is enough room for IPSEC encapsulation (no matter which mode I am using - I am considering the worst case Tunnel Mode). However, I would like to know what happens if I ALSO include in the Tunnel configuration the command ip tcp adjust-mss 1360 which is clear operates in layer 4 during the 3 way handshaking process to establish the TCP connection/session between opposite hosts (in this case the interaction is with the respective end routers). By adding this MSS command, I understand that I could also eliminate the initial fragmentation of the 1500 bytes LAN packets (before GRE encap) because the hosts are notified to send 1360 bytes packets to the Router and based on the previous, I would be able to transfer packets without "theorical" fragmentation between both ends.

One more question, how can affect if I include this additional command (TCP ADJUST-MSS) the performance (process + memory) of a router 3845 or 7200 without producing for example a entire crash of the device???. I understand that this TCP MSS negotiation is router process intensive but is less than IPSEC encryption/decryption.

thanks in advance for your comments.

[ajcamachou]

In addition to the previous, I am going to test my SSH connection using Wireshark and access the LAN interface of the Remote Router. By doing this, I will check if the MSS (1360) is effectively negotiated however I am not sure if this is the best option/scenario for testing the MSS negotiation process.

[Peter Paluch]

Hello Abraham,

When I made SSH (tcp=22) from a PC in the internal LAN at the Central  Site to the IP address of the Router LAN Interface at the Remote Site,  passing through the GRE TUNNEL connecting both sites, I got a MSS of 536  bytes (similar to the explanation presented in the following link) but,  why this situation if I already activated PMTUD, so the MSS should be  1400 - 40 = 1360???

The MSS is first set by the parties opening the TCP connection. The MSS is their own decision and it depends on their operating system settings and the application that opens a TCP socket. If the MSS size is already lower than the value of ip tcp adjust-mss, the router is not going to modify the MSS in any way (i.e. it is certainly not going to increase it).

So what you saw is completely okay.

My final question is: How the MSS for TCP Traffic going through the GRE  Tunnel is actually set to the IP MTU configure on both ends of the GRE  Tunnel - 40 bytes of headers??. 

The ip tcp adjust-mss is only going to decrease the MSS in traversing TCP segments, if the value exceeds the configured limit. However, if the MSS is already being advertised by the TCP peers as lower than the value of ip tcp adjust-mss, the router will not interfere - it is the decision of the TCP peers to have the MSS so small.

That is why this MSS manipulation by ip tcp adjust-mss is called as "MSS clamping" in Linux operating system. The MSS is only made smaller if necessary by the router, but it is never increased - doing so could lead to various problems with delivering or processing the TCP segments by the party whose MSS has been increased this way.

Best regards,

Peter

[ajcamachou]

Hi Peter, thank you so much for your explanation. I would like to comment that I am only using is IP MTU + PMTUD between both ends instead of  ip tcp adjust-mss, and I got the following results using Wireshark on my PC and generating a TCP connection from my PC (10.1.55.38) to a server located in the remote site LAN (10.14.32.37). Now I see that the IP MTU = 1400 configured in the interface Tunnel for GRE is responsible for setting the MSS=1360 and PMTUD is also useful in my case because it allows to adjust THE INTERNAL IP MTU for GRE in case that any router in the path from origin to destination requires a lesser MTU.

Based on my test below, The basic difference that I visualize between IP MTU/PMTUD and IP TCP Adjust-MSS is that  IP Adjust fixes the MSS Value so if there is required lesser MSS in the route from Origin to Destination, the packet will be fragmented in the path due to a higher MSS previously configured. Adjust MSS looks like more try and error than MTU/PMTUD (except when you have FW/ACL blocking ICMP Messages which affects MTU/PMTUD).

Peter, one more question, WHY do you think is the reason for which my PC sent an initial MSS value = 1260 during the TCP Handshaking process? Is this probably the value stored dinamically by the INTERNAL IP MTU (as I explained before) after interacting with the routers on the ISP network in order to reduce this MTU Value and notified to me by my default GW (GRE Tunnel Router) for this case?? On the other hand, the destination sent me a 1360 MSS value which could mean that any packet that I sent will be fragmented in the path origin-destination.

[Peter Paluch · Accepted solution]

Hello Abraham,

Is the IP MTU parameter the responsible for limiting the MSS size during  the 3-way tcp handshake to 1360 bytes ((IP MTU configured in the Tunnel  Interface = 1400) - (40 bytes headers)) as you see in the next image?  How the PMTUD is also participating in this process?

No, the IP MTU parameter configured on a router is not responsible for the initial MSS size as advertised by end hosts. The end hosts initially derive their starting MSS value by looking at their own interface MTU (not the router's MTU - they do not know it at all) and decreasing it by at least 40 bytes. This starting MSS may subsequently be lowered thanks to ip tcp adjust-mss on a router. During the TCP session, the real MSS used to talk to the other party may dynamically change according to the current MTU of the end host's interface, or it may be influenced by PMTUD during the session, but it may never exceed the negotiated MSS.

Quoting from RFC 1122, Section 4.2.2.6 stipulates:

            The maximum size of a segment that TCP really sends, the
            "effective send MSS," MUST be the smaller of the send MSS
            (which reflects the available reassembly buffer size at the
            remote host) and the largest size permitted by the IP layer:

               Eff.snd.MSS =

                  min(SendMSS+20, MMS_S) - TCPhdrsize - IPoptionsize

            where:

            *    SendMSS is the MSS value received from the remote host,
                 or the default 536 if no MSS option is received.

            *    MMS_S is the maximum size for a transport-layer message
                 that TCP may send.

            *    TCPhdrsize is the size of the TCP header; this is
                 normally 20, but may be larger if TCP options are to be
                 sent.

            *    IPoptionsize is the size of any IP options that TCP
                 will pass to the IP layer with the current message.

The PMTUD - if run by the end host engaged in the TCP communication - may influence the MMS_S element of the equation. Please do not confuse this Effective Send MSS with the value of the MSS option indicated in the TCP header during the 3-way handshake. That MSS is here represented as SendMSS and it may never be exceeded.

Based on my test below, I dont see any significant difference between  using IP MTU or IP TCP Adjust-MSS if the IP MTU is already responsible  for limiting TCP traffic through the GRE Tunnel connection.

The difference is rather strong. As I just explained, the IP MTU configured on a router is not directly influencing the MSS choice at the end host simply because the end host has no idea about the configuration of the router. In fact, the only way to find out is to use the PMTUD that has to be run by the end host again - not by the router. If the PMTUD is not performed, end hosts may end up negotiating much larger MSS than what would be necessary to prevent fragmentation, and the resulting IP packets sent by end hosts will have to be fragmented by the router.

The ip tcp adjust-mss is therefore indispensable.

Peter, if I am not wrong, the initial MSS value = 1260 sent by my PC  depends on the Operating System and Application based on your  orientation?

Yes, that is correct. What do you mean by "orientation" here?

Best regards,

Peter

[ajcamachou]

Hi Peter, sorry for answering so late.

I would like to say that I am not an english language native and probably I used the wrong word/expression (sorry about that). I mean by orientation that I followed your instructions/guidance to understand better this case and apply improvements to the current device configuration.

On the other hand, I decided to use MSS Adjust + MTU because in the PMTUD process are not participating the end hosts so they can send packets with an larger MTU that the one negotiated by the routers using PMTUD.

Additionally, an extended ping from site to site was made using a variable size from 1400 to 1505 and all the packets until 1500 bytes were received by the Carrier Router in both ends (just before the GRE Tunnel starts and ends). Now we can confirm that the packet size allowed by the transport network (carrier) can reach 1500 bytes so there is no reason for a fragmentation outside the tunnel connection.

I definitely believe that the fragmentation problem is focused on the UDP traffic. So the only improvement that I can implement is to configure MTU + MSS Adjust as I mentioned previously in order to at least reduce any TCP fragmentation. MTU value for this GRE Tunnel I think should be configured to 1476 instead of the Cisco recommended 1400 because there is only 24 bytes of additional header (20 for IP header + 4 GRE header - I am not using IPSEC) which represents the entire 1500 max size allowed by the network. On the other hand, TCP MSS Adjust would be (1476-40=1436).

Please let me know what do you think about the last.

Regards

Abraham