Solved: PLPMTUD vpn. ipsec mtu mss

AdamBudzinski · ‎11-19-2015

hi all,

can ayone please tell me, why sometimes when tunneling or any other sort of encapsulation is performed along the network path from A to B, it is neccsarad it is necessary to lower the MTU / MSS value on a router ?

Why is Packetization Layer Path MTU Discovery (PLPMTUD) no enough to sort the MTU issue out. As per RFC 4821

"Packetization Layer Path MTU Discovery (PLPMTUD) is a method for TCP or other Packetization Protocols to dynamically discover the MTU of a path by probing with progressively larger packets. It is most efficient when used in conjunction with the ICMP-based Path MTU Discovery mechanism as specified in RFC 1191 and RFC 1981, but resolves many of the robustness problems of the classical techniques since it does not depend on the delivery of ICMP messages.

The general strategy is for the Packetization Layer to find an appropriate Path MTU by probing the path with progressively larger packets. If a probe packet is successfully delivered, then the effective Path MTU is raised to the probe size."

Bet Regards

Adam

Peter Paluch · ‎11-26-2015

Hi Adam,

I apologize for the late answer - it's been a busy week.

Could you please tell me what do exactly mean " (just check your nearest TCP session in a Wireshark - are your IPv4 packets sent out with DF bit set? I doubt it.)"

In IPv4, PMTUD is based on sending IP packets with DF bit set, otherwise, routers on the path will be allowed to fragment these packets if necessary, and you will get no feedback about the need of decreasing the Path MTU. Because the results of PMTUD can be directly used by TCP, the fact whether your operating system and/or application are using PMTUD can be easily tested by checking TCP sessions originated by your computer. If PMTUD is being used, IP packets carrying TCP segments must be sent out with DF bit set. What I hinted to by my original remark is that the use of PMTUD in IPv4 is, sadly, quite rare, and even more sadly, it is often hindered by poor firewall and NAT implementations.

Also, to confirm, during the 3 way handshake when each site will advertise it's MSS, it will be based on the hosts NIC settings (MTU value?) not the MTU on the router interface that acts as the defualt gateway ?

The MSS should initially be derived from the host's MTU but may be lower depending on the TCP/IP stack settings on the host. However, in a single network, all devices should use the same MTU. It would be illegal for two hosts in the same network to use different MTU sizes. It should never happen that an end host uses a different MTU than its default gateway.

Note that in IPv4, a host has no direct means of discovering the MTU of its default gateway. If it sends a packet larger than the default gateway's interface MTU in this network, the default gateway may drop the packet silently. In IPv6, this MTU is advertised in ICMPv6 Router Advertisements, but even then, having hosts in a single network use different MTUs is considered a misconfiguration, and should be avoided at all costs.

Best regards,
Peter

View solution in original post

Peter Paluch · ‎11-19-2015

Hi Adam,

Path MTU Discovery is a fairly old technique; the PLPMTUD is a much more recent addition, and these two are distinct. However, even the fact that they are supported in a particular operating system on an end host does not guarantee that they will be used for every TCP session terminated on that host. The default settings of the operating system may actually keep both these mechanisms deactivated by default. An application can request the operating system to provide these services for its connections, but in that case, the application must actually be written with this in mind so that it turns on the PMTUD or PLPMTUD on its own sockets.

So, IPv4 applications wishing the make use of PMTUD or PLPMTUD on their TCP connections must do specific steps to activate these mechanisms.

What's even more, for non-TCP sockets, the implementation of PMTUD/PLPMTUD is much more demanding as in that case, the application must itself check for the reported maximum allowable MTU from the operating system, and it must itself take care of packaging its messages so that the resulting IP packet does not exceed this MTU. TCP can fragment the handled data anyway it wants; UDP, UDP-Lite, DCCP leave the fragmentation up to the application.

Specifically, for PMTUD, ICMP messages "Packet too big" have to be allowed on firewalls. Often, they aren't, breaking the whole mechanism.

So there's enough issues with PMTUD and PLPMTUD that they're not being universally used (just check your nearest TCP session in a Wireshark - are your IPv4 packets sent out with DF bit set? I doubt it.), and so the MTU issues with tunnels are not discovered.

IPv6 is a different story, as the PMTUD is mandatory, but even then, ICMPv6 packets again need to be allowed on firewalls, otherwise the whole mechanism cannot work.

Feel welcome to ask further!

Best regards,
Peter

AdamBudzinski · ‎11-19-2015

Hi Peter,

thank you very much for answering my question.

I see your points.

Could you please tell me what do exactly mean " (just check your nearest TCP session in a Wireshark - are your IPv4 packets sent out with DF bit set? I doubt it.)"

I took a look on some older captures, and sometimes the DF flag is set in in the packets sometimes it is not, but I'm not sure this is what yuo sugeested to do ?

Also, to confirm, during the 3 way handshake when each site will advertise it's MSS, it will be based on the hosts NIC settings (MTU value?) not the MTU on the router interface that acts as the defualt gateway ? I did some testing (changed the MTU value on MS OS to 1000) and left 1500 on the default gateway interface (on a VM running VyOS). So is this a normal behaviour also for CISCO devices?

Many thanks in advance and best regards Peter!

Adam

Peter Paluch · ‎11-26-2015

Hi Adam,

I apologize for the late answer - it's been a busy week.

Could you please tell me what do exactly mean " (just check your nearest TCP session in a Wireshark - are your IPv4 packets sent out with DF bit set? I doubt it.)"

In IPv4, PMTUD is based on sending IP packets with DF bit set, otherwise, routers on the path will be allowed to fragment these packets if necessary, and you will get no feedback about the need of decreasing the Path MTU. Because the results of PMTUD can be directly used by TCP, the fact whether your operating system and/or application are using PMTUD can be easily tested by checking TCP sessions originated by your computer. If PMTUD is being used, IP packets carrying TCP segments must be sent out with DF bit set. What I hinted to by my original remark is that the use of PMTUD in IPv4 is, sadly, quite rare, and even more sadly, it is often hindered by poor firewall and NAT implementations.

Also, to confirm, during the 3 way handshake when each site will advertise it's MSS, it will be based on the hosts NIC settings (MTU value?) not the MTU on the router interface that acts as the defualt gateway ?

The MSS should initially be derived from the host's MTU but may be lower depending on the TCP/IP stack settings on the host. However, in a single network, all devices should use the same MTU. It would be illegal for two hosts in the same network to use different MTU sizes. It should never happen that an end host uses a different MTU than its default gateway.

Note that in IPv4, a host has no direct means of discovering the MTU of its default gateway. If it sends a packet larger than the default gateway's interface MTU in this network, the default gateway may drop the packet silently. In IPv6, this MTU is advertised in ICMPv6 Router Advertisements, but even then, having hosts in a single network use different MTUs is considered a misconfiguration, and should be avoided at all costs.

Best regards,
Peter