It is about a IPSEC/GRE over WAN...
Would you please confirm or comment the following in terms of MTU:
1. On GRE tunnel interfaces "ip mtu" and "ip tcp adjust-mss" is mandatory. "tunnel path-mtu-discovery" is good to have and will allow DF bit to be set in the outer header. If "tunnel path-mtu-discovery" is to be applied, ICMP should not be blocked between routers.
2. On inside router interfaces "ip tcp adjust-mss" is mandatory and will be the same value as on the tunnel interfaces. This will make sure TCP traffic from inside hosts is OK.
3. It is mandatory that ICMP messages are not blocked between inside hosts and WAN routers in order for PMTUD for hosts to be working.
Thanks in advance,
1. On GRE tunnel interfaces "ip mtu" and "ip tcp adjust-mss" is mandatory - NO. "tunnel path-mtu-discovery" is good to have and will allow DF bit to be set in the outer header. If "tunnel path-mtu-discovery" is to be applied, ICMP should not be blocked between routers - NO.
2. On inside router interfaces "ip tcp adjust-mss" is mandatory and will be the same value as on the tunnel interfaces. This will make sure TCP traffic from inside hosts is OK. - NO.
3. 3. It is mandatory that ICMP messages are not blocked between inside hosts and WAN routers in order for PMTUD for hosts to be working. - YES.
Thanks for the response.
May be I misinterpreted the document:
No you have not mis-read the document - maybe just been lead down a path a little, my answers are based on experiance.
I have found that tunnel path-mtu-discovery/PMTUD/BlackHole MTUD do not work in 99.999% of the cases where I have had mtu issues - Windows OS has been where the issues lie. I have never encounted a time where the Windows OS has actually taken any notice of the ICMP fragmentation needed message has been recevied.
Some Cisco platforms cannot use the tcp mss adjust command on transient packets, only packets sourced from the deivce are effected.
Cisco firewalls, have default configuration in regards to fragementation - the packets will be fragemented prior to encrypting the packet and they copy the DF bit = the packet will be dropped due to being oversized.
What I do when dealing with GRE/IPSEC tunnels is either:-
1) Change the MTU of the workstations/servers - works in small enviroments, does not scale.
2) You do not have to worry about MTU/MSS sizes on internet sites generally, as the remote servers wil 99% negotiate a small MSS.
3) Use where possible tcp mss adjust on routers and firewalls (this is a great place, especially when you are not using GRE tunnels)
4) Perform packet captures to determine if an application will send ALL packets with the DF bit set, or as normal just the TCP handshake.
Below is a good example:-
My idea was the following:
All TCP traffic between hosts in sites - try not to do fragmentation at all (neither IPSEC nor GRE)
All UDP or other traffic - if hosts use large MTUs, hosts should use PMTUD to adjust the correct MTU
On the TCP side - yes agree with that, trying to avoid fragmentation will save you alot of wasted troubleshooting time.
All else, if you complete the above you wont have to worry about this.
It's a double edge sword to be honest, if you have an application/session that always sets the DF bit to 1, then you need to look at adjusting the negotiated MSS. If not - then it's all good.
Best practise - ensure that the MSS negotiated is lower than required if you are using GRE/IPSEC. I always use 1300. So the MSS will always negotiate to 1260, so the you have 240 bytes for any overhead.
How can you determine if the Microsoft pc/server is ignoring the icmp message? I'm having an issue with mtu/gre/ispec and have configured every possible solution. I've tried to send a clear the df bit and the tcp mss-adjust and still nothing.
It still might be a config but I want to verify that the hosts are atleast received and acknowledging the icmp. Any suggestions?
You can run a packet capture - you will see the ICMP Frag Required be sent from the router to the server/pc. To give some direction to your issues:-
I've tried to send a clear the df bit - this only works when you have qos pre-classify in the GRE tunnel. This is of course as long as the router/switch you are using supports this feature & GRE tunnels for that matter.
and the tcp mss-adjust and still nothing - again you need to run on a platform that actually supports it, as 100% of the platforms have it as a configurable option, however it only applies on connections TOO the device - not thru it.
What are you platforms?
Have you tried to clear the DF bit before traffic enters the GRE tunnel?
Are you trying to clear it before/after it gets encapsulated in IPSEC?
Have you tried pre-fragmentation before encryption?
Host pc connets to 3750 (c3750-advipservicesk9-mz.122-46.SE.bin), switch connects to 3845(c3845-ipvoicek9-mz.124-24.T1.bin), router connects to ASA 5550.
The ASA has a site-2-site ipsec vpn the another asa.
GRE tunnels are up and voip and ospf traffic is working fine. My data traffic gets the error - df set when I try to advertise via ospf and go through the tunnel.
Between the router and switch I'm using subinterfaces and trunking. I tried to put the tcp mss-adjust command on the routers subinterface.
On the asa, I've tried prefrag before encrypt.
I was thinking about changing the link between the switch and router to a layer 3 x.x.x.x/30 and try to change the mtu on the switches uplink port.
I'm open to suggestions.
OK - the best place to configure the tcp adjust will be in the GRE tunnel interface - only required on 1 side (in the hub) in a hub2spoke topology, with the correct value. REMEMBER the router will intercept the TCP syn/TCPsyn ack and replace the MSS value you configure.
So you must configure a value that will allow the max amount of data and the relevant header sizes.
IP Header = 20 Bytes
TCP Header = 20 Bytes
IPSEC Head = 56 Bytes
GRE Header = 24/8 Bytes
Ethernet Header = 26 Bytes
20+20+56+28+26 = 150 Bytes
MTU = 1500 - 150 = 1350. So the best MSS value will be 1300.