I have an Cisco 871 at a remote site and at the head end I have a Checkpoint Firewall; I can get the tunnel to come up and everything seems to work okay except for Outlook to Exchange server and File Mappings basically anything using AD Authentication. I took out the 871 and replaced it with a PIX501 and everything works perfectly. I also tried 2 different 871's with the same results so I am guessing I am missing some type of command in the 871 to fix this but I can't seem to figure out what it is. Anyone else ever have this problem and can you share the resolution please? I have attached my sample configs of the PIX and 871.
PIX firewall by default has the following configuration "sysopt connection tcpmss 1380" hence during the TCP three way handshare negotiation for the MSS size, it negotiates for a lower MTU size.
MTU size gets larger after it is being encrypted (with the ESP header) and ethernet normally has 1500 MTU size. If the MTU gets larger than 1500 bytes, the packet gets fragmented which causes issue that you are seeing.
Check out the "sysopt connection tcpmss" command explaination here:
Table of ContentsIntroductionVersion HistoryPossible Future
UpdatesDocuments PurposeNAT Operation in ASA 8.3+ SectionsRule Types
Network Object NATTwice NAT / Manual NATRule Types used per SectionNAT
Types used with Twice NAT / Manual NAT and Network Obje...
Table of Contents Introduction:This document describes details on how
NAT-T works. Background: ESP encrypts all critical information,
encapsulating the entire inner TCP/UDP datagram within an ESP header.
ESP is an IP protocol in the same sense that TCP an...