IP MTU & fragmentation

Answered Question
Dec 3rd, 2009

I'm having a problem with our DSL router related to Path MTU discovery, namely I'm trying to avoid the problem by getting the MTU's right in the first place due to some critical customer sites being PMTU black holes, but it's having no effect.

Our topology is multi-vendor:

Cisco 6509 <---> Juniper ex4200 <---> 2xCisco 7206 <---> Juniper ex4200 <---> telco fiber

All connections are using vlans on GigE (aside from the telco Gig fiber); the 7206s are DSL routers doing PPPoE to the DSL customers.

I have port mirroring on the ex4200 so I can tcpdump connections:

When a customer starts a tcp stream from a site that does Path MTU (which seems to be ubiquitous these days), I see an incoming frame 1518 bytes, to which the 7206 replies with Fragmentation Required (next mtu 1492), and nothing comes out the telco side, as would be expected.  I believe the problem is the addition of the PPP header, however when I increase the MTU to 1526 on the telco side of the 7206, nothing changes: it still says FR (1492).  I would expect the net mtu to increase by 8 at the very least, even if I'd miscalculated something and it needs to be larger.

If the telco links can't handle the larger size, I would expect the packets to disappear in the switch, but

* I'm told the gear actually handles over-mtu sized packets gracefully (though with other problems the new telco dsl gear has had, it wouldn't surprise me if otherwise).

* It's been a long time since I had to deal with bits on the wire, but I don't recall any signalling below L3 on packet sizes.

* I adjusted the mtu on the ex4200 just in case there was some signalling, to no avail

So I'm really puzzled what I need to do to get this thing to forward full packets...

I have this problem too.
0 votes
Correct Answer by Anupam Datta about 7 years 1 month ago

Hi !,

You can try by making DF bit to 0 . From your dagram it is not clear whether all the servers or destinations from your customer are behind the 6509 or not.

if it is, then make a policy to set the DF bit to 0 , and apply it to the ingress interface of 6509 (the interface connecting to internal server-farm switch). If servers or destinations are dirrectly connected to 6509 in various ports then use it on egress interface. If still no result , then you can try it on 7206 's both interface (ingress/egress). The configuration is as follows--

ip access-list extended FRAGMENTATION
deny   ip 10.X.X.X Y.Y.Y.Y
any / Which could be excluded or which IP/Servers are working properly, otherwise this line could be excluded.
permit ip any any

route-map MTU-ROUTEMAP permit 10
match ip address FRAGMENTATION
set ip df 0

Interface XXXX

ip policy route-map MTU-ROUTEMAP

Regards,

Anupam

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Giuseppe Larosa Fri, 12/04/2009 - 06:37

Hello Alan,

PPPoE overhead is 8 bytes.

a possible different approach is to reduce IP mtu to 1492 bytes on customer side.

the problem with MTU settings are that sometimes is not clear is when it refers to L2 frame size or to L3 packet size.

Hope to help

Giuseppe

useopenid Fri, 12/04/2009 - 09:48

That's what we've been doing previously, but this customer is a company with a large number of machines behind a router and that's not practical, leading to the desire to configure things so they can handle full packets.

Correct Answer
Anupam Datta Tue, 12/08/2009 - 02:40

Hi !,

You can try by making DF bit to 0 . From your dagram it is not clear whether all the servers or destinations from your customer are behind the 6509 or not.

if it is, then make a policy to set the DF bit to 0 , and apply it to the ingress interface of 6509 (the interface connecting to internal server-farm switch). If servers or destinations are dirrectly connected to 6509 in various ports then use it on egress interface. If still no result , then you can try it on 7206 's both interface (ingress/egress). The configuration is as follows--

ip access-list extended FRAGMENTATION
deny   ip 10.X.X.X Y.Y.Y.Y
any / Which could be excluded or which IP/Servers are working properly, otherwise this line could be excluded.
permit ip any any

route-map MTU-ROUTEMAP permit 10
match ip address FRAGMENTATION
set ip df 0

Interface XXXX

ip policy route-map MTU-ROUTEMAP

Regards,

Anupam

useopenid Thu, 12/17/2009 - 11:14

For clarification, the diagram should be expanded to be:

<---> Cisco 6509 <--->  Juniper ex4200 <---> 2xCisco 7206 <---> Juniper ex4200 <---> telco fiber <--->

This workaround was effective.  I would still rather get the path to handle full sized packets, but that may not be possible.

FWIW, if the customer modems are configured to use PPPoA (which the DSLAM converts to PPPoE, which is what we see on our DSL router), either 2 brands of modem or, more likely, the DSLAM, rewrites the MSS in the SYN packet from 1460 to 1452, avoiding the problem.  This does not happen when the modems use PPPoE directly.  While I dislike magic solutions, that does seem to be an effective way to avoid the problem to start with, but is out of our control...

Actions

This Discussion