cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4389
Views
0
Helpful
17
Replies

Weird MTU issue

trey.thompson
Level 1
Level 1

We're having a somewhat strange issue with MTU. We have a 50Mbps ethernet handoff from our provider (which, apparently, comes off of a fiber ring) which terminates directly on our ASA 5510. Speed and duplex are set to 100/full on both sides. The MTU for the outside interface is set to 1500 and our provider reports that the MTU on their router is set to 2450. From a consumer-class DSL connection at my home, I show the path MTU to the provider's router as 1500, which is expected. However, the path MTU to my ASA drops to 1020, which, theoretically, is only one hop back from the ISP's router. From inside the LAN behind the ASA, the path MTU to any site on the Internet is 1020, but the path MTU to anywhere on my LAN is 1500. We have noticed some latency to Internet sites which makes me wonder about packet fragmentation. Also, we have been testing a video conference unit that streams HD video using UDP packets and we are seeing a lot of loss. I'm wondering if the MTU change is part of the equation with the streaming as well. Any thoughts or places to check would be very much appreciated. Thanks!

17 Replies 17

Peter Paluch
Cisco Employee
Cisco Employee

Hello Trey,

This is an interesting issue. How do you measure the path MTU? Is it perhaps a traceroute or a similar command? My idea is that the path MTU discovery is based on ICMP messages sent by a router that needs you to decrease the packet size. Personally, the first thing I would look after is catching these ICMP messages and having a look what is the sender IP. Most probably, that machine would be the best place to start because, after all, it is precisely that machine that forces the MTU to be smaller.

Best regards,

Peter

Thanks for the reply. I used a small windows utility called mturoute.exe which, as I understood it, used ICMP packets of various sizes to try and determine the path MTU. I have not done a capture. Would I capture the packets on the machine that's sending the ICMP packets? Thanks!

Hi Trey,

Nice utility :) I haven't had the pleasure of knowing it until now (being an ardent Linux user, the Windows just pass by me :)

Yes, sniff the packets on your computer. I suggest using Wireshark for that purpose. Then, look for all ICMP packet that are sent back to you and watch for 'Fragmentation needed' ICMP messages.

Best regards,

Peter

Hi,

sometimes 'Fragmentation needed' ICMP messages are not passing the network (dropped by FWs or some L3 devices not supporting the feature).

So the only way working any time is setting DF-bit in your Pings to the destination and watching the replies.

Example:

Ping x.x.x.x -l 1472 -f

(in Windows world, corresponds to "Ping x.x.x.x size 1500 df-bit" in Cisco world).

When no reply comes back, decrease the packet size

Ping x.x.x.x -l 1200 -f

If a reply comes back, try to increase again.

In several steps, you can find the maximum packet size passing the network without fragmentation.

(Which, I believe, is exactly what mturoute does?)

BR,

Milan

I don't know mturoute but in software it's easy to combine mtu probing and traceroute so that it can tell you on which link over a path the MTU gets reduced.

Hi Milan,

Yes, you are right. But I assume that if Trey already got that software running and it did produce some information for him then the ICMP messages were probably permitted. But, yes, that command could have also tried to interpolate that value. We'll see what Trey has to say.

Best regards,

Peter

Hi Peter,

yes, I agree.

It would be interesting to see Trey's output of mturoute -t ...

in both directions (from his LAN to the Internet and from the Internet to his LAN).

I think I'll test mturoute.exe in my network "when I have some time free".

One thing I don't understand:

Looking to that tool author's web pages

(http://www.elifulkerson.com/projects/mturoute.php), there's an example:

"C:\mturoute-src\bin>mturoute -t www.slashdot.org

mturoute to www.slashdot.org, 30 hops max, variable sized packets

* ICMP Fragmentation is not permitted. *

* Maximum payload is 10000 bytes. *

1 --+---+++-+++-++ host: 192.168.2.1 max: 1500 bytes

2 --+---+++-+++-++ host: 10.90.160.1 max: 1500 bytes

3 --+---+++-+++-++ host: 68.87.56.165 max: 1500 bytes

4 --+---+++-+++-++ host: 68.87.56.213 max: 1500 bytes

5 --+---+++-+++-++ host: 68.87.56.209 max: 1500 bytes

6 --+---+++-+++-++ host: 68.87.56.201 max: 1500 bytes

7 --+---+-+++++++ host: 68.87.56.197 max: 1472 bytes

8 --+---+++-+++-++ host: 68.87.56.249 max: 1500 bytes

"

If router 7 permitted only 1472 Byte packet length, how could the tool recognize router 8 permitting 1500 again?

BR,

Milan

Hello Milan,

That has left me wondering as well until I had a better look at the author's page which states:

Due to the nature of the test, it is sensitive to packets dropped for non-MTU related reasons. Packets that are lost for other reasons will result in an incorrect MTU reading.

Apparently, the mturoute utility does not use the elicited ICMP Fragmentation Needed responses as the normal Path MTU discovery does, but simply reacts to all lost packets as being "too big" for that hop and uses a heuristic approach to determine the packet size.

This really lefts me wondering now if the mturoute is the good utility for this job. Trey, do you have a recent Linux handy with the traceroute or tracepath commands? They are based on real ICMP messaging to find out the MTU along a path.

Best regards,

Peter

I have access to a linux box. Is there a particular set of traceroute switches I should use to get the best output? I'll provide the output and let you all take a look. Many thanks for looking into this for me!

Hello Trey,

The traceroute v2.0.12 has a --mtu switch that tries to find out the path MTU. Try running it as:

traceroute -n --mtu DESTINATION

or

traceroute -I -n --mtu DESTINATION

The -I uses ICMP packets for traceroute operation (as opposed to using UDP), the -n is for numerical output (no IP to DNS name translation).

I am very interested in seeing what it produces.

Best regards,

Peter

Here are the results from outside our network. The first traceroute is to the default route from the firewall (the first hop on the ISPs network). The second is to the outside interface of our firewall.

media@media:~$ sudo traceroute -I -n --mtu 64.158.200.141

traceroute to 64.158.200.141 (64.158.200.141), 30 hops max, 65000 byte packets

1 69.69.224.1 57.646 ms F=1500 55.600 ms 55.678 ms

2 69.34.35.89 55.597 ms 55.616 ms 64.525 ms

3 4.79.18.209 66.992 ms 66.715 ms 66.981 ms

4 4.68.17.194 67.433 ms 70.662 ms 66.955 ms

5 209.247.11.89 69.391 ms 69.428 ms 69.431 ms

6 64.158.200.141 73.535 ms 73.750 ms 73.630 ms

media@media:~$ sudo traceroute -I -n --mtu 64.158.200.142

traceroute to 64.158.200.142 (64.158.200.142), 30 hops max, 65000 byte packets

1 69.69.224.1 57.578 ms F=1500 55.671 ms 55.343 ms

2 69.34.35.93 55.872 ms 55.152 ms 55.731 ms

3 4.59.146.41 67.395 ms 67.194 ms 67.267 ms

4 4.68.17.2 67.980 ms 67.670 ms 67.534 ms

5 209.247.11.89 69.929 ms 69.435 ms 69.481 ms

6 209.244.22.2 69.676 ms 69.697 ms 69.408 ms

7 * * *

8 * * *

9 * * *

10 * * *

11 * * *

12 * * *

13 * * *

14 * * *

15 * * *

16 * * *

17 * * *

18 * * *

19 * * *

20 * * *

21 * * *

22 * * *

23 * * *

24 * * *

25 * * *

26 * * *

27 * * *

28 * * *

29 * * *

30 * * *

Hello Trey,

This is interesting. First of all, the MTU is, according to your output, 1500 bytes and does not change further. As for the traceroute to 64.158.200.142, ASA firewalls often to not respond to ICMP so that might be a little problem here but nevertheless, the Linux traceroute did not confirm the MTU issue.

Where did you do these tests from? Were you inside that network which allegedly had the MTU issues?

Best regards,

Peter

Hi Trey,

i know this is a 2 year old issue, but perhaps you still remember... I am facing a similar problem at a Customer: 2x ASA5510 8.0(5) connected through a VPN tunnel  (outside-outside), drops packets greater than 1020byte (incl. ping between the outside interfaces). Could you tell me how did you solve your problem? IOS upgrade to which version, workaround, etc..

Thank you in advance

Fernando

Old or not, for anyone who stumbles over a similar issue: The ASA will block all incoming icmp traffic by default.

This was mentioned in the thread but probably the poster has not taken sufficient notice of this.

If this is omitted, you will never see any icmp messages from sources on the Internet.

So you need to explicitly allow icmp from outside. I am using an acl like this:

access-list icmp-out-in extended permit icmp any any echo-reply
access-list icmp-out-in extended permit icmp any any time-exceeded
access-list icmp-out-in extended permit icmp any any unreachable

access-group icmp-out-in in interface outside

regards,

Leo

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card