×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

Configuring Jumbo MTU Support on Nexus 7000 and 5000

Document

Thu, 02/26/2015 - 01:00
Apr 8th, 2014
User Badges:
  • Silver, 250 points or more

 

Introduction


New generation networking equipments like servers and Nexus switches support jumbo mtu so to send a big chunk of data in one go. This improves overall network throughput if all equipments support jumbo mtu size. However if jumbo mtu is not supported by all switches in the path it may result in packet fragmentation or in some cases packet drops. If the DF (Don't Fragment) bit within the IP header is not set by the server, the router will fragment the packets and forward on. This will impact both the router performance and the throughput that the servers are able to achieve between each other. If the server Operating System supports and is configured for Path MTU Discovery, it will set the DF bit in the IP packet header causing the Nexus to return an ICMP Type 3 Code 4 (Fragmentation Needed and Don't Fragment was Set). This will result in the hosts reducing the maximum packet size they sends.

 

 

 

Topology Description


There are two servers on two different subnets/vlans and they want to use jumbo frames to tranfer data. The servers are actually connected on two Nexus 5000 switches and these are layer 2 so the default gateways are on the Nexus 7000 switches.

 


Topology

 

 

Limitations


For Nexus 5K release prior to NX-OS 5.0(2)N1(1), an MTU mismatch is a Type 1 inconsistency and the vPC secondary peer will disable vPC member ports. As part of NX-OS 5.0(2)N1(1) Support for Type 2 vPC Consistency Checks was added which changed a number of parameters from Type 1 to Type 2 inconsistency as so downtime is less likely in this case.
Nexus 5K doesn't show the configured 9216-byte MTU in the output of commands such as show interface. You can see that the policy has been taken using the show queuing interface command though.

 

Note: For L2, you cannot set the MTU per interface but only system wide. The mtu is set under network-qos which can only be attached to system qos.
For L3 you can apply it to just one or a group-of interface, instead of the "system qos" just use "interface ethx/y".

 

 

MTU Size Considerations

For Layer 3 interfaces, you can configure the MTU to be between 576 and 9216 bytes (even values are required). For Layer 2 interfaces, you can configure the MTU to be either the system default MTU (1500 bytes) or the system jumbo MTU size (which has the default size of 9216 bytes). You can change the system jumbo MTU size, but if you change that value, you should also update the Layer 2 interfaces that use that value so that they use the new system jumbo MTU value. If you do not update the MTU value for Layer 2 interfaces, those interfaces will use the system default MTU (1500 bytes).

 

 

Configuration


For Jumbo mtu to work properly it should be configured on all interfaces, including physical interface and vlan interface (on Nexus 7K). If you have port channel configured then it should also have jumbo mtu enabled. Make sure that all possible paths between any jumbo enabled end-points are all jumbo enabled.

 

Nexus 5K config:

policy-map type network-qos JUMBO_MTU
  class type network-qos class-default
    mtu 9216

!

system qos
  service-policy type network-qos JUMBO_MTU

Nexus 7K config:

system jumbomtu 9216

!

interface ethernet x/y
mtu 9216

!

interface vlan xxx
mtu 9216

This document is created from following discussion
https://supportforums.cisco.com/discussion/12070561/jumbo-support-nexus-...

 

 

Verify

 

Use the command "show interface ethernet" or "show interface ethernet port/slot" to check if the size is set for jumbo mtu.

 

 


vPC Best Practices for Nexus 7000 and 5000
vPC Failover Scenarios and Troubleshooting Checklist

 

Loading.
mateja.jovanovic Thu, 02/26/2015 - 01:00
User Badges:

There is an annoying behaviour (reported bug actually) of N5k, when you do the "show interface" the MTU you will see will always be 1500, even though you set the default to 9k using the MQC.

Actions

This Document

 

 

Trending Topics: Other DC Subjects