Enable Jumbo Frame Support On UCS

Answered Question
Sep 15th, 2010
User Badges:

Could someone direct me to the documentation for the proper way to enable jumbo frame support on UCS?  I'm trying to troubleshoot a problem and have configured MTU of 9000 from ESX running on a blade to an NFS server off a Nexus 5k.  I have good documentation for configuring jumbo frame support on the Nexus 5k and on ESX but nothing really specific for UCS.


For Nexus 5k:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/nxos/Cisco_Nexus_5000_Series_NX-OS_Software_Configuration_Guide_chapter33.html#con_1150612


show run:


policy-map type network-qos jumbo

  class type network-qos class-default

    mtu 9216

system qos

  service-policy type network-qos jumbo

For ESXi:


esxcfg-vswitch -m 9000 vSwitch0


esxcfg-vswitch -A IP_Storage vSwitch0
esxcfg-vswitch -v 31 -p IP_Storage vSwitch0


esxcfg-vmknic -a -i 10.254.31.2 -n 255.255.255.192 -m 9000 "IP_Storage"
Testing:
I have been able to ping (with large payload) the NFS host interface from a temporary vlan interface on the N5k.  I cannot do the same to the ESX hosts or between hosts.
Thanks,
-R
Correct Answer by Manish Tandon about 6 years 8 months ago

Russell


What network adapter are you using? M81KR or the M71KR's or any other?


The MTU is set on a per CoS basis in UCS. If you do not have any QoS policy defined for the vNIC that is going to the vSwitch, then the traffic is going to Best-Effort Class. All you need to do to enable jumbo frames is to enter the MTU to be 9000 in the MTU pulldown for whatever class the traffic is going to.


Its also given in the user guide -

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/1.3.1/UCSM_GUI_Configuration_Guide_1_3_1_chapter18.html




MTU drop-down list

The maximum transmission unit for the channel. This can be:



An integer between 1500 and 9216. This value corresponds to the maximum packet size.


fc—A predefined packet size of 2240.


normal—A predefined packet size of 1500.

Note

This field is always set to fc for Fibre Channel.




--Manish

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Correct Answer
Manish Tandon Wed, 09/15/2010 - 22:54
User Badges:
  • Bronze, 100 points or more

Russell


What network adapter are you using? M81KR or the M71KR's or any other?


The MTU is set on a per CoS basis in UCS. If you do not have any QoS policy defined for the vNIC that is going to the vSwitch, then the traffic is going to Best-Effort Class. All you need to do to enable jumbo frames is to enter the MTU to be 9000 in the MTU pulldown for whatever class the traffic is going to.


Its also given in the user guide -

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/1.3.1/UCSM_GUI_Configuration_Guide_1_3_1_chapter18.html




MTU drop-down list

The maximum transmission unit for the channel. This can be:



An integer between 1500 and 9216. This value corresponds to the maximum packet size.


fc—A predefined packet size of 2240.


normal—A predefined packet size of 1500.

Note

This field is always set to fc for Fibre Channel.




--Manish

Russell Teague Thu, 09/16/2010 - 05:07
User Badges:

Manish, thank you, that did the trick.  We currently do not have any QoS implemented so I was thinking that the QoS policies would not have any effect, but as you pointed out, everything falls into the 'Best Effort' class.  I made the change to 9000 for the MTU on Best Effort and it works.  I had already set the MTU for the vNIC to 9000 which I assume is also required.


Thanks again,

-R

Manish Tandon Thu, 09/16/2010 - 10:09
User Badges:
  • Bronze, 100 points or more

Russell


Yes, config at the OS level is also required which you had already covered as per the show commands you had sent.

In Linux for example one would use ifconfig to set interface MTU etc.


Thanks


--Manish

Russell Teague Thu, 09/16/2010 - 11:57
User Badges:

When I mentioned vNIC, I was referring to the vNIC or vNIC Template in UCS.  In my scenario the configuration of the MTU value 'end-to-end' would be:


  • Virtual interface on the NFS storage
  • System wide both Nexus 5020s
  • 'Best Effort' QoS policy on UCS
  • vNIC Templates for the Service Profiles in UCS
  • vSwitch in ESXi (this also sets the MTU of the physical uplink interfaces)
  • vmknic for IP storage in ESXi


Cheers,

-R

stevecuddy Thu, 09/04/2014 - 05:25
User Badges:

Hi Guys,

Just a quick one, great post by the way.
I also need to to change my Best Effort traffic to 9212 but all my traffic does indeed fall into this CoS.
I'm in full production so does anyone know the impact it may have on the FI / vNICs or the current traffic passing through the interfaces.

Regards
Steve.


 

Mike Hulbert Mon, 03/14/2011 - 12:07
User Badges:

Manish,


I also wanted to add to this post that if you SSH into Fab Interconnect and do a show ethernet 1/x that you may see an MTU size listed here of 1500, even though you may have, say, NFS traffic going over an appliance port and vNIC's configured for jumbo in UCSM.


Your response:

The show output is expected.

The FI’s and Nexus 5000 do CoS based MTU and not interface based.

So if you set the MTU in the System class to 9000, any packet recd on any port on the FI with that CoS value will pass that 9000 byte packet.

Thought would be good info for the blog....

Actions

This Discussion

Related Content