cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
10891
Views
0
Helpful
8
Replies

Nexus 1000V Jumbo Frames MTU 9000

felixjai
Level 1
Level 1

I have an ESX host with 4 NICs and I added all four NICs to the 1000V DVS. All of the four NICs are part of a single port-profile (system-uplink). After I did that, everything works fine except the datastores are now not accessible. The datastores is a NFS mount which uses MTU 9000 and is mounting on a vlan. Before moving to 1000V DVS, the 4 NICs were configured with MTU 9000 in the standard vSwitch0. According to the Cisco config guide, MTU can't be set on vethernet interfaces. But the four NICs (eth3/3 - 3/6) are configured with system-uplink port-profile and MTU can't be set neither. The config guide is here http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0_4_s_v_1_2/interface/configuration/guide/n1000v_if_2basic.html#wp1221750

I'd like to know if anyone came across a similar situation. I'd hate to break the 4 NICs and move 1 or 2 NICs just to get it the MTU size correctly. I'm attaching part of the config for review. VLAN 142 is the storage vlan.

version 4.0(4)SV1(2)

vem 3
  host vmware id 35303738-3634-xxxxxxxxxxxxxxxxxxxx
vlan 1,142,221,223-224
vlan 101
  name control
vlan 102
  name packet
vlan 100
  name management
port-profile type vethernet Control
  vmware port-group
  switchport mode access
  switchport access vlan 101
  no shutdown
  state enabled
port-profile type vethernet Packet
  vmware port-group
  switchport mode access
  switchport access vlan 102
  no shutdown
  state enabled
port-profile type ethernet Unused_Or_Quarantine_Uplink
  description Port-group created for Nexus1000V internal usage. Do not use.
  vmware port-group
  shutdown
  state enabled
port-profile type vethernet Unused_Or_Quarantine_Veth
  description Port-group created for Nexus1000V internal usage. Do not use.
  vmware port-group
  shutdown
  state enabled
port-profile type vethernet data_vlan224
  vmware port-group
  switchport mode access
  switchport access vlan 224
  no shutdown
  state enabled
port-profile type ethernet data-uplink
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 142,221,223-224
  channel-group auto mode on sub-group cdp
  no shutdown
  state enabled
port-profile type vethernet data_vlan223
  vmware port-group
  switchport mode access
  switchport access vlan 223
  no shutdown
  state enabled
port-profile type vethernet service_console
  vmware port-group
  switchport mode access
  switchport access vlan 221
  no shutdown
  state enabled
port-profile type vethernet storage
  vmware port-group
  switchport mode access
  switchport access vlan 142
  no shutdown
  state enabled
port-profile type ethernet system-uplink
  vmware port-group
  switchport mode trunk
  switchport trunk allowed vlan 100-102,142,221,223-224
  channel-group auto mode on sub-group cdp
  no shutdown
  system vlan 100-102,142,221
  state enabled

interface port-channel1
  inherit port-profile system-uplink

interface port-channel2
  inherit port-profile system-uplink

interface Ethernet3/3
  inherit port-profile system-uplink

interface Ethernet3/4
  inherit port-profile system-uplink

interface Ethernet3/5
  inherit port-profile system-uplink

interface Ethernet3/6
  inherit port-profile system-uplink

interface Vethernet1
  inherit port-profile storage
  description VMware VMkernel, vmk1
  vmware dvport 160

interface Vethernet2
  inherit port-profile mgmt_network
  description VMware VMkernel, vmk0
  vmware dvport 352

8 Replies 8

Hello

did you try to set up the system MTU first?

Nexus1000v(config)# system jumbomtu 9000

Regards

Jlouis

Hi Jean-louis,

I believe we have resolved the issue. I found that the MTU setting needs to be set on the Port-channel instead of to the Ethernet interface.

I have another question. I have some Nexus 1000V instance that was installed for demo purpose, but I need to uninstall it from vCenter, but I keep getting error when object was still in used. I checked all hosts were removed and couldn't figure out what else needs to remove. Do you have a link for uninstalling? Thanks.

Felix Lai | Network & Security Architect | CCIE CCSP JNCIS-FWV JNCIA-SSL | Northeast | Presidio Networked Solutions

One Penn Plaza, Suite 1924, New York NY 10119 | flai@presidio.com

D: 212.652.5717 | C: 917.805.1431 | F: 212.244.1685 www.presidio.com<>

Funny enough, i was trying to do exactly the same thing today.

After a lot of pain i managed to get rid of them using the troubleshooting guide:

You are logged in to the vSphere Client and the Nexus 1000V.


Step 1 From the vSphere Client, choose
Inventory

Networking
.


Step 2 Select the Nexus 1000V DVS and select the
Hosts
tab.


Step 3 Right-click each host, and choose
Remove from Distributed Virtual Switch.


Step 4 On the VSM, enter the
show vmware vc extension-key
command to verify the extension key.


Step 5 Delete the extension key that is present on vCenter Server using the Managed Object Browser (MOB) Unregister Extension API, as follows:


a. Go to the extension manager [https:///mob/?moid=ExtensionManager]


b. Click
Unregister Extension
[https:///mob/?moid=ExtensionManager&method=unregisterExtension]


c. Paste Cisco_Nexus_1000V_495736846 (your extension key attached to the DVS) and click
Invoke Method.


Step 6 From the Nexus 1000V Connection Configuration mode, remove the DVS from the vCenter Server as follows:.


n1000v# conf t
n1000v(config)# svs connection VC
n1000v(config-svs-conn)# no vmware dvs

If that doesn't help, may be you can post your error message

good luck

jlouis

thanks for the quick reply. What if the VSM is no longer there or was deleted before removing the SVS connection?

I might be wrong but i think your only way is to recreate a Nexus1K using the same extension key, register it again and unregister properly

I couldn't find any other way to remove it from the DVS. I m sure a lot of people will have the same problem and we should have the option on the next version(if we re lucky)

Sorry

Jlouis

Just as a sidenote here (and a word of caution), there's currently an issue where the VEM modules will not retain the 9000 MTU setting upon reboot/restart, and will boot themselves @ 1500 MTU until they get their instructions from the VSM. This may cause some headaches for things like mounting network storage, etc... so just a variable to consider. In addition, it can be particularly ugly if your VSM storage is network-attached.

As I understand it, it is slated to be fixed in the next version of Nexus 1000v software later this month.

There are a lot of excellent posts out there regarding MTU, yet for weeks my colleagues and I continue to be confused about MTU on Nexus100v.

The confusions were around:

-How to set jumbo on N1kv

-Relationship between "system mtu" and "mtu"

-If "system mtu" is needed

-How to set "system mtu"

I wrote a short summary, based on multiple conversations with Cisco, and finally the definitive source. Hope it helps.

http://www.seanxwang.com/2010/07/nexus1000v-when-to-use-system-mtu.html

Darian O'Dirling - TCE
Cisco Employee
Cisco Employee

Hi @felixjai
I see you are working with MTU configuration. I encourage anyone who is running into this issue or wants to learn more about MTU configuration on Nexus 9000 series switches to watch this video made by a colleague mind. The video outlines the need for proper MTU in the network and explains how to remedy MTU related interface errors, via interface MTU configuration.

https://www.youtube.com/watch?v=GbBeB6h9MKk

This video provides steps for configuring MTU on a Nexus 9000 switch. The video outlines the need for proper MTU in the network and explains how to remedy MTU related interface errors, via interface MTU configuration.
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: