Cisco Support Community
Community Member

Nexus vswitch - connectivity VSM to VEM

Hey all,

I posted the below in VMware's forum yesterday, now I'm cross posting here for advise. I'd like to add after a day of research though -

There is a LOT of advice on the Internet around this problem, generally suggesting the CONTROL VLAN isn't talking. I also believe that using the new Layer 3 communication, this advice is redundant, as the CONTROL VLAN is only used to communicate between the Active and Standby VSMs in this new format.

The irony is that "show module" indicates that the active/standby modules are talking to each other fine, and if I the standby I see event logs about it dropping a heartbeat and getting it back. Accordingly, I'm quite sure the CONTROL VLAN is working.

With this said, if I understand correctly, L3 communication really amounts to "can the VSM connect to the ESXi vmkernel". Currently I can ping the vmk from VSM without any issue. So where can the issue lie?

I do note one thing I appear to do different - the vmkernel I'm using for management of my ESXi host is on a standard vSwitch - I felt that - in situations like this one - it's more important I can continue to access the VM console to fix issues like this.

The environment is configured with L3 connectivity on the VEM,

The environment is:

One server (for now). Obviously, VEM and VSM are on the same machine.

vswitch0 (standard) -> pnic0 (yes I know I will need more pnics in future)

vmk0 (vlan13)

pg_management (vlan17)

This is connected by a trunk port to a Cisco switch, with the ESXi management on VLAN 13. I do intend on locking this down once things work.

There is a management VLAN 17 which all switches are on, and hence, so is the "management" port group of the Nexus.

The upstream switch has VLAN 13,17 and 19 all configured and trunked to all ports (not that it should matter across the same host)

vswitch1 (DVS)

How about I just paste the config. Relevant parts below.

What I'm seeing ultimately is that "show module vem missing" lists the local host.

port-profile type vethernet n1kv-L3
capability l3control
vmware port-group
switchport mode access
switchport access vlan 17
no shutdown
system vlan 17
state enabled
port-profile type ethernet system-uplink
vmware port-group
switchport mode trunk
switchport trunk allowed vlan 1-3967,4048-4093
channel-group auto mode active
no shutdown
system vlan 13,17,19
state enabled
port-profile type vethernet Server_VLAN
vmware port-group
switchport mode access
switchport access vlan 13
no shutdown
state enabled
port-profile type vethernet CONTROL
vmware port-group
switchport mode access
switchport access vlan 19
no shutdown
system vlan 19
state enabled

domain id 100
control vlan 1
packet vlan 1
svs mode L3 interface mgmt0

Any assistance appreciated.

Community Member

Nexus vswitch - connectivity VSM to VEM

Other things I've tried today that have not helped:

Create an additional VMKernel within the n1kv-L3 port group using the VLAN 17 IP range to see if it would communicate with that.

Reduce the upstream to a single port

Cisco Employee

Nexus vswitch - connectivity VSM to VEM

Hello Technionn,

A few things -

-The active/standby VSM always communicate over L2.  L3 communication tunnels traffic from VSM's mgmt0 interface to the vmkX on the ESX host.

-I see you have an LACP port-channel configured - via 'mode active'.  Please ensure 'feature lacp' is enabled on the VSM.  Also verify the upstream Cisco switch has a matching channel-group with mode active.

-Since you have not moved the vmk0 from the vSwitch to N1k dVS, the module will not register.  Both a vmk in port-porfile

n1kv-L3 and a vmnic in port-profile system-uplink is required.  I suggest migrating vmnic1 to N1k port-profile system-uplink, then migrating the vmk0 or create a different vmk in the same subnet.  At that point you should see the module register.


CreatePlease to create content