I posted the below in VMware's forum yesterday, now I'm cross posting here for advise. I'd like to add after a day of research though -
There is a LOT of advice on the Internet around this problem, generally suggesting the CONTROL VLAN isn't talking. I also believe that using the new Layer 3 communication, this advice is redundant, as the CONTROL VLAN is only used to communicate between the Active and Standby VSMs in this new format.
The irony is that "show module" indicates that the active/standby modules are talking to each other fine, and if I the standby I see event logs about it dropping a heartbeat and getting it back. Accordingly, I'm quite sure the CONTROL VLAN is working.
With this said, if I understand correctly, L3 communication really amounts to "can the VSM connect to the ESXi vmkernel". Currently I can ping the vmk from VSM without any issue. So where can the issue lie?
I do note one thing I appear to do different - the vmkernel I'm using for management of my ESXi host is on a standard vSwitch - I felt that - in situations like this one - it's more important I can continue to access the VM console to fix issues like this.
The environment is configured with L3 connectivity on the VEM,
The environment is:
One server (for now). Obviously, VEM and VSM are on the same machine.
vswitch0 (standard) -> pnic0 (yes I know I will need more pnics in future)
This is connected by a trunk port to a Cisco switch, with the ESXi management on VLAN 13. I do intend on locking this down once things work.
There is a management VLAN 17 which all switches are on, and hence, so is the "management" port group of the Nexus.
The upstream switch has VLAN 13,17 and 19 all configured and trunked to all ports (not that it should matter across the same host)
How about I just paste the config. Relevant parts below.
What I'm seeing ultimately is that "show module vem missing" lists the local host.
port-profile type vethernet n1kv-L3 capability l3control vmware port-group switchport mode access switchport access vlan 17 no shutdown system vlan 17 state enabled port-profile type ethernet system-uplink vmware port-group switchport mode trunk switchport trunk allowed vlan 1-3967,4048-4093 channel-group auto mode active no shutdown system vlan 13,17,19 state enabled port-profile type vethernet Server_VLAN vmware port-group switchport mode access switchport access vlan 13 no shutdown state enabled port-profile type vethernet CONTROL vmware port-group switchport mode access switchport access vlan 19 no shutdown system vlan 19 state enabled ....
svs-domain domain id 100 control vlan 1 packet vlan 1 svs mode L3 interface mgmt0
-The active/standby VSM always communicate over L2. L3 communication tunnels traffic from VSM's mgmt0 interface to the vmkX on the ESX host.
-I see you have an LACP port-channel configured - via 'mode active'. Please ensure 'feature lacp' is enabled on the VSM. Also verify the upstream Cisco switch has a matching channel-group with mode active.
-Since you have not moved the vmk0 from the vSwitch to N1k dVS, the module will not register. Both a vmk in port-porfile
n1kv-L3 and a vmnic in port-profile system-uplink is required. I suggest migrating vmnic1 to N1k port-profile system-uplink, then migrating the vmk0 or create a different vmk in the same subnet. At that point you should see the module register.
Topology & Design:
Two ACI fabrics
Stretching VLANs using OTV
Both fabrics are advertising BD subnets into same routing domain
Some BDs(or say VLANs) are stretched, but some are not.
Endpoints can move betwee...
VMware Trunk Port Group is supported from ACI version 2.1
VMM integration must be configured properly
ASA device package must be uploaded to APIC
ASAv version must be compatible with ACI and device package version
Topology &Design:Traffic flow within same fabric:Endpoint moves to Fabric-2Bounce Entry Times OutTraffic Black-holedSummarySolutionAppendix:
In the Previous articles of ACI Automation, we are using Postman/Newman a...