Just got my hands on a few C240 M3′s with the VIC 1225 adapter and began setting them up as a VMware 5.1 U1 cluster. I’ve configured 8 vNICs per host: 2 for Management, 2 for VMotion, 2 for iSCSI, and 2 for VM traffic. Each pair of vNICs has one pinned to uplink 0 and the other to uplink 1. vNICs are set to trunk mode in the VIC since I am tagging multiple VLANs from VMware. The hosts are physically connected to a stacked pair of Cisco 4500-X switches; physical uplink 0 to one switch, uplink 1 to the other switch. Switch ports are trunked to allow the VLANs we plan to use. Portfast is enabled. VMware vSwitches using Source Port ID for Load Balancing and the vNICs are active/active for Management and VM traffic. No port channel or Etherchannel is configured on the swtiches.
After creating a VM for vCenter and joining the hosts to it, I started migrating existing VMs from an old cluster and saw odd network behavior. It seems whenever a VM or vmkernel interface are on any vNIC pinned to the same physical uplink on the same host, they can’t talk to each other.
Example: VM1 and VM2 are on host 1. VM1′s network adapter is pinned to vNIC1 using VLAN 10 which uses physical uplink 0. VM2′s network adapter is pinned to vNIC2 using VLAN 10 which uses physical uplink 1. VM1 and VM2 can ping each other. If I move VM2′s network adapter to any vNIC that uses physical uplink 0, just like VM1, then VM1 and VM2 can no longer ping each other. Or if host 1′s management IP is pinned to vNIC 1 on physical uplink 0, any VM on host 1 using a vNIC also pinned to physical uplink 0 cannot ping the host.
Anyone have any ideas? Is something configured incorrectly on the switch? On the VIC?
This is exactly the issue. We've also been told by Cisco support that a Nexus switch can accomodate multiple vNICs in an active/active configuration just like one would set up on the B-Series running VMware, but a Catalyst cannot.
To get around this we've pinned Management traffic to a vNIC on physical uplink 0 and VM traffic to physical uplink 1 so they'll have no issues talking to each other.
Moquery is the command line cousin of Vizore, it's very helpful and efficient sometimes during the troubleshooting. This article aims to provide moquery cheat sheet to the users for some most common seen scenarios.
Here is the checklist before customers/partners contact Cisco TAC:
Firmware Version of APIC and Switch
Download Switch and APIC techsupport logs
Problem description (Symptoms with details)
Business impact (eg, what kind of services...
moquery usageAPIC moquerySwitchmoquery
This document discuss a common issue observed during the VMM integration & VM workload migration to ACI fabric.
VMware Virtual machines are hosted in Cisco UCS-B seri...