I am having some problems with VSM/VEM connectivity after an upgrade that I'm hoping someone can help with.
I have a 2 ESXi host cluster that I am upgrading from vSphere 5.0 to 5.5u1, and upgrading a Nexus 1000V from SV2(2.1) to SV2(2.2). I upgraded vCenter without issue (I'm using the vCSA), but when I attempted to upgrade ESXi-1 to 5.5u1 using VUM it complained that a VIB was incompatible. After tracing this VIB to the 1000V VEM, I created an ESXi 5.5u1 installer package containing the SV2(2.2) VEM VIB for ESXi 5.5 and attempted to use VUM again but was still unsuccessful
I removed the VEM VIB from the vDS and the host and was able to upgrade the host to 5.5u1. I tried to add it back to the vDS and was given the error below:
vDS operation failed on host esxi1, Received SOAP response fault from [<cs p:00007fa5d778d290, TCP:esxi1.gooch.net:443>]: invokeHostTransactionCall Received SOAP response fault from [<cs p:1f3cee20, TCP:localhost:8307>]: invokeHostTransactionCall An error occurred during host configuration. got (vim.fault.PlatformConfigFault) exception
I installed the VEM VIB manually at the CLI with 'esxcli software vib install -d /tmp/cisco-vem-v164-18.104.22.168.2.2.0-3.2.1.zip' and I'm able to add to to the vDS, but when I connect the uplinks and migrate the L3 Control VMKernel, I get the following error where it complains about the SPROM when the module comes online, then it eventually drops the VEM.
2014 Mar 29 15:34:54 n1kv %VEM_MGR-2-VEM_MGR_DETECTED: Host esxi1 detected as module 3 2014 Mar 29 15:34:54 n1kv %VDC_MGR-2-VDC_CRITICAL: vdc_mgr has hit a critical error: SPROM data is invalid. Please reprogram your SPROM! 2014 Mar 29 15:34:54 n1kv %VEM_MGR-2-MOD_ONLINE: Module 3 is online 2014 Mar 29 15:37:14 n1kv %VEM_MGR-2-VEM_MGR_REMOVE_NO_HB: Removing VEM 3 (heartbeats lost) 2014 Mar 29 15:37:19 n1kv %STP-2-SET_PORT_STATE_FAIL: Port state change req to PIXM failed, status = 0x41e80001 [failure] vdc 1, tree id 0, num ports 1, ports state BLK, opcode MTS_OPC_PIXM_SET_MULT_CBL_VLAN_BM_FOR_MULT_PORTS, msg id (2274781), rr_token 0x22B5DD 2014 Mar 29 15:37:21 n1kv %VEM_MGR-2-MOD_OFFLINE: Module 3 is offline
I have tried gracefully removing ESXi-1 from the vDS and cluster, reformatting it with a fresh install of ESXi 5.5u1, but when I try to join it to the N1KV it throws the same error.
I am seeing the same thing. However I have hosts that were installed as 5.5U1 and upgraded from 5.1U1. I only saw the modules disconnect when I added a 5.1 host to the 1KV. It was not too long after the module came online that all the modules dropped. I was only able to get them to come back up by adding the system-vlan command to the control port-profile (we are running in L2 mode).
I determined the issue a few weeks ago, the culprit was Jumbo Frames which I had correctely turned on end to end but I learned my NIC's didn't support it. I went back to a host that was still on a previous version of the VEM and tried MTU 9000 pings succesfully, but when I added a -d to the command it failed, indicating I never really had Jumbo Frames working. The new version of the VEM must be more intolerant of this misconfig than the old version. I have since turned off Jumbo Frames everywhere and am successfully keeping the VEM's connected to the VSM's.
Thanks for the KB on the SPROM cosmetic issue, I had originally thought that was the problem but now I know it's not.
This document will provide screenshots to outline the steps to setup
TACACS+ configuration to ACI and also the configuration required on
Cisco ACS server. Please find the official Cisco guide for configuring
TACACS+ Authentication to ACI:
Is it supported or NOT supported? It's a frequently asked question.
Before APIC, release 2.3(1f), transit routing was not supported within a
single L3Out profile. In APIC, release 2.3(1f) and later, you can
configure transit routing with a single L3Out pr...
Cisco Documents are usually accurate, but when it came to the document
on Cisco APIC Signature-Based Transactions it was slightly off the mark.
This document is for those novices to API like me who cant seem to
figure out how to go about performing signat...