Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

2.0 Upgrade

Has anyone completed the upgrade to 2.0?  And if so, how was your experience?

I have the firmware bundles downloaded and am preparing to run the upgrade but wanted to know if anyone has run into any issues.  I have checked and do not have any FCoE VLAN overlap. 

41 REPLIES
New Member

2.0 Upgrade

We will be upgrading this week in our test lab (6120 4 chassis), and than moving onto production (6140 14 chassis). Will keep everyone posted.

Craig

New Member

2.0 Upgrade

I went ahead and ran the upgrade this morning.  We only have a couple of production VMs running on our UCS as we wanted to wait for 2.0 before migrating our entire production environment.

I used the 1.4 to 2.0 upgrade instructions as my point of reference but the process is no different than a usual minor version upgrade.  Everything went well. The process took around an hour for me to do both FIs, UCSM and our blades (we have one chassis).  Both VMs remained up and fully accessible the entire time except for one icmp time out during a vMotion between hosts to allow me to reboot all blades.  

I will let you know if I run into any post-upgrade issues.

Cisco Employee

2.0 Upgrade

Craig/Peter,

We greatly value your feedback.  Please do let us know if you have any questions or issues with the upgrade.

Look forward to hearing back once you've deployed UCSM 2.0

Regards,

Robert

New Member

2.0 Upgrade

Robert,

So far I have not run into any issues... but then again I have not yet tried to take advantage of any of the new features in 2.0 or made any significant config changes.  I have merely upgraded each component and then ensured that my pre-upgrade configuration is running stable; which it appears to be. 

My only comment so far is that the process of applying the service profile to the blade, and all the other POST/memory test/configuration that occur when you power on a blade, seems to take considerably longer.  The O/S itself boots just as quickly but it takes a while to get to that stage.

Regards,

Pete

Cisco Employee

2.0 Upgrade

Peter,

That's the first I've heard/seen that.  Let me chase up development to see if there were any BIOS/POST changes to increased the diags.

==> Have you already updated the blade BIOS to the latest also?

Thanks for the feedback.

Regards,

Robert

New Member

2.0 Upgrade

==> Have you already updated the blade BIOS to the latest also?

I created a Host Firmware Packages and a Management Firmware Package for my B200 M2s.  In the host firmware package I selected 2.0(1m) for the Palo, S5500.2.0.1a.0.08112011606 for the B200 M2 BIOS.  For the Management Firmware Package I selected 2.0(1m) for the B200 M2.   I applied both of these packages to my service profiles and restarted.   Is there anything else I need to do?

Cisco Employee

Re: 2.0 Upgrade

No Peter  - You have all the bases covered.

I discussed this with our BIOS team and with the new EP BIOS on B200's there are some additional steps in the POST process which explain the slightly longer diag.  Also, we've switched from using an internal PXE boot to using vMedia for the association & blade programming process.  All this will improve stability, but comes at a slight increase in time for POST.  I have also discussed this with other colleagues and this has been noticed - so it's expected behavior.

Hope this helps.

Regards,

Robert

New Member

2.0 Upgrade

Hi!

I did the upgrade yesterday. I had the overlapping FCoE VLAN in my default VSAN. So far I have noticed one problem:

On our ESXi servers we use the Palo (M81KR) adapter. After updating BIOS, Adapter and CIMC our vNICs have changed their PCI numbering. Here is what I see looking in the esx.conf file in ESXi:

Before upgrade:

vmnic0 - 08:00.0

vmnic1 - 09:00.0

vmnic2 - 10:00.0

vmnic3 - 11:00.0

vmnic4 - 12:00.0

vmnic5 - 13:00.0

vmhba1 - 14:00.0

vmhba2 - 15:00.0

After upgrade (notice how ESXi changed the vmnic names since the PCI address changed):

vmnic0 - 08:00.0

vmnic6 - 08:00.1

vmnic7 - 08:00.2

vmnic8 - 08:00.3

vmnic9 - 08:00.4

vmnic10 - 08:00.5

vmhba3 - 08:00.6

vmhba4 - 08:00.7

Since vmnic0 still has its old PCI address I can still reach my admin interface on ESXi, but my other vSwitches have problems with the new names.

Anyone else seeing this?

Regards

Erik

2.0 Upgrade

Are you using a vNIC/vHBA placement policy to force your vNICs/vHBAs to be deteceted in a specific order?

New Member

2.0 Upgrade

No, I am not using vNIC / vHBA placement policies.

2.0 Upgrade

That is probably why the PCIe ordering changed then. With a placement policy you can force the correct order so that you can gaurantee vmincX will always be dedected by ESXi as the same adapter.

New Member

2.0 Upgrade

Thanks for your answer.

I am not sure that is the problem. The order is the same, but the PCI numbering changed. Look at for example vmnic1:

Before:

vmnic1 - 09:00.0

After:

vmnic6 - 08:00.1

So to me it looks like the order is still the same, but instead of numbering them 08, 09, 10 it has changed to 08:00.0, 08:00:1, 08:00:2 etc.

New Member

Re: 2.0 Upgrade

Erik,

Could you please let me know what blade model was this observed on?

I have a test result on a B440, after the the upgrade to 2.0(1m) ESXi renumbered from 0,1,2,3 to vmnic 4,5,6,7.

I have also seen the same being affected for B230 as well. This disrupted all network access to hosts. I had to manually edit the /etc/vmware/esx.conf file return the values to its original

Thanks

New Member

2.0 Upgrade

Hi skumar7193!

I have B200 M2 blades with Palo, M81KR, adapters. I have tried to downgrade CIMC, Adapter and BIOS on the blade. Still the same PCIe numbering. I then reinstalled ESXi. No change... Whatever happened I can not get the "old" numbering back.

Do you have any vNIC/vHBA placement policies?

Cisco Employee

Re: 2.0 Upgrade

Can you both also advise if youi're using one or two adaptors (in case of B250/B440).

Thanks,

Robert

New Member

Re: 2.0 Upgrade

I have a dual adapter and yes,I have a placement policy. The ESX version is 5 if it matters, I missed that in my earlier post

-Sukesh Kumar

New Member

2.0 Upgrade

Hi Robert!

Since we only have B200 M2 there is only one adapter in my case.

/Erik

Cisco Employee

2.0 Upgrade

Let me fire this up in my lab and get back to you today.  I'll test both blades with ESX 4 & 5.

Stay tuned.

Regards,

Robert

New Member

2.0 Upgrade

Update went well in our lab (6120 with 4 chassis). Did note that the IOM's were in "Updating" Status for like 15 minutes simultaneously after FI’s rebooted. That was odd, but no interruption to chassis traffic. Other than that, went like any other code update. We are not doing any virtualization at this time, so cannot comment on that. 

Will be doing our production system (6140 with 4 chassis) that is under heavy load. That will be the real test.

Craig

My UCS Blog http://realworlducs.com
New Member

2.0 Upgrade

In our lab after the update this critical error showed up. Appears and FCOE vlan and default VLAN have ID of 1.

My UCS Blog http://realworlducs.com
Cisco Employee

2.0 Upgrade

Overlapping FCOE VLAN and Ethernet VLANs  no longer allowed with UCS 2.0.

New Member

2.0 Upgrade

So best fix for that (we are not using FCOE) would be to change FCOE vlan to and unused?

Craig

My UCS Blog http://realworlducs.com
Cisco Employee

2.0 Upgrade

Changing the FCOE vlan-id will clear the overlapping issue.

2.0 Upgrade

Yes, that is a best practice anyway that should have been followed pre-2.0. If you were to deploy Nexus 5ks with FCoE you would have to do the same for FCoE traffic.

Cisco Employee

2.0 Upgrade

The FCoE VLAN Change was documented in the release notes:

http://www.cisco.com/en/US/docs/unified_computing/ucs/release/notes/OL_25363.html#wp226040

There was also a Software Advisory in the Release notes pertaining to this change. 

http://www.cisco.com/web/software/datacenter/ucs/UCS_2_0_Software_Advisory.htm

As Jermey pointed out, your FCoE VLAN should never be used for anything other than FCoE, and does not need to exist outside of UCS.

Regards,

Robert

Cisco Employee

2.0 Upgrade

In regards to the PCI re-ordering issue we've identified two bugs.

CSCts86890 - B230-M1: PCI bus numbers change after an upgrade from 1.4 to 2.0

Description:       

When the BIOS is upgraded on a B230-M1 blade from 1.4 release to 2.0 release, the PCI bus enumeration will shift by one bus number.  This can potentially cause  certain OS like VMWARE ESX, Windows to see the old VNIC’s and VHBA’s with new PCI address and could result in those interfaces being non operable unless the configuration is changed in the OS.  In case of ESX, a workaround would be edit the esx.conf with the new PCI address  and to modify the vswitch configuration.

Workaround: 

At this time there are only two ways to correct the issue

1. In case of ESX, manually modifying the ESX.conf file to re-map the vmnic numbering.

2. Revert entire UCS System from 2.x -> to 1.4.x : VMware & Windows PCI ordering reverted back to original mapping and old NIC numbering restored Additional workarounds are under investigation.

CSCts96949 - PCI Device address changes after an upgrade from 1.4 to 2.0 release on VIC device

Description:       

When a system is upgraded from UCS 1.4 and earlier releases,  the PCI device address of existing Service Profile VNICs and VHBAs in case of VIC adapters changes.  This can potentially cause certain OSes such as VMware ESX and Windows to see the old VNICs and VHBAs with new PCI addresses, which could result in those interfaces being non operable unless the configuration is changed in the OS.  In the case of ESX, a workaround would be to edit the esx.conf with the new PCI address and to modify the vswitch configuration.

Workaround:

There are two current workarounds.

1. In case of vSphere, manually modifying the ESX.conf to re-map the PCI -> vmnic numbering

2. Downgrading the BIOS of the blade from 2.x -> 1.4.x

**Software patches for these two issues are currently being developed and will be made availalbe as soon as possible once they clear QA.

I will update this thread as soon as the patches have been released.

Regards,

Robert

New Member

2.0 Upgrade

On our 6104 UCS with 14 Chassis of B200 M2 Blades we ran into bug

(CSCTR74091)

http://www.cisco.com/web/software/datacenter/ucs/UCS_2_0_Software_Advisory.htm

Right after our first step the UCSM activation all the blades went into re-config and went offline (crashed the OS). Many had re-numbered NIC's.

FCoE VLAN was changed to 2 before update. There were no VLAN overlaps etc, UCS was perfect before update, 0 errors across top left of screen. After UCSM Activeate to 2.0 Code all blades has this error:

http://realworlducs.com/images/619207123.png

In the end we had to disassociate and associate all the service profiles to get the blades back online. These are all Windows 2008R2 machines in an Armada 4.1 encoding farm.

Also one of our vNIC pin groups will not work on failover mode after rollback. Had to switch to one fabric, and not check Enable Failover. Will work with TAC to figure that out.

TAC said a patched version of 2.0 will be out in 1 to 2 weeks that does not crash your UCS while updating. Looking forward to that.

A picture is worth a thousand words so I will summarize our 2.0 upgrade with this:

http://blogs.e-rockford.com/applesauce/files/2011/05/car-flying-off-cliff.jpg

Craig

My UCS Blog http://realworlducs.com
New Member

2.0 Upgrade

Would you summarize this as "completed, but not working?"

In all seriousness, thanks for the information, hopefully we can avoid our own trainwreck.  Andy L found your articles/posts and shared them with us via facebook.  Good stuff.

New Member

2.0 Upgrade

Hi Craig, do you figured out the  Pin Group Issue? i think i have the same problem, i created a Pin Group  for 2 Uplinks, at service profile 1 nic is pinned to FIA and the other  one is pinned to FIB, all Blades are ESXi 4.1.

I  tried the Failover feature of UCS disabling one Uplink port of two in  the Pin Group in this case was FIA, Mgmt traffic failover to FIB and was  working, when i tried to get it back VMware detects that the first NIC  is online and it tries to forward trafic to the restored Uplink but its  not working, i had to disable the FIA uplink to get ESXi Management  Console to get the cluster up again.

Is this kind of your scenario?

Regrads

Luis Lozano

5946
Views
1
Helpful
41
Replies