UCS B200-M1 Setup - your advice is truly appreciated

Unanswered Question
May 12th, 2010
User Badges:
  • Cisco Employee,

hi:


I am new in UCS. Need to implement a UCS project and I have read quite a bit, but really need someone with hands-on experience to vet through the following:


BOM - 1 unit of UCS 6120XP, 1 unit of UCS 2104XP, 1 unit of UCS 5108 chassis, 2 unit of UCS B200-M1 blades, no Cisco rack mount kit, 1 unit of Cisco M81KR mezzanine card per blade., 2 units of SAS 146GB HDD per blade, 2 UCS 6120 power supplies, 2 power supplies for UCS 5108, 1

Single phase AC power module for UCS 5108
, etc.


Steps

=====


1. Install the hardware (simplify):


    - slide the 2 blade into the UCS 5108 chassis,

    - connect the 4 Fabric Extender cables from UCS 5108 chassis to the single UCS 6120XP

    - connect the 4GB FC uplink from the UCS 6120 XP's Expansion Module to a SAN Switch in the infra

    - Connect the 10GB Ethernet uplink from UCS 6120 XP's fixed module to a LAN Switch in the infra

    - Power On all the hardware


2. Setup the single unit of UCS 6120 XP via the console port with the RJ-45-RS-232 adapter and a DB9 adapter and assign a public IP, connect up with a CAT 5/6 UTP cable from Management Port, M0, to a Production Switch's port for Out of Band Management.

3. Launch the UCS Manager via a browser pointing to the assigned IP at step 1.

4. Setup Chassis Discovery Policy

5. Setup the Power Policy


Question: Based on what I have for the power supplies, which policy should I choose? Non-redundant or N + 1?


6. Use UCS Manager to setup Server Port and Uplink Port

7. Configure VLAN


Question: How many vNIC will be presented by M81KR?


Question: So the VLAN id has to be a Production VLAN id, I presume? I just can not put in any fictitious VLAN id, I guess. How about native VLAN? do we have to check it for all the VLAN I am going to create, I presume no? How about Trunking? Do I have to check it for all the VLAN?


Question: Since I only have 1 UCS 6100XP, what should I do in this case with regard to the vNIC? How many vNIC do I need?

Question: I will be using UCS M81KR so is there any special procedures here since VN-Link will be used? Do I need the Cisco Nexus 1000K software and is it free?


Question: How many VLAN do I really need? Which vNIC (say there is 2 being presented per blade, vNIC0 and vNIC1) should be paired with the specific VLAN?


VLAN1 is the default so do not touch

VLAN2, for example, for B200-M1 blade KVM, hence blade derives their IP from this

VLAN3, for example, for VSAN assuming I have only 1 SAN

VLAN4. for example, for VMware Service Console

VLAN5, for example, for VMware vMotion

VLAN6, for example, for customer production application 1,

VLAN7, for example, for customer production application 2,

etc.



8. Configure VSAN


Check with customer for the right VSAN id to use.

Configure NPIV in the SAN switch. Assign a unique VLAN to the Production VSAN. Does this VLAN has to be unique i.e. not shared with anyone? How to associate this VSAN with a vHBA presented by Cisco M81KR since I have only 1 UCS 6120XP i.e. 1 fabric only? I presume I do not have to care about which vNIC to use in this case, since selecting vNIC0 or vNIC1 will end up the same with ONLY 1 physical connectivity from the single piece of UCS 6120 t o a SAN switch.


Question: How many vHBA will be presented by M81KR?


9. Pool


Question: Must I create the various pool like MAC pool, KVM Management IP Pool, UUID Pool, WWName Pool, etc?


10. Service Profile


I know I must create Service Profile.


As I have only 2 blade and let assume that the 2 blades are running different applications-App A in blade 1 in VLAN10 and App B in blade 2 in VLAN20. Assuming if blade 1 fails, App A will fail over to blade 2 and vice-versa.


Question: In this scenario, how many service profile must I create? I presume it is two. How about those policies and can we skip some of them like Boot Policy if assuming we are all booting from the local HDD?


Question: During Service Porfile creation, I guess I can JUST CHECK either vHBA0 for Fabric A or vHBA1 for Fabric B since I have only 1 UCS 6120XP? Am I right?


Question: For the WWNodeName, must I select from the WWName pool created from Step 9 above? Is it OK if I select the default burnt-in WWNodeName on the M81KR itself? I believe I can not move the Service Profile associated with the blade in this case.


Question: For the vNIC during Service Profile creation, I guess I should just check either Fabric A or B and DO NOT CHECK Enable Failover, am I right?

How about the VLAN Trunking? How about the Native VLAN? Which VLAN should I select in this case?


This is the typical questions which will be asked by lot of other newbies like me.


I truly appreciate some kind souls out there and to them, I think it is a piece of cake.


Thanks.

Ben

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Jeremy Waldrop Thu, 05/13/2010 - 03:48
User Badges:
  • Silver, 250 points or more

To start off with I would highly recommend you add another Fabric Interconnect to this configuration if it is for a production deployment. With just one 6120 you have a single point of failure. If it is for lab then having just 1 6120 is fine.


Power


For just two blades you can set the N+1 power policy but if you were to fully populate the chassis it wouldn't be redundant anymore.


M81KR (Palo) vNICs


The number of vNICs presented to a blade is based on how many vNICs are defined in the Service Profile. You can present from 1 to 54 with the Palo. With just one 6120 that number may be less though. When you create your vNICs you will have to make sure you always select Fabric A because you do not have a Fabric B.


VLANs


The northbound uplinks to your LAN switch have to be trunked with 802.1q and any VLANs on your production LAN will that you will need in UCS have to be allowed on the trunk. In UCSM you will then have to define each one of your VLANs. The VLAN IDs have to match what you are using on your LAN. The number of VLANs you need depend on your existing LAN. If you are only using 1 VLAN then that is all you can use in UCS, you don't make up VLANs for UCS if you are not using them in your existing LAN.


As for the FCOE VLAN ID you need to use something that doesn't exist as a LAN VLAN ID. I usually use high number for this. For example if your VSAN is 4 then use 400 for the FCOE VLAN. If you only have VSAN 1 then you don't have to create any VSANs or worry about the ID because VSAN 1 is the default VSAN.


If you have other VSANs beside VSAN 1 then you must create it as a VSAN in UCSM and the ID must match. After you create the VSAN it is tied to the FC northbound uplinks that goe to your FC switches.


You really only need two vHBAs unless you are uplinking the 6120 to more than two FC switches.


Pools


To take advantage of the statelessness in UCS you must create pools for MACs, WWNN, WWPNs and UUIDs. The KVM IP management pool is required for console (KVM) access to blades.


If blade 1 failes the application on that blade goes down with it, there is no auto-failover unless these blade are running VMware ESX and HA is setup in vCenter.

You can however easily move the Service Profile from one blade to another and bring that failed application back online. This requires that you are booting from SAN though. I would not recommend using the local disk on the blades for the OS, we usually spec out blades with no disks.


Hope this helps, let me know if you have anymore questions.

benlau Tue, 05/18/2010 - 08:23
User Badges:
  • Cisco Employee,

Jeremy:


Thank for the excellent answers, but I still have the following doubts under the heading of Question:


To start off with I would highly recommend you add another Fabric Interconnect to this configuration if it is for a production deployment. With just one 6120 you have a single point of failure. If it is for lab then having just 1 6120 is fine.

*Power*

For just two blades you can set the N+1 power policy but if you were to fully populate the chassis it wouldn't be redundant anymore.


Question-
Customer has ordered 2 units of 2500W power supply unit for a single unit UCS 5108.

Whether we select No Redundancy or N+1 option from the UCSM GUI, physically, do we have to do anything with the 2 units of power supply and/or the power circuit or power source or not?


*M81KR (Palo) vNICs*

The number of vNICs presented to a blade is based on how many vNICs are defined in the Service Profile. You can present from 1 to 54 with the Palo. With just one 6120 that number may be less though. When you create your vNICs you will have to make sure you always select Fabric A because you do not have a Fabric B.

Question-
Since I will have 2 ports per vNIC, so in this case, do I have to match vNIC0 to Fabric A, then match vNIC1 to Fabric A again or not as I ONLY have 1 Fabric Interconnect? The same question for vHBA.

*VLANs*

The northbound uplinks to your LAN switch have to be trunked with 802.1q and any VLANs on your production LAN  that you will need in UCS have to be allowed on the trunk. In UCSM you will then have to define each one of your VLANs. The VLAN IDs have to match what you are using on your LAN. The number of VLANs you need depend on your existing LAN. If you are only using 1 VLAN then that is all you can use in UCS, you don't make up VLANs for UCS if you are not using them in your existing LAN.

As for the FCOE VLAN ID you need to use something that doesn't exist as a LAN VLAN ID. I usually use high number for this. For example if your VSAN is 4 then use 400 for the FCOE VLAN. If you only have VSAN 1 then you don't have to create any VSANs or worry about the ID because VSAN 1 is the default VSAN.

If you have other VSANs beside VSAN 1 then you must create it as a VSAN in UCSM and the ID must match. After you create the VSAN it is tied to the FC northbound uplinks that goe to your FC switches.

You really only need two vHBAs unless you are uplinking the 6120 to more than two FC switches.


Question:
I do not get the meaning of above sentence i.e. You really ,,,

Jeremy Waldrop Tue, 05/18/2010 - 18:12
User Badges:
  • Silver, 250 points or more

*Power*

For just two blades you can set the N+1 power policy but if you were to fully populate the chassis it wouldn't be redundant anymore.


Question-
Customer has ordered 2 units of 2500W power supply unit for a single unit UCS 5108.

Whether we select No Redundancy or N+1 option from the UCSM GUI, physically, do we have to do anything with the 2 units of power supply and/or the power circuit or power source or not?


Answer-

If you have two power sources then plug one power supply into one and one into the other. Other than that there is nothing special about the power. I think this asnwers your question but if not please let me know.


*M81KR (Palo) vNICs*

The number of vNICs presented to a blade  is based on how many vNICs are defined in the Service Profile. You can  present from 1 to 54 with the Palo. With just one 6120 that number may  be less though. When you create your vNICs you will have to make sure  you always select Fabric A because you do not have a Fabric B.Question-
Since I will have 2  ports per vNIC, so in this case, do I have to match vNIC0 to Fabric A,  then match vNIC1 to Fabric A again or not as I ONLY have 1 Fabric  Interconnect? The same question for vHBA.


Answer-

Yes, you will need to selct Fabric A for all vNICs/vHBAs created in the Service Profiles.


You really only need two  vHBAs unless you are uplinking the 6120 to more than two FC switches.


Question:
I do not get the  meaning of above sentence i.e. You really ,,,


Answer-

If you are going to uplink the single 6120 to a single fiber channel switch then you only need 1 vHBA in your service profile. If you are going to uplink a single 6120 to two separate fiber channel switches you will create 2 vHBAs and select the appropriate VSAN for each that corresponds to the uplink fiber channel switch.


Let me know if you need further explanations on this.

Robert Burns Thu, 05/13/2010 - 03:59
User Badges:
  • Cisco Employee,

Answers Inline in Red.


Regards,


Robert


Question: Based on what I have  for the power supplies, which policy should I choose? Non-redundant or N  + 1?

1. This depends on the source of the power, do you have two separate circuits or not?

6. Use UCS Manager to setup  Server Port and Uplink Port

7. Configure VLAN

Question: How many vNIC will be presented by M81KR?

2. The M81KR will present up to a /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} maximum of 58 vNICs (FC+Ethernet) per host.  This is dependent on having 4 uplinks per IO modules.

The formula used to calculate this is:

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}


Max VIFs Per Adapter Interface = {15 * (min acknowledged links between switch and IOM on fabric)} – 2


Question: So the VLAN id has to  be a Production VLAN id, I presume? I just can not put in any fictitious  VLAN id, I guess. How about native VLAN? do we have to check it for all  the VLAN I am going to create, I presume no? How about Trunking? Do I  have to check it for all the VLAN?

3. This is completely dependent on your network infrastructure.  To explain the entire concepts of VLANs & Trunking here would be outside of the scope of this support forum.  I suggest you do some reading here to help you design your VLAN/Network infrastructure.

http://www.cisco.com/en/US/netsol/ns340/ns394/ns431/networking_solutions_design_guidances_list.html


Question: Since I only have 1 UCS 6100XP, what should  I do in this case with regard to the vNIC? How many vNIC do I need?

4. Again this is dependent on your application and networking requirements.  Though you can create multiple virtual adapters, you're still bound by the phsyical limitations of the adapter.  ie. Just because you can create 58 vNICs, doesn't give you 58 x 10G connectivity.  The ability to create multiple vNICs is so you can implement different policies, ACLs, QoS on a per vNIC adapter basis.  The underlaying OS will treat each vNIC as if it were a phsyical and unique PCI device.


Question: I will be using UCS  M81KR so is there any special procedures here since VN-Link will be  used? Do I need the Cisco Nexus 1000K software and is it free?

5. You don't have to use the 1000v software with UCS, but you definately can.  It's not free, but you can download and install it with an included 60 day trial license for up to 16 sockets from www.cisco.com/go/1000v


Question: How many VLAN do I  really need? Which vNIC (say there is 2 being presented per blade, vNIC0  and vNIC1) should be paired with the specific VLAN?

6. Dependent on design.  If you're configuring a Service Profile for a Window host you may only need a single VLAN configured - essentially this would be an access port.  In regards to a hypervisor installation such as ESX, you will most likely want the vNIC to carry mutiple VLANs (trunking) so you can carry your Service Console, VM Data VLANs, VMotion VLAN etc over the same link.  Normally people will have 2 x Fabric Interconnects so you have redudancy.  You will not have this with a single FI.  Even though you can logically create multiple vNICs with the M81KR, you will still have a single point of failure.  So unless you're implementing QoS on your different interface types (Service Console, VMotion, Data etc) you might be fine with a single vNIC carrying all your VLANs.


VLAN1 is the default so do not  touch

VLAN2,  for example, for B200-M1 blade KVM, hence blade derives their IP from  this

VLAN3,  for example, for VSAN assuming I have only 1 SAN

VLAN4. for example, for VMware  Service Console

VLAN5, for example, for VMware vMotion

VLAN6, for example, for customer  production application 1,

VLAN7, for example, for customer production  application 2,

etc.

8. Configure VSAN

Check with customer for the  right VSAN id to use.

Configure NPIV in the SAN switch. Assign a unique VLAN to the  Production VSAN. Does this VLAN has to be unique i.e. not shared with  anyone? How to associate this VSAN with a vHBA presented by Cisco M81KR  since I have only 1 UCS 6120XP i.e. 1 fabric only? I presume I do not  have to care about which vNIC to use in this case, since selecting vNIC0  or vNIC1 will end up the same with ONLY 1 physical connectivity from  the single piece of UCS 6120 t o a SAN switch.

7. The VLAN used to carry your VSAN if referred to as the "FCoE VLAN".  Since you're carrying FC traffic over Ethernet, we should separate that traffic using VLANs from other types of traffic.  Your FCoE VLAN should be unique and not used anywhere else in your network.  It's used only between the hosts & Fabric Interconnects.  This will be defined in the SAN tab of UCSM.  Please see the Configuration Guide for more detail.  You don't "assign" this VLAN to your vNICs, it will be utilized automatically.

Question: How  many vHBA will be presented by M81KR?

8.  See Answer 2.  You only have a single fabric, so there's no point in presenting more than 1.

9. Pool

Question: Must I create the  various pool like MAC pool, KVM Management IP Pool, UUID Pool, WWName  Pool, etc?

Yes.

10. Service  Profile

I know I must create Service  Profile.

As I have  only 2 blade and let assume that the 2 blades are running different  applications-App A in blade 1 in VLAN10 and App B in blade 2 in VLAN20.  Assuming if blade 1 fails, App A will fail over to blade 2 and  vice-versa.

Question: In  this scenario, how many service profile must I create? I presume it is  two. How about those policies and can we skip some of them like Boot  Policy if assuming we are all booting from the local HDD?

One service profile must be created for each server instance.  You only need to configure the policies you want and that are applicable to your implementation.

Question: During Service Porfile  creation, I guess I can JUST CHECK either vHBA0 for Fabric A or vHBA1  for Fabric B since I have only 1 UCS 6120XP? Am I right?

Since you only have one Fabric Interconnect you should only be using the vHBA that corresponds to your Fabric Interconnect.

Question: For the WWNodeName,  must I select from the WWName pool created from Step 9 above? Is it OK  if I select the default burnt-in WWNodeName on the M81KR itself? I  believe I can not move the Service Profile associated with the blade in  this case.

You are correct.  The benefit of "stateless" computing is not being tied to the hardware.  Create logical pools (for MACs, WWNN, WWPN etC) and use these to assign to your Service Profiles.  This way they can "follow" the profile to whatever hardware the profile is associated to.

Question:  For the vNIC during Service Profile creation, I guess I should just  check either Fabric A or B and DO NOT CHECK Enable Failover, am I right?

Yes.  With a single Fabric Interconnect, there's no failover possible.


How about the VLAN Trunking?  How about the Native VLAN? Which VLAN should I select in this case?

See Answer 3.

benlau Tue, 05/18/2010 - 08:33
User Badges:
  • Cisco Employee,

Robert:


Thank for your answers as well. Just 1 more question.


Question: Based on what I have  for the power supplies, which policy should I choose? Non-redundant or N  + 1?

1. This depends on the source of the power, do you have two separate circuits or not?


My Question: Do you mind explain the scenario of having 1 circuit and 2 circuits?


Thanks.

Bennie

Robert Burns Tue, 05/18/2010 - 19:24
User Badges:
  • Cisco Employee,
  • non-redundant—All installed power  supplies are turned on and the load is evenly balanced. Only smaller  configurations (requiring less than 2500W) can be powered by a single  power supply.

  • n+1—The total number of power  supplies to satisfy non-redundancy, plus one additional power supply for  redundancy, are turned on and equally share the power load for the  chassis. If any additional power supplies are installed, Cisco UCS  Manager sets them to a "turned-off" state.

  • grid—Two power sources are turned  on, or the chassis requires greater than N+1 redundancy. If one source  fails (which causes a loss of power to one or two power supplies), the  surviving power supplies on the other power circuit continue to provide  power to the chassis.

http://www.cisco.com/en/US/partner/docs/unified_computing/ucs/sw/gui/config/guide/1.2.1/UCSM_GUI_Configuration_Guide_1_2_1_chapter12.html


Robert

kumararul Wed, 07/31/2013 - 02:30
User Badges:

Dear All,


I just want to know wheather i can install windows 2012 on my cisco UCS B200 M1.


i am bit cnfused because cisco HW and SW interoperability tool says no.


can anyone say wheather is it feasible or not?


Thanks in advance.


regards,

M Kumar.

Keny Perez Wed, 07/31/2013 - 08:24
User Badges:
  • Blue, 1500 points or more

Arul,


The answer is... you might make it work but will not be supported by TAC.


Whenever you have that kind of questions, this tool is extremely helpful

http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html


As you may see there is no possible combination to make W2K12 appear with a B200-M1, but that is definitely possible and supported with the B200-M2s, beggining with UCSM 2.0(5).



Hope that helps you.



-Kenny

kumararul Fri, 08/09/2013 - 21:38
User Badges:

Thanks for the reply Keny.


your link was usefull to conclude on my requirement.


I have my next query below.


Now i am in the process of upgrading DIMM's for B250 M1. i referred supported DIMMS matrix for this model using the link

http://www.cisco.com/en/US/docs/unified_computing/ucs/hw/blade-servers/install/B250.html


I procured DIMMS's having  CISCO PID- UCS-MKIT-041RX-B which is not listed in the above supported link.


Even though it is not listed I am curious to know wheater i can use it in the same blade for building some test setup?


If yes i am interested to know related links to do the same.


Thanks in advance.


regards,

M Kumar





Keny Perez Mon, 08/12/2013 - 11:29
User Badges:
  • Blue, 1500 points or more

Arul,


the PID UCS-MKIT-041RX-B is a memory Kit that if I am not wrong, reseller/distributors use so I can't find that specific part ID, could you please take a look at the actual memory DIMM label and tell me what is the PID there?


Since UCS-MKIT-041RX-B is not the real PID but the name of a DIMM bundle, that might be the reason you cant find that PID in the data sheet.



-Kenny

Actions

This Discussion