Management of Blades Outband and Inband IP

[fr.mueller]

Hi,

i have a question about the KVM IP Adresses used by the blades.

If i understand it right, i have one IP for the blade himself and one for each Service Profile. I could also use only  the blade one. If 

the IP for the blade isn´t move around. the IP for the Service Profile could move to an other blade because of switching Profile to other hardware.

The Blade IP Adresse has to be from the same subnet as the Fabric Interconnects belong to. Please correct me if i am wrong.

Is it possible, to use a IP Adresse from a different IP Range for my service profiles? Or is the different between inband and outband only, that i did not use the mgmt Port on the FI´s.

I did only find the following description in the Management gui configuration guide

"You need to configure an Inband Profile with an Inband VLAN group to select an Inband Network (VLAN)
in Service Profiles and Service Profile templates."

If that is true, how to configure an inband profile?

Thanks 

Frank

[Manuel Velasco]

Hi Frank,

Starting firmware 2.2, you can now access the KVM of the M3 blade server via inband network.  In other words, before with outband you can only access the kvm via the same network as your fabric interconnect management network. With inband you can now use the same network that the blades use for data traffic.

This is how you can configure it.  Please see attached screenshots.

1) You need to create a vlan group with the vlans(s) that you want to use for kvm management.  NOTE: do not associate any uplink interface for this group this is not require. If you do it is possible to bring your production network down. Just assign a name and select the require vlan(s).  If this vlan doesn't exit you will need to create first  in the LAN tab (see step_1_vlan_group)

2) For this step you will create create the inband profile, here you need to associate the vlan group with the profile, notice that you can also select a vlan here for global configuration (see step_2_inband_profile)

3) Once you created the vlan group and inband profile, you can now configure the inband ip for the kvm. You can do this at the server level or service profile (see step_3_cimc_inband or step_4_profile_inband)

Please let me know if this help or if you have any questions.

[fr.mueller]

Hi,

thats what i am looking for. But during my tests, i found out, if i only use the inband ip adress, i am not able to reach the kvm from a browser. 

Also i am not able to reach the kvm from the vcenter server web client.

Is that not possible?

Frank

[Manuel Velasco]

What to you mean by vcenter?   Do you mean ucs central?

[fr.mueller]

If you are using the cisco plugin for vcenter, you are able to start a kvm session insight your vcenter web client. That is only possible when using outband ip adress.

The same for direcly use the ip adress in my browser to open a kvm session.

Frank

[Walter Dey]

Frank

Can you please post the CIMC information, where we can see the IP address and MAC ?

Can you ping successfully this IP from the outside

[fr.mueller]

Hi,

i am able to ping the inband ip from other vlans. Also working with kvm insight the ucs manager works as you could see in the first picture (kvm.jpg).

Also you see that i am not able to open the kvm from a webbrowser.

But only when i am using an outband ip adress i am able to open the kvm from a webbrowser.

Frank

[Walter Dey]

Hi Frank

-> what is  your out of band mgt IP and VIP adresses ?

...Also working with kvm insight the ucs manager works.... -> where do you route between the different subnets

-> any firewalls ?

[fr.mueller]

Hi,

ok, to clarify, I want only to use inband. Not inband and outband together. If possible.

If i using an outband ip adress, its working well. But the ip adress has to be on the same subnet as the FI´s.

Therefor, i wan´t to use a dedicated ip range for KVM. So i created a new vlan, and the configuration manvelas described here. 

I am able to ping the IP Adress for inband when it is assigned to a server with a service profile. There is no firewall between client and ucs.

If i only use the inband ip adress, i am not able to use kvm. I have to configure an outband ip adress. 

Frank

[Walter Dey]

Hallo Frank

Did you try different browsers ?

Which UCS version are you using ? CIMC version ?

What kind of blades ?

Anyway, I would open a TAC case !

Cheers Walter.
 

[Benjamin BOHEC]

Hello Franck,

What is your physical server model?

Note from UCSM Admin Guide 2.2 page 18:

Only Cisco UCS B-M3 and C-M3 later server platforms support inband CIMC access. Inband CIMC access for Cisco UCS B-M1, B-M2, and Cisco UCSC-M1, and C-M2 server platforms is not supported

.An other one in same document page 564:

All Cisco UCS B-M3 and C-M3 servers in Cisco UCS Manager that do not have an inband configuration for the server CIMC will receive an inband network VLAN and IPv4 or IPv6 configuration from an inband service profile when the when the service profile is associated with the server. Removing the network or IP pool name from an inband service profile will delete the inband configurations from a server, if the server configuration was derived from an inband service profile.

So if you have a M3 or later server generation, you can use inband config and only one IP adress assigned to the service profile can be used instead of 2 (service profile and physical server).

If you have M1 or M2 server generation, only OOB config can be applied and you must have 2 IP adresses for the Service Profile and for the physical server.

I found another information on the Admin Guide page 248:

You can configure inband and out-of-band (OOB) VLAN groups to use to access the Cisco Integrated Management Interface (CIMC)onblade andrackservers. Cisco UCS Managersupports OOB IPv4 and inband IPv4 and IPv6 VLAN groups for use with the uplink interfaces or uplink port channels.

I didn't tested this configuration but maybe this could works in case on M1/M2 servers. Try to create a VLAN Group with your CIMC VLAN and no interface definition (globally applied to all FI links). Add this VLAN Group to your LAN Global policy with the CIMC Network.

Add the VLAN to you switch port where your FI mgmt are connected in trunk (switchport mode trunk allowed vlan....) and check VLAN continuity from this port to your gateway. Push your OOB IP address on the expected subnet for the Service Profile and for the Physical Server.

[Walter Dey]

The screenshot that Frank provided shows that it is a B200-M3; and the CIMC shows the proper inband IP address !

[Benjamin BOHEC]

Ok right.

So Franck, if I'm right, you would like to use the KVM Direct Access feature comming with UCS 2.2(1) release.

You would like to open the KVM console directly by entering the CIMC IP adress on a web browser.

Info from > https://communities.cisco.com/community/technology/datacenter/ucs_management/blog/2013/12/18/ucs-manager-221-el-capitan-released

• Direct KVM Access
  • Direct KVM access launches KVM via URL:
    • http://<IP_address of CIMC> or
    • https://<IP_address of CIMC
  • System admins allow server admins to access the KVM console without requiring the UCSM IP address
  • The CIMC IP URLs are hosted on the Fabric Interconnect
  • Supported over out-of-band only

Information confirmed on UCS 2.2 Admin Guide page 681:

Only out-of-band IPv4 management interface addresses are supported for KVM direct access.

With inband CIMC > you can only go to UCS Cluster IP with http and run UCS Manager and/or KVM Manager

Regards,

[Benjamin BOHEC]

Hello,

Thanks for your configuration example.

From our part, we are not able to perform running Inband CIMC configuration with UCSM 2.2(1d) and UCS Central 2.1(2a) with Global Service Templates.

We created on UCS Central a global VLAN CIMC and it is applied to the inband configuration of Service Profiles Templates Management IP Adresses (IP Pool not defined). We created also MGMT IP Pools applied on each services profiles manually depending of the UCS Domain association.

CIMC VLAN is identical between all UCS Domains but with different IP subnets. 

On each UCS Domain we applied the CIMC VLAN to a VLAN Group (no interfaces). Then on LAN Global Policies we added the VLAN Group and we selected the CIMC network (no IP pool selected). On this LAN configuration, root permitions on global policies are disabled by default so if I'm right, all VLAN are available on all orgs.

We are not able to run KVM console (right inband IP applied check on UCSM) and we are not able to ping this interface from subnet gateway. CIMC VLAN distribution check between Gateway and UCM domains OK.

Does we forget some config? We tried local global policies/VLANs/SP/Manual IP inband > No results

Out of band on FI subnet is running well but this IP range cannot be used for CIMC (only done for testing).

Thanks for your help.

[Benjamin BOHEC]

Ok after another check with Network Enginners it was a spanning tree misconfiguration between TOR switches (N5K) and Core Switches (N7K).

Test with UCS Central Global SPT with IP Pools applied to Global SP and UCSM LAN Global Policies (VLAN Groups/ Inband Policy) OK