cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4506
Views
10
Helpful
4
Replies

UCS integration with ACI

abahati
Cisco Employee
Cisco Employee

Hello Team,

 

I am currently studying the VMM integration for ACI fabric in particularly with the hosts connected or operated through vCenter. I have a few clarificiations if you can help me with these please:

 

1. Is it a mandate to integrate VMM with ACI on through AVS or DVS? Can we not integrated the VMs or hosts which are using generic vSwitch? Any particular document to support either of the options with suitable reason or justification?

 

2. This is regarding UCS integrating with ACI. Now we know UCS runs in end host mode. Behind the scenes, UCS has as many hosts as many are the blades in that particular domain. Now when we integrate with ACI, how does the ACI sees the UCS hosts?

Does ACI sees one host with different number of sub-hosts under that particular host? How can we monitor the hosts in the UCS domain that must be using generic vSwitch within the ACI?

 

3. Third and final question is - Is it possible to map or define different hosts in the same UCS domain under different EPGs in ACI fabric? I am sure we can do it but not sure how. To my understanding, one theory can be is to do a static pinning from the hosts in the downstream with particular FI ports on FI which would further be connecting to particular Leaf ports. These Leaf ports then can be mapped statically or mentioned under static path under the required Application EPG.

 

I would be really glad if you could help me clarify these doubts. Let me know if you would like me to add any thing more on this?

2 Accepted Solutions

Accepted Solutions

Jason Williams
Level 1
Level 1

1. Is it a mandate to integrate VMM with ACI on through AVS or DVS? Can we not integrated the VMs or hosts which are using generic vSwitch? Any particular document to support either of the options with suitable reason or justification?

Yes, either AVS or DVS is supported. If you want to use a standard vSwitch, then you cannot use VMM integration. Instead, ACI can connect to the hypervisors as if they are baremetal hosts. Use physical domains with static paths/ports.  

 

2. This is regarding UCS integrating with ACI. Now we know UCS runs in end host mode. Behind the scenes, UCS has as many hosts as many are the blades in that particular domain. Now when we integrate with ACI, how does the ACI sees the UCS hosts?

Does ACI sees one host with different number of sub-hosts under that particular host? How can we monitor the hosts in the UCS domain that must be using generic vSwitch within the ACI?

 

ACI is vendor hardware agnostic to VMM environments; however, there is a minor differnet when intergrating with a rack-mounted/fixed server (ex: UCS C-series) as opposed to blade servers (ex: UCS B series). The APIC can dynamically provision VMM VLANs onto leaf interfaces based on CDP/LLDP adjacencies with leafs and hypervisors. For rack-mounted servers, in order to provision VLANs onto the leaf interfaces, the leaf node must report CDP/LLDP adjacency of the host. The same host must also report CDP/LLDP to the VMM controller (example: vCenter) and VMM controller will report that adjacency to the APIC. APIC will provision VLANs based on that information. For blade hosts, the leaf node will report CDP/LLDP of the leaf <> blade switch neighbors to the APIC. The hypervisors will report CDP/LLDP adjacency of host <> blade switch neighbors to the VMM controller. VMM controllers sends this information to the APIC. APIC will see that both the leaf and the hypervisor are connected to the same blade switch, therfore dynamically pushing the approrpriate VLANs to leaf interfaces which connect to the common blade switch. In regards to host visibility, APIC will see each individual host as they appear to the VMM controller. For example, vCenter will report each UCS-B blade as an ESXi host. APIC will be have info for each ESXi host. 

 

3. Third and final question is - Is it possible to map or define different hosts in the same UCS domain under different EPGs in ACI fabric? I am sure we can do it but not sure how. To my understanding, one theory can be is to do a static pinning from the hosts in the downstream with particular FI ports on FI which would further be connecting to particular Leaf ports. These Leaf ports then can be mapped statically or mentioned under static path under the required Application EPG.

Each EPG isn't necessarily "host" based or "UCS domain" based. The typical mapping of EPGs is 1 EPG to 1 VLAN. You can create static paths from leaf to FI, then decide which hosts can trunk those VLANs in UCSM. 

-JW

View solution in original post

1. One VLAN, say X, can be used only once with one EPG in one Tenant. Can we use the same VLAN X in another EPG outside this tenant (in another Tenant)?

Assuming you're using default configuration, then no. The tenant is not a networking construct. It is only a logical container in the APIC for your network constructs (VRFs and BDs). When programming VLANs to the leaf, it does not use the concept of the tenant. It only goes by VLAN ID's (EPGs) and VXLAN ID's (VRF, BD, EPG). If you were to use the Per-port VLAN feature, then you could use the same VLAN across multiple EPGs (doesn't matter which Tenant the EPGss reside). However when using per-port VLAN to re-use VLAN ID's each EPG for the re-used VLAN must be in a separate BD and separate physical port of the leaf switch. This also uses up more CAM space than the default configuration. 

 

2. So if we would need to define hosts in UCS in separate EPGs in ACI, can we do some thing like below:

(For example):

 

Chassis 1's blades are to be put under EPG A in Tenant A and Chassis 2's blades are to be put in EPG B in Tenant B.

Lets say UCS port 1/1 is for EPG A connecting to Leaf2 1/1 and UCS port 1/2 is for EPG B connecting to Leaf2 1/2.

From the UCS end, we can bind the VLANs and veths from chassis 1 to take path out of UCS only through port 1/1 on UCS. At the ACI end, we can give a static VLAN pool for the VLANs comming in fabric from chassis 1. Also under Application EPG, we can give a static path as 2/1/1 with the respective VLAN.

Similarly we can do for chassis 2 and in the ACI end, the static binding would be 2/1/2.

You could do this, but it is not required. You could pass traffic for different EPG/VLAN/Tenants over the same physical links if you wanted. Example: 

 

Tenant-1 > VRF-1 > BD-1 > EPG-1 using VLAN-1 

-> Static path Leaf 101 Eth1/1

 

Tenant-2 > VRF-2 > BD-2 > EPG-2 using VLAN-2

-> Static path Leaf 101 Eth1/1

 

Leaf 101 Eth1/1 <> Eth1/1 FI-A

 

FI-A Eth1/1 Trunks VLAN 1 & 2 

Blades in Chassis 1 all share VLAN 1

Blades in Chassis 2 all share VLAN 2 

 

You will still get the full separation that you need between the two different EPGs. 

 

If question 2, were to reflect question 1 (re-use VLAN 1 for 2 different EPGs with VLAN 1 for UCS chassis 1 goes through FI-A uplink Eth1/1 and VLAN 1 for chassis 2 goes through FI-A uplink Eth1/2), then I do not think it would be ideal to try to configure this with the UCS environment mentioned. Due to the disjoint layer 2 concept, if you try to separate VLAN 1 across 2 differnet uplinks of the fabric interconnect then you will run into issues with BUM traffic being sent downstream to the fabric interconnect from the ACI fabric. I don't believe the FI has a per-port VLAN feature either. 

 

-JW

View solution in original post

4 Replies 4

Jason Williams
Level 1
Level 1

1. Is it a mandate to integrate VMM with ACI on through AVS or DVS? Can we not integrated the VMs or hosts which are using generic vSwitch? Any particular document to support either of the options with suitable reason or justification?

Yes, either AVS or DVS is supported. If you want to use a standard vSwitch, then you cannot use VMM integration. Instead, ACI can connect to the hypervisors as if they are baremetal hosts. Use physical domains with static paths/ports.  

 

2. This is regarding UCS integrating with ACI. Now we know UCS runs in end host mode. Behind the scenes, UCS has as many hosts as many are the blades in that particular domain. Now when we integrate with ACI, how does the ACI sees the UCS hosts?

Does ACI sees one host with different number of sub-hosts under that particular host? How can we monitor the hosts in the UCS domain that must be using generic vSwitch within the ACI?

 

ACI is vendor hardware agnostic to VMM environments; however, there is a minor differnet when intergrating with a rack-mounted/fixed server (ex: UCS C-series) as opposed to blade servers (ex: UCS B series). The APIC can dynamically provision VMM VLANs onto leaf interfaces based on CDP/LLDP adjacencies with leafs and hypervisors. For rack-mounted servers, in order to provision VLANs onto the leaf interfaces, the leaf node must report CDP/LLDP adjacency of the host. The same host must also report CDP/LLDP to the VMM controller (example: vCenter) and VMM controller will report that adjacency to the APIC. APIC will provision VLANs based on that information. For blade hosts, the leaf node will report CDP/LLDP of the leaf <> blade switch neighbors to the APIC. The hypervisors will report CDP/LLDP adjacency of host <> blade switch neighbors to the VMM controller. VMM controllers sends this information to the APIC. APIC will see that both the leaf and the hypervisor are connected to the same blade switch, therfore dynamically pushing the approrpriate VLANs to leaf interfaces which connect to the common blade switch. In regards to host visibility, APIC will see each individual host as they appear to the VMM controller. For example, vCenter will report each UCS-B blade as an ESXi host. APIC will be have info for each ESXi host. 

 

3. Third and final question is - Is it possible to map or define different hosts in the same UCS domain under different EPGs in ACI fabric? I am sure we can do it but not sure how. To my understanding, one theory can be is to do a static pinning from the hosts in the downstream with particular FI ports on FI which would further be connecting to particular Leaf ports. These Leaf ports then can be mapped statically or mentioned under static path under the required Application EPG.

Each EPG isn't necessarily "host" based or "UCS domain" based. The typical mapping of EPGs is 1 EPG to 1 VLAN. You can create static paths from leaf to FI, then decide which hosts can trunk those VLANs in UCSM. 

-JW

Hello Jason,

 

THANK YOU SO MUCH FOR A DETAILED EXPLAINATORY AND A LOT BETTER ANSWER THAT I HAVE BEEN LOOKING FOR!!!!!

 

Now in regard to the answer for my third question, I wish to clarify a few more things:

1. One VLAN, say X, can be used only once with one EPG in one Tenant. Can we use the same VLAN X in another EPG outside this tenant (in another Tenant)?

2. So if we would need to define hosts in UCS in separate EPGs in ACI, can we do some thing like below:

(For example):

 

Chassis 1's blades are to be put under EPG A in Tenant A and Chassis 2's blades are to be put in EPG B in Tenant B.

Lets say UCS port 1/1 is for EPG A connecting to Leaf2 1/1 and UCS port 1/2 is for EPG B connecting to Leaf2 1/2.

From the UCS end, we can bind the VLANs and veths from chassis 1 to take path out of UCS only through port 1/1 on UCS. At the ACI end, we can give a static VLAN pool for the VLANs comming in fabric from chassis 1. Also under Application EPG, we can give a static path as 2/1/1 with the respective VLAN.

Similarly we can do for chassis 2 and in the ACI end, the static binding would be 2/1/2.

 

Please confirm if I am understanding it correctly?

 

Best Regards,

Aadish Bahati

1. One VLAN, say X, can be used only once with one EPG in one Tenant. Can we use the same VLAN X in another EPG outside this tenant (in another Tenant)?

Assuming you're using default configuration, then no. The tenant is not a networking construct. It is only a logical container in the APIC for your network constructs (VRFs and BDs). When programming VLANs to the leaf, it does not use the concept of the tenant. It only goes by VLAN ID's (EPGs) and VXLAN ID's (VRF, BD, EPG). If you were to use the Per-port VLAN feature, then you could use the same VLAN across multiple EPGs (doesn't matter which Tenant the EPGss reside). However when using per-port VLAN to re-use VLAN ID's each EPG for the re-used VLAN must be in a separate BD and separate physical port of the leaf switch. This also uses up more CAM space than the default configuration. 

 

2. So if we would need to define hosts in UCS in separate EPGs in ACI, can we do some thing like below:

(For example):

 

Chassis 1's blades are to be put under EPG A in Tenant A and Chassis 2's blades are to be put in EPG B in Tenant B.

Lets say UCS port 1/1 is for EPG A connecting to Leaf2 1/1 and UCS port 1/2 is for EPG B connecting to Leaf2 1/2.

From the UCS end, we can bind the VLANs and veths from chassis 1 to take path out of UCS only through port 1/1 on UCS. At the ACI end, we can give a static VLAN pool for the VLANs comming in fabric from chassis 1. Also under Application EPG, we can give a static path as 2/1/1 with the respective VLAN.

Similarly we can do for chassis 2 and in the ACI end, the static binding would be 2/1/2.

You could do this, but it is not required. You could pass traffic for different EPG/VLAN/Tenants over the same physical links if you wanted. Example: 

 

Tenant-1 > VRF-1 > BD-1 > EPG-1 using VLAN-1 

-> Static path Leaf 101 Eth1/1

 

Tenant-2 > VRF-2 > BD-2 > EPG-2 using VLAN-2

-> Static path Leaf 101 Eth1/1

 

Leaf 101 Eth1/1 <> Eth1/1 FI-A

 

FI-A Eth1/1 Trunks VLAN 1 & 2 

Blades in Chassis 1 all share VLAN 1

Blades in Chassis 2 all share VLAN 2 

 

You will still get the full separation that you need between the two different EPGs. 

 

If question 2, were to reflect question 1 (re-use VLAN 1 for 2 different EPGs with VLAN 1 for UCS chassis 1 goes through FI-A uplink Eth1/1 and VLAN 1 for chassis 2 goes through FI-A uplink Eth1/2), then I do not think it would be ideal to try to configure this with the UCS environment mentioned. Due to the disjoint layer 2 concept, if you try to separate VLAN 1 across 2 differnet uplinks of the fabric interconnect then you will run into issues with BUM traffic being sent downstream to the fabric interconnect from the ACI fabric. I don't believe the FI has a per-port VLAN feature either. 

 

-JW

Hello James,

 

Thanks again. That really cleared my doubts. Thanks a lot! And yes, ofcourse I do not about the VLAN 1 being used in the UCS which is not possible actually. I am myself from Cisco UCS TAC Team and know about the VLAN mapping and communication work :)

I actually meant to use the same VLAN 1 with ACI coming from a different domain or a different UCS.

 

But I am in a lot more confident and comfortable position now in regard to connecting UCS to ACI!

 

Thanks again. Have a great one ahead.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Save 25% on Day-2 Operations Add-On License