People new to Cisco UCS need to get used to new devices in the data center: the 6100XP Fabric Interconnects. There are two critical points to understand about 6100s:
The 6100s are NOT switches, and there are a number of reasons that this is important.
What the difference is between End-Host Mode and Switch Mode, and more importantly which one to choose.
It's important to understand these two points otherwise the 6100s will be mis-understood leading to incorrect configurations, unexpected behaviour and a significant lengthening of an otherwise simple adoption.
6100s are not switches
Just VMware's ESX is wrongly accused of "being Linux", the 6100s are wrongly accused of "being switches" and the similarity is accurate.
VMware ESX is wrongly accused of being Linux because two components are Linux-derived. First, the Console OS in ESX is a Red Hat derivative. Second, some I/O drivers are derived from Linux 2.6 drivers. But ESX is significantly more than just these components.
Likewise, 6100s do have layer 2 switching functions but they have much, much more. A cut-down NXOS does exist on the 6100s, but it isn't like a Nexus 5k or anything like that.
6100s have some things in common with Nexus switches, but much more that is unique to 6100 such as reduced and automated configuration and administration and all the server configuration aspects like BMCs, and integrated FEXs.
End-Host Mode and Switch Mode
In almost all cases, use End-Host Mode. There are three exceptions to this:
You want to connect a NAS directly to the 6100s.. Noooooooooooooooo….
You have a disjoint LAN above UCS because EHM will not expect nor handle such craziness… Your outbound traffic is dynamically pinned which means EHM expects the same L2 network consistently across all the uplink ports.
You connect the 6100s into a pair of HSRP routers that are STP roots on different VLANs. Normally, the 6100s connect into your garden variety Catalyst or Nexus switches though…
Read the original post on ViewYonder.com for more detail, but these two points are very important when first understanding Cisco's UCS and the 6100s in particular.