HSRP and Loop Problems with UCS

[gerrit.lehr]

Hi community,

I am currently working with the Cisco support on this but since this won't progress for a couple of days I thought it can't hurt to ask around the community and share my situation.

I have a UCS system with two FIs connected via one 1GB Uplink each to an upstream Enterysys e7 Switch. This worked just fine for a couple of weeks. When the system was due to go into production, we added another pair of 1GB Uplinks to the FIs. This caused two Cisco Routers (3750 and 3850) configured to run HSRP to the LAN to fail over the virtual HSRP IP and back all the time, of course disrupting all connections over the default LAN gateway. This problem only got resolved when we took the UCS system off the LAN.

Since I didn't want to risk another network dispution, I connected the FIs to an unmanaged test switch. Even stranger, on this switch I was only able to connect one uplink port overall (from only one FI). As soon as I added another one from the same FI or a first uplink from the second FI, both links went down with loop errors. This happend with Firmware 2.0 and 1.43. Since this second problem occured first on the unmanagemed test switch, I am assuming that the unmanaged switch is the cause of the problem. However, I thought there are no requirements to the upstream switch at all, as long as the FIs are running in End-Host mode. Is there anything I am missing regarding the switch?

I am going to test this on the production switch soon and hope that the loop problem was caused by the unmanaged switch. However, this leaves me with the HSRP problem. This kinda sounds like a spanning tree related problem since from what I know spanning tree issues can cause trouble with HSRP. But there should be no STP involved at all since the FIs run in Ethernet End-Host mode.

Any ideas on this one?

Best regards,

Gerrit

[HAROLD MEIER]

Hi Gerrit,

My organization is also using Enterasys e7 switches upstream from UCS, and we experienced a number of STP and multicast issues after installing UCS. We uplink 4x10Gb ports from each FI. I'd say to make sure STP is turned off for all the uplink ports. Even though UCS is in end host mode, treat it like you would a VM host with a virtual switch inside of it. The enterasys switches will see traffic for the same mac address coming out of different ports and will assume it is a switch. We did initially see issues with our Enterasys core routers running VRRP.

Our problems were not always directly UCS or Enterasys related either, but instead we have had a combination of the nuances in Enterasys, UCS, and some 3rd component on the network causing problems.

For example, we had a Microsoft multicast NLB load balancing cluster in VMware on UCS and a MS TMG proxy server on a single VLAN with 2 different subnets. This caused an incident where the TMG proxy saw traffic for the second subnet and responded with ICMP redirect messages. The UCS does not forward IGMP group info from the NLB cluster, which causes multicast storms in the Enterasys segment with a destination on the 2nd subnet (which was picked up by the TMG server), and there was an Enterasys firmware bug that required extra CPU to process ICMP redirects. The back and forth between the NLB and the TMG proxy combined into a gigantic multicast storm, and all the ICMP redirects pushed the N7 CPUs to their limits.

As you can imagine this was not easy to diagnose. Please post if you find a solution. It's nice to find another person out there with UCS on Enterasys.

Regards,

Harry

[gerrit.lehr]

Hi Harold,

thanks a lot for the reply. This does indeed sound very similar to the issues my customer is seeing and I will check the STP config of the switch tomorrow and definately let you know about the result!

Did you manage to solve the problems by disabling STP for the uplink ports or are you still having some troubles?

Thanks again,

Gerrit

[HAROLD MEIER]

As far as the STP issues, yes disabling it on the uplink ports worked. You may also want to bond those uplinks into a port channel in UCS and configure a "lag" on the Enterasys.

[gerrit.lehr]

Hi Harold,

I did try to get this working but had no luck so far. Neither disabling STP on the uplink ports nor enabling and configuring it as edge ports worked. It still does a topology change as soon as  more than one uplink is connected and screws up the HSRP. Currently the config is as folllows:

Next thing is trying to find someone who is more familiar with this switch since neither me nor the customer has in depth knowledge of the device and it is EOL. Do you happen to have any further advice?

Regards,

Gerrit

[HAROLD MEIER]

I consulted with our network team and they concurred that setting up a lag (port-channel) port would be their next step. I would recommend setting up a lag and disabling stp on both the physical port and the lag. That would make the interface appear as a single port like it was before. In UCS you can bond those uplink ports into a port-channel with the ID number given in the "adminkey" parameter in Enterasys. Once you have it working for 2 ports on 1 switch, then try adding the other.  

Here is an example on how to configure:

set lacp singleportlag enable

set lacp flowregeneration enable

set port lacp port ge.1.1 aadminkey 50

set port lacp port ge.1.2 aadminkey 50

set port lacp port ge.1.1 enable

set port lacp port ge.1.2 enable

set lacp aadminkey lag.0.1 50

set spantree portadmin lag.0.1 disable

set spantree portadmin ge.1.1 disable

set spantree portadmin ge.1.2 disable

Commands to check lacp/lag status

sh lacp ge.1.1

sh port lacp port ge.1.1 status summary

sh port lacp port ge.1.1 status detail

sh port status lag.0.1

[l.waldenberger]

have you configured end-host-mode or switch-mode?

[gerrit.lehr]

The switch is running in Ethernet End Host mode.

Thanks heaps for all the information, I will try configuring a portchannel between the devices.