Teaming Problem /Microsoft exchange server 2003

faisal_ghaus · ‎05-31-2008

Hi

Please help me out with problems we are faceing we have a Microsoft Exchange server 2003 with Microsoft clustering service that with teaming configured Active/Active it functions properly the server is IBM 3650 server with broadcom Nic with Teaming configured .

The problem we face often is we loost connectivity to the server nodes at that time it si only accessible from LAN but not from WAN when you check the ARP cahhe of the router you find ip address of the server with HSRP mac but not the actual mac of the server but when you add static entry of the server with ip and mac it resolves once you remove the static entry and check the arp cache you find server ip address and its proper mac please advise resolution of the servwer

mslavin · ‎06-03-2008

Hi Faisal,

I noticed on another post you recommended users use Active/Standby, but in this post, you are using Active/Active. Personally, without going any deeper, I'm also a fan of Active/Standby and have seen it resolve a lot of strangeness and unexpected behavior with teaming.

Question 1) Have you tried this with Active/Standby?

Beyond that, do you have more details on the issue?

Question 2) How often does it happen?

Question 3) In the router that has the wrong MAC, is this in the same IP subnet as the teamed IP address?

Question 4) Can you share the output of a "show mac-add d" command on the router while things are working and while things are failing?

Question 5) Do you have a network diagram you can share that shows where everything hooks up and IP's and MAC's?

Thanks, Matt

faisal_ghaus · ‎06-03-2008

Hi

Yes I did suggested.I was in discussion with different techies for the same and everone recommended me the Active/Standby but no one have a proper justification for it.

Yes we have configured Active/Standby yesterday only so far no problem faced

before we were useing Active/Active.

Details of the problem is al follow we have several problems pertaining to this server.

It is a Exchange server 2003 running Microsoft clustering service IBM server 3560.

With teaming configured Broadcom NIC with teaming configured.Few findings I have done with my server team .

First time we have seen duplication of IP address message on our core routers with HSRP gateway we found out that the Mac address apperaing on the router was of Exchange server LAN card once disconnected the server from LAN every thing turned ok.

Second incident virtual ip address of the cluster was pingable from LAN but not from WAN when we traced the ip address it was showing the loop on the routers ethernet .

remedy we did for the resolution was we have added static ARP entry with the ip address of the server the issue was resolved .

Why did we added static ARP entry in router arp cache when we cleared arp entry and again pinged the server with debbuging of arp found out that ping repy one is coming from Server MAC address and second from Routers HSRP mac address arp cache is configured with no on entire router .

It has happened 3rice in a months time.

Yes the server an LAN segment of router is on same segment

mslavin · ‎06-04-2008

Hi Faisal,

On the discussion between A/A and A/S, there are a lot of variables and it differs by vendor and what A/A you choose.

For example, if both uplinks go to a common switch (or same virtual switch, such as a 3750 stack or 6500 VSS pair), then you can use A/A based on EtherChanneling (Broadcom calls this Trunking and usually offers both static (they call this "Generic Trunking") and LACP, and you need to make sure you configure the Cisco side for the proper Etherchanneling as well). In my experience, this sort of A/A works very well and I highly recommend it.

If, however, the NICs go to two separate physical switches, then you can not use an A/A based on EtherChannel, and you are then stuck with relying on the vendors proprietary forms of A/A, which are not generally well defined to the public and do seem to have strange interactions on upstream networks (based on my own experiences). For Broadcom, their proprietary A/A (they call it SLB and requires putting multiple NICs into the "Active" group) is session based (at least it was last time I tested it), which means that a given session between two devices will always use the same NIC, but which NIC it uses for any given flow is not configurable (at least not the last time I checked) and can not be controlled by the end user.

A couple of areas of heartburn with this form of A/A:

A) If you have an issue on one side of the network that does not take the NIC link down hard (i.e. an upstream failure), from an end-user perspective, the problem will look intermittent (some sessions work, others fail), and this is much more difficult to troubleshoot if the thing were to either just work or fail completely. Yes there are tools such as Broadcom's "Live Link" feature that can help to address some of this, but this is not without it's own set of issues that I don't want to go into here :-)

B) In certain cases, it will not load balance very effectively. If you don't believe me, run some tests in both A/S and A/A and compare your results. In my experience, it sometimes can make good use of both NICs, but usually not as much as most users think, and I've never seen what I would call "great" load balancing..

C) I have seen many cases where upstream load-balancers can get very confused seeing streams of data from the same end IP coming from more then a single direction. This results in some very strange and intermittent operation with upstream load-balancers (I have worked with many customers trying A/A with load balancers and many report intermittent issues until they switch to A/S).

So the end result, with proprietary A/A, you may not get the load balancing you expect, the data flows are non-deterministic, making troubleshooting more difficult, and upstream load-balancers can get very confused by it. This leads me to tell customers that, yes, I have made A/A work, but owing to all of the many variables and possible complications, I just do not recommend it.

So that details some of the trials and tribulations with A/A teaming :-)

For your specific issue, I can't say I've ever seen this specific issue, where the MAC is from the HSRP and not the server.

Question: What is the default gateway assigned to the Server? Is it the HSRP address that you indicated was using the servers MAC, or is it a different routing device?

Thanks, Matt