Restarting Exchange Services

[jns]

We are running Unity V2.4.5. We restart the Exchange services on<br>Monday mornings on our Exchange Servers. We dont restart the<br>services on the Unity server. Upon restart we see many red stops<br>in the application event log on the Unity server, mostly the AV<br>errors are 10002, 1003, 100, 29002.<br>I am curious what impact that restarting the Exchange services<br>has on Unity and what is happening and why? Do we have to restart<br>Unity after Exchange services on another server has been<br>restarted?<br><br><br>

[#2]

There’s a couple of issues to keep in mind here.

First, the Notifier component in Unity is logged into every suscriber’s mailbox in the Exchange site. It’s watching for any inbox activity that needs a notification or a lamp event triggered for that user. When one of the Exchange servers goes down, it will notice this and log errors in the event log until the home server is back up and happy again.

Second, if any subscriber that’s homed on an Exchange server you’ve taken off line calls into Unity to check their messages, they wont be able to get at them. In versions of Unity prior to 2.4.0 this could cause a voice port to get “burned” since the session that user was on when we tried to get at the mailbox is sitting there waiting for Exchange to return a handle to their mailbox. Unfortunately Exchange has a very long timeout for this (somewhere around 12 minutes or so) and this caused all kinds of headaches. In 2.4.0 and later we have a more aggressive “traffic cop” keeping tabs on which Exchange server is up or down in the site which allows us to prevent users from getting that far… if they try to get at their messages they’ll be routed to the failsafe conversation (“I’m sorry, I can’t talk to you right now…”) instead of getting into a huge timeout wait.

During this time, of course, messages can be left for users on that server since the MTA on the Unity box will take care of queuing them up and retrying the send every 10 minutes for 24 hours (by default… you can twiddle with this in Exchange).

As long as you’re taking your servers off line at a time when users wont be calling in to check your messages, you should be cool. The event log errors are there to help us diagnose problems when home servers are going off line when they really shouldn’t be so we can find out what happened later. In future versions of Unity we’ll have a little “heads up” display showing the up/down status of all the Exchange servers we care about (i.e. any that has a subscriber homed on them)… that’ll be handy.

You can, of course, also cycle the Unity server if you want. You don't have to but in my opinion it's not a bad idea... you're cycling your other Exchange servers, why not us? I have a little tool on my webpage just for this purpose… it shuts down Unity “nicely” (i.e. it wont hang up on folks right away if they happen to be in the system at the time) and then force a restart of the server.


Jeff Lindborg
Unity Product Architect
Active Voice
jlindborg@activevoice.com
http://members.home.net/jlindborg

[jns]

We took our exchange servers, except for unity, down for preventive maintenance reasons for about a half hour, if not longer. Upon restarting the exchange servers Unity saw them come back on line, but when I went to synch the mwi there was no response and the integration monitor showed no records to synch the mwis. Do you have any idea what happened? Is there some sort of time out and Unity loses mapi login connection? As soon as I restarted Unity all functionality returned. Any ideas?
Thanks!
Jack

[#4]

Jeff,
I have a related issue. I had a customer where one of the 2 Exchange servers is at another location, the other is co-located with Unity (Xpressions really, but figured you wanted to keep posts Unity oriented, so that you can post this. I don't mine if you edit provided event IDs so that Unity is the focus). The customer's remote Exchange server went down and Unity logged a 29003 event recognizing that the server was down. Unity began logging 29005 errors with the following data:

5/29/01,10:58:46 AM,AvWM_MC,Error,Error ,29005,N/A,Xpressions,Access denied to .
Locks: 1
Thread: 0
Locked At: 1/1/70 (GMT)

These errors continued even after the 29002 event indicating that the Exchange server was back up. Eventually, Unity services were stopped and restarted, which cleared the events. About 5 minutes later the Exchange server went down again but Unity did not log 29005 errors.

Why would Unity continue to log 29005 events after the remote Exchange server had come back up and have "locked" mailbox messages? Also, Unity could not deliver messages to subscribers on the co-located Exchange server either, I believe represented by the following event:

5/29/01,10:58:42 AM,SiemensConv_MC,Error,Error ,7011,N/A,XPRESSIONS,Subscriber [James Scott]: Unable to get subscriber's mailbox storage, [IAvDohMailUser::get_PrimaryMailbox()] returned [0x8004000c] on line 4590 of file E:\X2.5.0\Sources2.5.0-dev\UI\Function\FC_General.cpp.
5/29/01,10:58:42 AM,AvWM_MC,Error,Error ,29005,N/A,XPRESSIONS,Access denied to .

Thanks,
Dan

[#5]

Xpressions questions are fine out here...

I believe the root of the problem is that in earlier versions (i.e. 2.4.5 based systems which your Xpressions is probably at) did not handle MAPI timeouts terribly well. In some cases this would burn ports and/or prevent anyone from logging in to check their messages and the like.

There were only two MAPI threads for everyone... one for external callers who left their messages on behalf of the Unity Messaging System account and one for all subscribers to share. This had to do with threading issues in MAPI on Exchange's end of things which were fixed in SP4 for Exchange 5.5. Unity 2.4.6 switched to a model where each port had it's own thread such that if a subscriber went to check messages on a server that was down (i.e. we didn't know it was down yet) it only hangs up that one MAPI thread... everyone else can still check messages and the like.

Those errors were probably related to continuing cascading timeouts in Unity waiting for those MAPI calls to return (Microsoft had some outrageous repeating timeouts for these going for 20 minutes or more before it'll give up). Once that thread is trahed, we're pretty much in trouble. It SHOULD have terminated the thread after the time out (i.e. 20 minutes or so) and created a new thread and moved on with life, but that may not be the case. Most folks don't wait that long regardless.

In general 2.4.6 (which requires SP4) is much more resilient to external Exchange servers going up/down.


Jeff Lindborg
Unity Product Architect/Answer Monkey
Cisco Systems
lindborg@cisco.com
http://www.AnswerMonkey.net (new page for Unity support tools and scripts)

[ajones]

Jeff is reason why you get the failsafe conv because it cant talk to that specific exchange box.

[#7]

that's one reason why you can get the fail safe. If we go to try and get a handle to your mailbox and filter on it (so we can present your messages via the phone) and we fail to get the handle, we have to bail out. The subscriber conversation will bail out and pass it back to the Arbiter component with an error message indicating it couldn't execute properly and the Arbiter then routes the call to the failsafe to tell the user there was a serious problem.

There's other reasons why you can get to the failsafe (i.e. missing prompts, corrupt directory object etc...) but in this case the reason would be because of the inability to get a handle to the mailbox.


Jeff Lindborg
Unity Product Architect/Answer Monkey
Cisco Systems
lindborg@cisco.com
http://www.AnswerMonkey.net (new page for Unity support tools and scripts)