Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Community Member

LMS 3.2 - ANIServer Running with busy flag set

Have LMS 3.2 running on a Windows 2003 Server Enterprise box. I'm having issues with Campus Manager where the ANIServer has either failed to start or more recently is now running with the busy flag set:

ANIServer Running with busy flag set 7408 0 0 7/20/2009 2:05:17 PM Not applicable

Not sure what's the next step to take. I have tried rebooting which didn't help and have also stopped and started the Daemon Manager.

The last stop/start has the ANIServer running with the busy flag set, but I still can't open up the Home page of Campus Manager, run user tracking reports, or open up Toplogy Services. Below is the error message I get:

Cannot connect to ANIServer:...

1 ACCEPTED SOLUTION

Accepted Solutions
Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

Then I'm betting the problem is a corrupt database. The database might be hitting a bad page, and dying. Can you try reinitializing the ANI database, and see if the engine stays running? The command is:

NMSROOT/bin/perl NMSROOT/bin/dbRestoreOrig.pl dsn=ani dmprefix=ANI

18 REPLIES
Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

If you've stopped and started ANIServer by itself, restart Daemon Manager, wait 30 minutes, reproduce the problem, then port the ani.log and ANIServer.log.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

Hmm. Was just about to do that when I noticed that there was a Data Execution Prevention message from the Windows 2003 Server which had shutdown the SQL Anywhere Network Server service. Never had this happen before, but maybe this was related to an OS update I did this morning.

Anyway, I've added it to the exclude list. I'll give a heads up if this worked or not.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

Here's a screenshot of where I set that for reference.

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

Actually, this is not sufficient. We typically recommend you set DEP for essential Windows programs and services only, then reboot.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

Thanks. Wasn't sure on that. I've set it back. I'll post up those logs in a bit.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

I have re-produced the problem. Attached are the ani.log and ANIServer.log files.

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

There are no errors here. How many jobs do you have on the system under Common Services > Server > Admin > Job Browser?

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

21 active jobs (that are to be scheduled to run).

2836 job records in total (inclusive of active jobs).

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

I have rebooted the server again and I can get into Topology Services now and open the Home page.

I am getting the following error in trying to run a User Tracking Report:

Application error: JZ006: Caught IOException: java.net.ConnectException: Connection refused: connect.

Also, although I can click Start Data Collection in the Campus Manager Home page, I don't see it as running. However, when I open Topology Services, I can see the data collection is running (as opposed to being idle).

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

This error indicates that the ANIDbEngine is not running. Post the new pdshow command output as well as the new ani.log and Cmapps.log.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

Thanks. See attached.

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

Something seems to be horribly wrong with your installation. Post the output of netstat -a -n -o -b from this server.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

See attached.

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

Okay, the ANIDbEngine has crashed silently. If DEP is still killing it, then something is wrong with your DEP settings. Make sure you have the radio button, "Turn on DEP for essential Windows programs and services only", and that you have rebooted after setting that radio button. Also, if you have on-access anti-virus scanning running, make sure it is disabled for NMSROOT.

If DEP is not killing this process, then something else is. Check the Windows Event Viewer and the console of the server for clues.

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

Don't know what's killing that process at startup. I've even disabled DEP as a test and that didn't change things.

So far, the only way I've found to restore Campus Manager to working service is to go and manually start the CiscoWorks ANI database engine after a reboot.

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

Then I'm betting the problem is a corrupt database. The database might be hitting a bad page, and dying. Can you try reinitializing the ANI database, and see if the engine stays running? The command is:

NMSROOT/bin/perl NMSROOT/bin/dbRestoreOrig.pl dsn=ani dmprefix=ANI

Community Member

Re: LMS 3.2 - ANIServer Running with busy flag set

Thanks Joe!!!

So far, after a reboot the ANI Service stays up and running. I've completed a data collection and user tracking acquisition and I'm now able to run user tracking reports.

Could I have been hitting bug CSCsb52873?

Cisco Employee

Re: LMS 3.2 - ANIServer Running with busy flag set

No, this is most likely not caused by a bug, but just plain old corruption. Typically problems like this are caused by power failures, disks filling up, AV scanners, file system level backups that do not understand how to backup running databases, etc. In most cases of corruption, the database engine outright refuses to start. In some cases, the database starts, and runs just fine...until a certain page (e.g. table or row) is accessed. At that point, the corruption is found, and the engine throws an assertion, GPF, or segmentation fault, then dies.

What made this scenario particularly interesting is that Daemon Manager didn't catch that the engine had died. It wasn't until I saw that there wasn't any process listening on the required TCP port that I knew it was dead.

It's good the reinit worked. Make sure you take regular LMS backups to give yourself something on which to fall back in the future in case other problems occur.

836
Views
0
Helpful
18
Replies
CreatePlease to create content