LMS3.2 - Service Temporarily Unavailable

Answered Question
Feb 18th, 2010
User Badges:

Hi there


Our Windows Cisco Works Server ran out of disk space, I freed up some space and rebooted the server. After I restarted the server I now get the following message when trying to use a web browser to connect -


Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.


Eventualy I then just get "page cannot be displayed"


I have tried restarting the services using -

net stop crmdmgtd

net start crmdmgtd


Any help on how to start troubleshooting this would be gratefully recieved.


Cheers

Tim

Correct Answer by Joe Clarke about 7 years 3 months ago

You appear to have found a neat bug.  Within the UT servlet code which handles UTU, if an error occurs running the DB query, the serlvet will kill Tomcat.  If you open a TAC service request, and have your engineer contact me directly, I will provide you a patch.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Tim Davies Thu, 02/18/2010 - 08:13
User Badges:

After increasing the virtual memory and performing another server reboot I can now connect to Cisco Works using a web browser.

Cisco Works seems to function fine until I attempt a search using the "User Tracking Utility" and then the "Tomcat Servlet Engine Service" stops and then I can no longer browse to the server, I get the service temporarily unavailable message again. Cisco Works seems to function fine until I attempt to use the User Tracking Utility and then this causes the Tomcat Servlet Engine service to stop.


Any ideas?

Joe Clarke Thu, 02/18/2010 - 09:58
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Post the NMSROOT/MDC/tomcat/logs/stdout.log and stderr.log.

Joe Clarke Fri, 02/19/2010 - 07:44
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

When were these logs captured?  I don't see any problems here.  You need to reproduce the problem, then immediately capture the logs before restarting anything.

Joe Clarke Fri, 02/19/2010 - 09:55
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Post the output of the pdshow command before and after reproducing this problem.

Tim Davies Mon, 02/22/2010 - 03:15
User Badges:

Please find the requested files attached.

Over the weekend the Cisco Works backup also reported the following -

"Backup failed on 2010/02/20 at 20:46:19. REASON: Fatal error: Database engine 'rmengEng' could not be started on database 'rmengdb' in Bulk mode.."


This may be relevant?

Joe Clarke Mon, 02/22/2010 - 09:10
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Check the Windows Event Viewer for errors when the Tomcat service stops.  Also, search the file system for hs_err* files.


As for the RME problem, your RME database (or its transaction log) is corrupt.  You should start a new thread for that issue.

Tim Davies Tue, 02/23/2010 - 01:34
User Badges:

The following event is logged when the TomCat service stops -


"The Java Virtual Machine has exited with a code of 0, the service is being stopped."


There is also the following errror straight after the TomCat service starts up -


" The Apache service named  reported the following error: >>> [Mon Feb 22 10:32:20 2010] [warn] The Alias directive in C:/Program Files/CSCOpx/MDC/Apache/conf/cmf-app.conf at line 8 will probably never match because it overlaps an earlier Alias.     ."



I have attached hs_errr* files however these are date stamped 2009 so not sure if they are relevant.




Correct Answer
Joe Clarke Tue, 02/23/2010 - 18:54
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

You appear to have found a neat bug.  Within the UT servlet code which handles UTU, if an error occurs running the DB query, the serlvet will kill Tomcat.  If you open a TAC service request, and have your engineer contact me directly, I will provide you a patch.

Tim Davies Mon, 03/01/2010 - 07:59
User Badges:

In spening a bit more time looking at this I noticed that the ANIServer process wasn't running.

I restored the ANIServer.properties file to a date before the server hard disk ran out of space and this seems to have resolved the issue.


I still need to double check that everything is now working but so far so good.


Thanks for all your help.

Joe Clarke Mon, 03/01/2010 - 09:26
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

That could have definitely triggered the bug I found in UTBandServlet.  In any event, you should have received the patch, and it would be a good idea to apply it to prevent potential Tomcat crashes in the future.

Tim Davies Tue, 03/02/2010 - 05:10
User Badges:

Patch received and applied, thanks once again Joe.

Actions

This Discussion