HUM1.1 - UPMProcess shutdown by itself

[glenn.ong]

UPMProcess seems to shut itself down randomly (typically once every 4->5 days) on our new CiscoWorks LMS 3.1 (running CS3.2).

I can't see any error at all in upm_process.log. Under CS process manager, it says 'Administrator has shut down this server' but none of the admin has manually shutdown the service.

Any feedback is welcomed. Thanks.

[Joe Clarke]

On what platform is LMS running?

[glenn.ong]

Hi JC,

it's running on a physical x86 server with Win2k3 Std Edition SP2.

[Joe Clarke]

Post the UPMProcess.log and the upm_process.log along with the output of the pdshow command when UPMProcess is down.

[glenn.ong]

Hi JC,

Was waiting for the issue to reappear by itself after restarting the process and it did. Logs are attached as required.

Thanks in advance.

[Joe Clarke]

It looks like you've hit a JVM bug most likely caused by memory starvation. How much memory is on this server? How many pollers do you have defined in HUM?

[glenn.ong]

Interesting. It's a physical server with 3.25Gb RAM. Only 4 pollers are configured but "No. of Objects Monitored" is 809.

If this this indeed an issue - would appreciate you can point me to any literature that documents the relationship b/w spec vs pollers configured. Would like to use it as a justification for potential server upgrade and/or configuration optimisation.

[Joe Clarke]

This is fine assuming you have enough free memory on the server (see http://www.cisco.com/en/US/docs/net_mgmt/ciscoworks_health_and_utilization_monitor/1.1/user/guide/HUMtrfaq.html for scaling details). When the crash occurs, how much memory is free?

[glenn.ong]

According to our server guys, the total memory utilisation is always constant (at approx. 84%) and also looking at the page file usage, available bytes etc, they observed no evidence of memory thrashing during the timeframe UPMProcess stopped itself.

[Joe Clarke]

Then the problem is most likely starvation within the UPMProcess JVM. It could also be another bug. Either way, I recommend you open a TAC service request. They can provide you instructions on how to increase the JVM heap size to see if that helps. They can also pass the Java crash details onto development.

[panditupendra]

Process= IPMProcess

State   = Administrator has shut down this services.

PID   =0

RC=  0

Signo=0

Core = not applicable 

info= server started by admin request

we are using win 2k8 R2 server with cisco prime software 4.1