CRS Node Manager failing

[mmelbourne]

Occasionally, the CRS Node Manager will fail on our CRS cluster Master node, after which the cluster fails over to the Secondary Node.

In the Event Log, there is Event 7031:

"The Cisco CRS Node Manager service terminated unexpectedly. It has done this 1 time(s). The following corrective action will be taken in 60000 milliseconds: Restart the service."

The service does restart, but the cluster fails over to the Secondary Node.

There are no obvious clues in the MIVR/MADM/MCVD logs. All that occurs is that a new log file is created when the service restarts.

We are running CCM 4.1(3)SR1 with UCCX 4.0(5). There don't appear to be any obvious bugs associated with the CRS Node Manager and CRS 4.0(5).

[Riccardo Bua]

Hi,

most likely reason is a JVM crash, please collect the before mentioned logs and the Java ones and raise a TAC Service Request.

Thanks,

Riccardo

[mmelbourne]

OK, thanks for that. Where are the Java logs?

[Riccardo Bua]

HI again,

to capture useful info on the JVM do the following:

To turn on Verbose:gc for Engine, follow below steps

- Open regedit on each CRS node and look for the following key -

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco

Systems,Inc.\CRS\Properties\application.MIVR.properties

- On the right side, scroll down until you find this key..

com.cisco.node.native.vm_params

- Right-click on the above key and select Modify. At the end of the

current line that exists, add the following

-verbose:gc;-XX:+PrintGCTimeStamps

- Note that there should already be a ; before the -verbose:gc (the last

parameter in that list currently has a ; at the end).

- You will need to restart engine after this change. If the customer is

not willing to restart, then make the changes and they will be picked up

after the next crash. The gc info will be logged in JVM.log in the MIVR

logs dir.

Good luck with your Service Request.

Riccardo

[mmelbourne]

Does that key need to end with a semi-colon (;) after adding the additional parameters:

;-verbose:gc;-XX:+PrintGCTimeStamps;

[Riccardo Bua]

Yes indeed, last parameter is closed with a colon as written. :-))

[mmelbourne]

I thought it probably was :-) I have made the registry change in a lab here and can see the JVM.log file in the MIVR log directory.