How to Troubleshoot Cluster Problem

Unanswered Question
Feb 2nd, 2010

I have problem between two callmanager publisher and subscriber.

If I tried to login to Cisco Unified Serviceability and I chose the subscriber from the drop list, I receive the following:

In the same time all phones registered to subscriber, and I am able to ping and ssh the server, but I am not able to browse it using the IP.

Once I connected to the server using SSH I got:

java.io.FileNotFoundException: /var/log/active/platform/log/cli.bin (Read-only file system)

        at java.io.RandomAccessFile.open(Native Method)

        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)

        at com.cisco.iptplatform.fappend.ciscoRollingFileAppender.restoreIndex(ciscoRollingFileAppender.java:104)

        at com.cisco.iptplatform.fappend.ciscoRollingFileAppender.setFile(ciscoRollingFileAppender.java:43)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:585)

        at org.apache.log4j.config.PropertySetter.setProperty(PropertySetter.java:196)

        at org.apache.log4j.config.PropertySetter.setProperty(PropertySetter.java:155)

        at org.apache.log4j.xml.DOMConfigurator.setParameter(DOMConfigurator.java:530)

        at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:182)

        at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:140)

        at org.apache.log4j.xml.DOMConfigurator.findAppenderByReference(DOMConfigurator.java:153)

        at org.apache.log4j.xml.DOMConfigurator.parseChildrenOfLoggerElement(DOMConfigurator.java:415)

        at org.apache.log4j.xml.DOMConfigurator.parseRoot(DOMConfigurator.java:384)

        at org.apache.log4j.xml.DOMConfigurator.parse(DOMConfigurator.java:783)

        at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:666)

        at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:616)

        at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:584)

        at org.apache.log4j.xml.DOMConfigurator.configure(DOMConfigurator.java:687)

        at sdMain.main(sdMain.java:511)

java.lang.NullPointerException

        at com.cisco.iptplatform.fappend.ciscoRollingFileAppender.updateIndex(ciscoRollingFileAppender.java:117)

        at com.cisco.iptplatform.fappend.ciscoRollingFileAppender.nextFileName(ciscoRollingFileAppender.java:92)

        at com.cisco.iptplatform.fappend.ciscoRollingFileAppender.append(ciscoRollingFileAppender.java:74)

        at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:221)

        at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:57)

        at org.apache.log4j.Category.callAppenders(Category.java:187)

        at org.apache.log4j.Category.forcedLog(Category.java:372)

        at org.apache.log4j.Category.info(Category.java:674)

        at sdMain.main(sdMain.java:525)

log4j:ERROR No output stream or file set for the appender named [CLI_LOG].

Then I issued: utils service list

System SSH [STARTED]

Cluster Manager [STARTED]

Service Manager is running

Getting list of all services

>> Return code = 0

A Cisco DB[STARTED]

A Cisco DB Replicator[STARTED]

Cisco AMC Service[STARTED]

Cisco AXL Web Service[STOPPED]  Commanded Out of Service

Cisco CAR Scheduler[STARTED]

Cisco CDP[STARTED]

Cisco CDP Agent[STARTED]

Cisco CDR Agent[STARTED]

Cisco CDR Repository Manager[STOPPED]  Component is not running

Cisco CTIManager[STARTED]

Cisco CTL Provider[STARTED]

Cisco CallManager[STARTED]

Cisco CallManager Admin[STOPPED]  Commanded Out of Service

Cisco CallManager Attendant Console Server[STOPPED]  Component is not running

Cisco CallManager Cisco IP Phone Services[STOPPED]  Commanded Out of Service

Cisco CallManager Personal Directory[STOPPED]  Commanded Out of Service

Cisco CallManager SNMP Service[STARTED]

Cisco CallManager Serviceability[STOPPED]  Commanded Out of Service

Cisco CallManager Serviceability RTMT[STOPPED]  Commanded Out of Service

Cisco Certificate Expiry Monitor[STARTED]

Cisco DHCP Monitor Service[STARTED]

Cisco DRF Local[STARTED]

Cisco DRF Master[STARTED]

Cisco Database Layer Monitor[STARTED]

Cisco Dialed Number Analyzer[STOPPED]  Commanded Out of Service

Cisco Extended Functions[STARTED]

Cisco Extension Mobility[STOPPED]  Commanded Out of Service

Cisco Extension Mobility Application[STOPPED]  Commanded Out of Service

Cisco IP Manager Assistant[STOPPED]  Commanded Out of Service

Cisco IP Voice Media Streaming App[STARTED]

Cisco License Manager[STARTED]

Cisco Log Partition Monitoring Tool[STARTED]

Cisco Messaging Interface[STOPPED]  Component is not running

Cisco RIS Data Collector[STARTED]

Cisco RTMT Reporter Servlet[STOPPED]  Commanded Out of Service

Cisco Serviceability Reporter[STARTED]

Cisco Syslog Agent[STARTED]

Cisco Tftp[STARTED]

Cisco Tomcat[STOPPED]  Component is not running

Cisco Tomcat Stats Servlet[STOPPED]  Commanded Out of Service

Cisco Trace Collection Service[STARTED]

Cisco Trace Collection Servlet[STOPPED]  Commanded Out of Service

Cisco UXL Web Service[STOPPED]  Commanded Out of Service

Cisco WebDialer Web Service[STOPPED]  Commanded Out of Service

Host Resources Agent[STARTED]

MIB2 Agent[STARTED]

Native Agent Adapter[STARTED]

SNMP Master Agent[STOPPED]  Component is not running

SOAP -Log Collection APIs[STOPPED]  Commanded Out of Service

SOAP -Performance Monitoring APIs[STOPPED]  Commanded Out of Service

SOAP -Real-Time Service APIs[STOPPED]  Commanded Out of Service

System Application Agent[STARTED]

Cisco Bulk Provisioning Service[STOPPED]  Service Not Activated

Cisco CAR Web Service[STOPPED]  Service Not Activated

Cisco Certificate Authority Proxy Function[STOPPED]  Service Not Activated

Cisco DirSync[STOPPED]  Service Not Activated

Cisco SOAP - CDRonDemand Service[STOPPED]  Service Not Activated

Cisco TAPS Service[STOPPED]  Service Not Activated

Cisco Unified Mobile Voice Access Service[STOPPED]  Service Not Activated

As You can see most services are down, like Tomcat and DirSync and other.

I do not need to restart the server from the beginning, I need to know if there is any way to troubleshoot and to know the reasons.

I tried to restart the services but it is fail.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
r.verstraten Mon, 04/19/2010 - 09:08

Hello,

Did you find out what the issue was? I've got the same issue at a CUCM cluster.

Thanks a lot,

Actions

This Discussion