02-06-2014 01:40 PM - edited 03-16-2019 09:38 PM
The Host Resources Agent service on our 8.6.1 CER server has stopped, and thus is no longer SNMP pollable.
Whenever I try to start the service, it says starting, never starts and stays stopped.
I rebooted the server and the service is still down.
Any ideas?
02-06-2014 01:49 PM
Have you checked the event viewer logs under Serviceability?
HTH
Regards,
Yosh
02-06-2014 02:02 PM
Here is what I am seeing in there:
database
2014-02-06 02:21:44,734 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 04:21:59,100 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 06:22:18,522 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 08:22:30,544 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 09:52:40,244 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 11:52:53,872 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 13:53:08,475 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 14:23:12,969 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 15:53:23,381 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 16:47:56,014 | WARN | CER_GROUP | Disconnected from Cisco Emergency Responder : Default_DE / /10.200.0.13
2014-02-06 16:47:56,015 | INFO | CER_CALLENGINE | Cisco ER Exiting : Graceful shutdown
2014-02-06 16:56:49,317 | INFO | CER_GROUP | Active Cisco Emergency Responder set to : Publisher / CER/10.200.0.13
2014-02-06 16:57:22,836 | INFO | CER_GROUP | Connection established with Cisco Emergency Responder : Default_DE / /10.200.0.13
2014-02-06 16:57:23,001 | INFO | CER_GROUP | Connection established with Cisco Emergency Responder : Aggregator / /10.200.0.13
2014-02-06 16:57:24,094 | WARN | CER_TELEPHONY | 911Route Point received OUT_SERVICE event from Provider 10.200.0.10
2014-02-06 16:58:30,259 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 16:59:39,265 | INFO | CER_DATABASE | Failed to get the fully qualified host name. DNS might not be enabled. Putting Ipaddress in Cluster database
2014-02-06 17:00:33,156 | WARN | CER_TELEPHONY | 911Route Point received OUT_SERVICE event from Provider 10.200.0.10
02-06-2014 02:08 PM
What about under System Logs>Platform Logs>Syslog>CiscoSyslog?
HTH
Regards,
Yosh
02-06-2014 02:10 PM
Here is what I am seeing:
Feb 6 16:34:24 CER local7 2 : 18: CER: Feb 06 2014 21:34:24.937 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 16:34:24 CER local7 2 : 19: CER: Feb 06 2014 21:34:24.937 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 16:35:39 CER local7 2 : 20: CER: Feb 06 2014 21:35:39.93 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 16:35:39 CER local7 2 : 21: CER: Feb 06 2014 21:35:39.93 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 16:36:55 CER local7 2 : 22: CER: Feb 06 2014 21:36:55.229 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 16:36:55 CER local7 2 : 23: CER: Feb 06 2014 21:36:55.229 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 16:37:14 CER local7 2 : 24: CER: Feb 06 2014 21:37:14.322 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 16:37:14 CER local7 1 : 25: CER: Feb 06 2014 21:37:14.323 UTC : %UC_SERVICEMANAGER-1-ServiceExceededMaxRestarts: %[ServiceName=Host Resources Agent][RestartCount=3][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service failed to start and exceeded maximum allowed restarts.
Feb 6 16:54:56 CER local7 3 : Alarm Interface bind failed for catalog: 6 GenericAlarmCatalog
Feb 6 16:54:56 CER local7 3 : Unknown Alarm Catalog 32 GenericAlarmCatalog
Feb 6 16:54:59 CER local7 3 : Alarm Interface bind failed for catalog: 6 GenericAlarmCatalog
Feb 6 16:54:59 CER local7 3 : Unknown Alarm Catalog 32 GenericAlarmCatalog
Feb 6 16:56:10 CER local7 3 : Alarm Interface bind failed for catalog: 6 CDPAlarmCatalog
Feb 6 16:56:10 CER local7 3 : Unknown Alarm Catalog 32 CDPAlarmCatalog
Feb 6 16:56:27 CER local7 3 : Alarm Interface bind failed for catalog: 6 JavaApplications
Feb 6 16:56:31 CER local7 3 : Unknown Alarm Catalog 32 JavaApplications
Feb 6 16:57:23 CER local7 2 : 0: CER: Feb 06 2014 21:57:23.590 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 16:57:23 CER local7 2 : 1: CER: Feb 06 2014 21:57:23.590 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 16:58:44 CER local7 2 : 2: CER: Feb 06 2014 21:58:44.809 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 16:58:44 CER local7 2 : 3: CER: Feb 06 2014 21:58:44.809 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 17:00:04 CER local7 2 : 4: CER: Feb 06 2014 22:00:04.918 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 17:00:04 CER local7 2 : 5: CER: Feb 06 2014 22:00:04.919 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 17:01:25 CER local7 2 : 6: CER: Feb 06 2014 22:01:25.702 UTC : %UC_SERVICEMANAGER-2-ServiceStartFailed: %[ServiceName=Host Resources Agent][Reason=Unsuccessful completion][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Failed to start service.
Feb 6 17:01:25 CER local7 2 : 7: CER: Feb 06 2014 22:01:25.703 UTC : %UC_SERVICEMANAGER-2-ServiceFailed: %[ServiceName=Host Resources Agent][Reason=Service stopped abruptly][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service terminated.
Feb 6 17:01:25 CER local7 1 : 8: CER: Feb 06 2014 22:01:25.704 UTC : %UC_SERVICEMANAGER-1-ServiceExceededMaxRestarts: %[ServiceName=Host Resources Agent][RestartCount=3][AppID=Cisco Service Manager][ClusterID=][NodeID=CER]: Service failed to start and exceeded maximum allowed restarts.
02-06-2014 02:21 PM
Have you checked your current "Processes" to make sure that you don't have a service that is pegging your CPU. I would verify the "Processes" first and then if nothing seems excessive I would stop all the services and try to just start the "Host Resources Agent" service.
You could also check the "boot log" it might give a pointer in the right direction.
HTH
Regards,
Yosh
02-06-2014 02:47 PM
Tomcat is the highest consumer, at 1.1%
I am going to try bringing down the services now. Thank you for all your suggestions Yosh.
02-06-2014 02:49 PM
You're welcome, I know CER is a tricky one to troubleshoot, since its so different from all the rest.
Regards,
Yosh
02-06-2014 03:09 PM
Yosh, shut down all the services (all but some that wouldn't not allow me to bring them down for obvious reasons) and tried to fire up the host resource server but it just looped like it always does and then fails to start after maximum tries.
Guess it's time for a TAC case?
02-07-2014 05:30 AM
Sounds like it. I'm curious to know what the issue is, keep me posted on solution.
Regards,
Yosh
02-07-2014 11:02 AM
Yosh - here is the problem:
02-07-2014 11:59 AM
I appreciate the follow up. So this has solved your issue?
Regards,
Yosh
02-12-2014 09:41 AM
Sorry just saw this - yes the COP file provided by TAC solved the issue.
02-12-2014 09:51 AM
Good to know.
Thanks,
Yosh
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide