Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

ICM router process restarting intermittently

Hi Guys,

My cuurent setup :

UCCE 8.5.4

Side A:

Rogger A, PGA (AgentPG),VRU PGA, AW/HDS A, pub, sub1,sub2,CVPA,CVPB

Side B:         

Rogger B, PGB (AgentPG),VRU PGB, AW/HDS B, sub3,sub4,CVPC,CVPD

I am getting error in the Rogger B as below,

1. Connectivity with duplexed partner has been lost due to a failure of the private network, or duplexed partner is out of service.

2. MDS is out of service.

3. MDS has reported failure to the router that it is out of service.

4. Message Delivery Service (MDS) feed from the Router to the Logger has failed.

5. Central Controller service is unavailable.

6. Requesting MDS termination due to error.

7. Application Gateway has been taken out of service. Application Gateway ID - 5000

8. Client rtr stopping due to error.

9. Client hlgr stopping due to error.

10. Client clgr stopping due to error.

11. Synchronizer is unable to establish connection to peer.

12. Process rtr on ICM\suly\RouterB has detected a failure. Node Manager is restarting the process.

13. Process rtr on ICM\suly\RouterB is down after running for 366 seconds. It will restart after delaying 1 second for related operations to complete.

14. ICM\suly\LoggerB node process hlgr exited cleanly and requested that it be restarted by the Node Manager.

but when I check with network team, they have confirmed that there is no issue at network side.

Check the router logs. mds logs. ccagent logs: all these couldnt give other than stopping due to error. what might be causing it to close all these process in the rogger B.

could you guys please advice me to troubleshoot this issue.

Regards,

Shalid K.C

  • Contact Center
Everyone's tags (6)
11 REPLIES

ICM router process restarting intermittently

Does the rtr service ever go active?  If so, have you reviewed the startup log to ensure you're not seeing these same errors?  If you're not seeing any errors most of the time, then it seems that you have a network issue or perhaps a route change and the network is fixing it, but not fast enough for the router's liking.  My thought would be to not play the cat and mouse game with the network team.  Have them put a network sniffer on your segment and wait for the next time this happens.  This will be the fasted and easiest way to solve this asap.

david

ICM router process restarting intermittently

Kindly check the speed of the network cards.

Ashfaque

Cisco Employee

ICM router process restarting intermittently

Hello Shalid,

As david suggested Network sniffer would really help in this scenario.

Additionally, there are few Best practices need to be followed based on the UCCE Windows environment. I would suggest you to crossverify that as well.


http://www.cisco.com/en/US/products/sw/custcosw/ps1001/products_tech_note09186a00808160f4.shtml

http://docwiki.cisco.com/wiki/Contact_Center_Networking:_Offload,_Receive_Side_Scaling_and_Chimney

Regards,

Senthil

ICM router process restarting intermittently

Thank you guys..

Let me check as you suggested and keep you posted....

Regards,

Shalid K.C

ICM router process restarting intermittently

Hi Senthi and others,

one more thing which I noticed in this is below

Rogger, Agent PG and VRU pg of side B is installed in a single VM ware machine,

today i noticed that all three showing the similar kind of behavior, restarting its essencial process intermittently.

so i suspect there might be some issue with the system itself

but due to my insufficient knowldge in vmware machine i dont know what to check here..

could you guys can help me to suggest what check needs to bedone to isolate the issue..

Regards,

Shalid K.C

Cisco Employee

ICM router process restarting intermittently

I think you have the deployment in the UCS Servers which has 3 VM's for each component(Rogger/Agent PG/VRU PG)

Do you use B Series or C series Servers ? I would suggest you check the Network Recommendations for UCCE on UCS servers

http://docwiki.cisco.com/wiki/UCS_Network_Configuration_for_UCCE#UCCE_on_UCS_C-Series_Network_Configuration

http://docwiki.cisco.com/wiki/UCS_Network_Configuration_for_UCCE#UCCE_on_UCS_B-Series_Network_Configuration

Do you have Windows 2008 or 2003 OS ? Can you check the given recommendation on the below given NIC

http://docwiki.cisco.com/wiki/Contact_Center_Networking:_Offload,_Receive_Side_Scaling_and_Chimney

Regards,

Senthil

Rate if it helps

ICM router process restarting intermittently

Hi Guys,

I have checked the offload setting and please find below,

C:\Users\icmadmin>netsh int ip sh offload

Interface 1: Loopback Pseudo-Interface 1

udp transmit checksum supported.

tcp transmit checksum supported.

udp receive checksum supported.

tcp receive checksum supported.

Interface 11: Public

ipv4 transmit checksum supported.

udp transmit checksum supported.

tcp transmit checksum supported.

tcp large send offload supported.

ipv4 receive checksum supported.

udp receive checksum supported.

tcp receive checksum supported.

Interface 13: Private

ipv4 transmit checksum supported.

udp transmit checksum supported.

tcp transmit checksum supported.

tcp large send offload supported.

ipv4 receive checksum supported.

udp receive checksum supported.

tcp receive checksum supported.

C:\Users\icmadmin>netsh int sh offload

C:\Users\icmadmin>netsh int tcp show global

Querying active state...

TCP Global Parameters

----------------------------------------------

Receive-Side Scaling State          : enabled

Chimney Offload State               : automatic

NetDMA State                        : enabled

Direct Cache Acess (DCA)            : disabled

Receive Window Auto-Tuning Level    : normal

Add-On Congestion Control Provider  : ctcp

ECN Capability                      : disabled

RFC 1323 Timestamps                 : disabled

and check the offload for both private and public and it is enabled as RX and TX enabled.

So i am going to disable these offload as per the docmnet.

Please let me know if you guys have any suggestion on it.

Regards,

Shalid K.C

New Member

Re: ICM router process restarting intermittently

Hi Shalid,

This is happening due to only private link flaps, your network team/WAN team should look to this issue.

Set up wireshark on Private IPs and run mds logs or enable performance monitor on rogger and PGs.

Ensure when u set all this your network is not carrying too much of traffic or it would heavly impact the production enviornment.

thanks

ICM router process restarting intermittently

Hi Bala,

I havent perform the above step mentioned.. i will working with network team to get this done.

@ Senthil:,

done the complete configuration as mentioned in the doc, but no luck . still the PGB and RoggerB is critical process getting restarted.

Regards,

Shalid K.C

2424
Views
12
Helpful
11
Replies
This widget could not be displayed.