Cups 7.x and CupC 7.x No CTI ctrl of phones-fine for 1yr, stopped 2days ago

[dmooreami]

Have a Cups clusters, (v7.x)  at two geographic locations connnected by a 60ms wan link.  All CupC 7.x functions working find for over 1 year. 

Friday, all CUPC CTI control of phones failed.  But CITGateway is working on the Subs and pubs. Attendent console uses CTI and has no issues.

All users (60+) across two different Cups servers can't control their phones. 

All other features of CupC and Cups, presence, softphone, chat, all work

All  pass the Cups desktop phone troubleshooter, reboot of the two Cups boxes has not fixed the problem.

In "server health" I can see CUPC client trying to connect but just gets a "not connected" to the primary and backup.

Ran the "troubleshooter" on one of the Cup boxes. Everything passed except got this below:

Verify whether Cisco Unified Personal Communicator (CUPC) application profiles have any assigned users   

Warn    The following application profiles currently have no users assigned:

CTI Gateway.Default_cti_tcp_profile_synced_000

CTI Gateway.Default_cti_tls_profile_synced_000

CTI Gateway.Auto Registered Devices_cti_tcp_profile_synced_000

CTI Gateway.Auto Registered Devices_cti_tls_profile_synced_000

CTI Gateway.IP Communicator Phones_cti_tcp_profile_synced_000

CTI Gateway.IP Communicator Phones_cti_tls_profile_synced_000

CTI Gateway.NYC non-SRST Phones_cti_tcp_profile_synced_000

CTI Gateway.NYC non-SRST Phones_cti_tls_profile_synced_000

CTI Gateway.Boca Raton non-SRST Phones_cti_tcp_profile_synced_000

CTI Gateway.Boca Raton non-SRST Phones_cti_tls_profile_synced_000

CTI Gateway.Woodland Hills non-SRST Phones_cti_tcp_profile_synced_000

CTI Gateway.Woodland Hills non-SRST Phones_cti_tls_profile_synced_000

CTI Gateway.NYC SRST Phones_cti_tcp_profile_synced_000

CTI Gateway.NYC SRST Phones_cti_tls_profile_synced_000

CTI Gateway.Boca Raton SRST Phones_cti_tcp_profile_synced_000

CTI Gateway.Boca Raton SRST Phones_cti_tls_profile_synced_000

CTI Gateway.Woodland Hills SRST Phones_cti_tcp_profile_synced_000

CTI Gateway.Woodland Hills SRST Phones_cti_tls_profile_synced_000

CTI Gateway.NYC System Devices_cti_tcp_profile_synced_000

CTI Gateway.NYC System Devices_cti_tls_profile_synced_000

CTI Gateway.Boca Raton System Devices_cti_tcp_profile_synced_000

CTI Gateway.Boca Raton System Devices_cti_tls_profile_synced_000

CTI Gateway.Woodland Hills System Devices_cti_tcp_profile_synced_000

CTI Gateway.Woodland Hills System Devices_cti_tls_profile_synced_000

CTI Gateway.System Unity Connection_cti_tcp_profile_synced_000

CTI Gateway.System Unity Connection_cti_tls_profile_synced_000

What does this mean we have no users assigned. Nothing has changed on either Presence server, nor have there been any major changes other than phone MAC's to the Pub.

I have all these profiles created on both nodes of the Cups cluster. I did not install this, a consultant did about 2yrs ago in April. Guess its about time for stuff to start breaking, right.

[Aaron Harrison]

Hi

When you set up a CUPS cluster it automatically makes a 'CTI Profile' based on the configuration of your CUCM device pools.

As users sync in to CUPS (i.e. the 'license capability' options in CCMAdmin are checked) they are assigned whichever profile is marked as 'default'. Usually I would create a device pool and make it default, so the 'synced' profiles listed in your post end up with no users in - that's normal.

If you have checked 'server health' and you see servers that you know run CTI listed, then the CUPS config is probably OK. As you say it says 'not connected' I would check:

1) That the TCP ports are actually reachable (i.e someone else hasn't stuck in an ACL or firewall rule that prevents access). Try 'telnet ctimanagerip 2748' and see if you get a black screen (OK) , or an error.

2) Check that the user you log in with has CTI Enabled, and phone/EM profiles associated, and a correct 'primary' line assigned.

3) Perhaps try restarting the CTIManager service on CUCM

Regards

Aaron

Please rate helpful posts..

[dmooreami]

1-tcp ports reachable

2-All users CTI rights enabled. Remember it worked great for 60+ people for almost 2yrs, then friday... nobodby could get CTI control of phone to work

3-Nope, CTISever on Pub and Subs working great. The attendent console software uses CTI to control phone, no problems with it.  Also did a restart of all CTImanagers on pub and subs, plus rebooted the sub that is the primary CTIT Server for most CupC users.

[dmooreami]

I have pulled the logs off my Sub that CUPC access for the CTIGateway and get the below error:

Nov 20 15:15:25, BOCACM03, Error, Cisco CTIManager, : 17: Nov 20 20:15:25.302 UTC : %CCM_CTI-CTI-3-kCtiProviderOpenFailure: CTI application failed to open provider. CTIconnectionId:23 Login User Id:dmoore Reason code.:-1932787595 IPAddress:10.122.17.34 IPV6Address: Cluster ID:StandAloneCluster Node ID:BOCACM03, 2990

Nov 20 15:31:28, BOCACM03, Error, Cisco CTIManager, : 18: Nov 20 20:31:28.592 UTC : %CCM_CTI-CTI-3-kCtiProviderOpenFailure: CTI application failed to open provider. CTIconnectionId:24 Login User Id:dmoore Reason code.:-1932787595 IPAddress:10.122.17.34 IPV6Address: Cluster ID:StandAloneCluster Node ID:BOCACM03, 2991

-----------------

[dmooreami]

More fun.

Log from CupC indicates its an LDAP issue with CUCM, but all LDAP is running fine and we are using the global Cat port to pull LDAP updates.

2011-11-20 15:15:25,401 [0x00000538] WARN  LCCallControl -                      
(CCMSG_QBE_PROVIDEROPENFAILED) ProviderOpen failed: Directory login failed -    
timeout

[Aaron Harrison · Accepted solution]

Hi

Usually this is (as you suggest) a matter of using the GC. When you say 'pulling updates' do you mean that the AD directory/auth settings in CUCM are set to 3268?

If so try doing a 'utils network packet capture' on the CUCM CLI to capture the exchange between CCM and LDAP so you can see at what point the delay occurs. It can just be something simple like a reference being handed out to a DC that has been removed and not tidied up properly but I've found it's as easy to look at the packet traces as the logs..

Aaron

[dmooreami]

That was it. We didn't know that the sub's had DC Ldap server.  Our network admin decomissioned a DC that was in only ONE of the subs.  Fixed that server name for the ldap query and all was good.

[Aaron Harrison]

Smart work - good to hear you found the problem :-)