Solved: CUCM7 CDR Agent cannot SFTP to CDR Repository

thompsonr123 · ‎09-23-2010

CUCMBE 7 single server cluster

CDR flat files are being created under /var/log/active/cm/cdr/

But CDR Agent is unable to SFTP the files to the CDR repository directory. It cannot authenticate. It keeps trying and trying, but it is not able create the SFTP connection even though it is from one folder to another on the same server.

See section from CDR agent trace file below. I have removed ip addresses and DNS names, but they are correct in the trace file.

I have restarted all the CDR services but this hasn't resolved the issue. Any help would be appreciated.

Thanks

2010-09-16 08:50:29,899 INFO [main] cdragent.cdrAgent (cdrAgent.java:171) - waking up

2010-09-16 08:50:29,899 INFO [main] cdragent.cdrAgent (cdrAgent.java:353) - handleCDRFiles -->

2010-09-16 08:50:30,086 INFO [main] cdragent.cdrAgent (cdrAgent.java:254) - At the moment, # of files ready to be transfered to Publisher is 80082

2010-09-16 08:50:30,871 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:395) - sendFile(): Try bldSendFileCmds

2010-09-16 08:50:30,872 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:399) - FtpManager - sendFile() : The Parent directory of the remote directory send the files : /var/log/active/cm/cdr_repository/

2010-09-16 08:50:30,872 DEBUG [main] sftpapi.ftpClient (ftpClient.java:783) - bldSendFileCmds(): -->

2010-09-16 08:50:30,872 DEBUG [main] sftpapi.ftpClient (ftpClient.java:796) - bldSendFileCmds(): - no need to build command file

2010-09-16 08:50:30,872 DEBUG [main] sftpapi.ftpClient (ftpClient.java:797) - local directory: /var/log/active/cm/cdr/

2010-09-16 08:50:30,872 DEBUG [main] sftpapi.ftpClient (ftpClient.java:798) - remote directory: /var/log/active/cm/cdr_repository/trans/

2010-09-16 08:50:30,872 DEBUG [main] sftpapi.ftpClient (ftpClient.java:799) - final remote directory: /var/log/active/cm/cdr_repository/tmp/

2010-09-16 08:50:30,872 DEBUG [main] sftpapi.ftpClient (ftpClient.java:801) - file: cdr_StandAloneCluster_01_200909221116_0

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:803) - bldSendFileCmds(): return <--

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:433) - sendFiles(): -->

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:444) - sendFiles(): file to send = /var/log/active/cm/cdr/cdr_StandAloneCluster_01_200909221116_0

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:445) - sendFiles(): remoteDirectory = /var/log/active/cm/cdr_repository/trans/

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:452) - sendFiles():remoteDirectoryFinal(NOT NULL)=/var/log/active/cm/cdr_repository/tmp/

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:453) - sendFiles():remoteFilePathFinal=/var/log/active/cm/cdr_repository/tmp/cdr_StandAloneCluster_01_200909221116_0

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:455) - sendFiles(): localFilePath = /var/log/active/cm/cdr/cdr_StandAloneCluster_01_200909221116_0

2010-09-16 08:50:30,873 DEBUG [main] sftpapi.ftpClient (ftpClient.java:456) - sendFiles(): remoteFilePath = /var/log/active/cm/cdr_repository/trans/cdr_StandAloneCluster_01_200909221116_0

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.ftpClient (ftpClient.java:460) - sendFiles(): uploadFile

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:100) - sendFileThroughSFTP -->

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:122) - The sftp connection object is NULL

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:123) - Connection to the remote server might not have been established

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:124) - Please check whether, remote server is in network or can be manually connected

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.ftpClient (ftpClient.java:463) - sendFiles(): After sftpconnection.sendFileThroughSFTP, result=false

2010-09-16 08:50:30,874 DEBUG [main] sftpapi.ftpClient (ftpClient.java:485) - sendFiles(): return false

2010-09-16 08:50:30,874 ERROR [main] cdragent.cdrAgent (cdrAgent.java:467) - sftp failed for file cdr_StandAloneCluster_01_200909221116_0

2010-09-16 08:50:30,875 ERROR [main] cdragent.cdrAgent (cdrAgent.java:477) - Alarm: Unable to successfully send file to repository node.

2010-09-16 08:50:30,875 INFO [main] cdragent.cdrAgent (cdrAgent.java:506) - raising alarm CDRAgentSendFileFailureContinues

2010-09-16 08:50:30,875 INFO [main] alarmutil.AlarmSender (AlarmSender.java:101) - alarmName: CDRAgentSendFileFailureContinues

2010-09-16 08:50:30,875 DEBUG [main] alarmutil.AlarmSender (AlarmSender.java:107) - paramName[0] = CDRRepositoryNodeAddress -> paramValue[0] = /*removed*/

2010-09-16 08:50:30,875 DEBUG [main] alarmutil.AlarmSender (AlarmSender.java:107) - paramName[1] = CDRAgentNodeAddress -> paramValue[1] = /*removed*/

2010-09-16 08:50:30,875 DEBUG [main] alarmutil.AlarmSender (AlarmSender.java:113) - sendAlarm ...

2010-09-16 08:50:30,875 INFO [main] cdragent.cdrAgent (cdrAgent.java:533) - <-- handleCDRFiles

2010-09-16 08:50:30,876 DEBUG [main] sftpapi.ftpClient (ftpClient.java:282) - disconnect(): <-ftpClient->

2010-09-16 08:50:30,876 DEBUG [main] sftpapi.ftpClient (ftpClient.java:286) - disconnect(): SFTP Connection -->

2010-09-16 08:50:30,876 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:421) - disconnect(): SFTP Connection -->

2010-09-16 08:50:30,876 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:185) - After failure to send files in the previous attempt. Creating new ftpClient object

2010-09-16 08:50:30,876 DEBUG [main] sftpapi.ftpClient (ftpClient.java:147) - ftpClient constructor

2010-09-16 08:50:30,876 DEBUG [main] sftpapi.ftpClient (ftpClient.java:149) - host = /*removed*/

2010-09-16 08:50:30,876 DEBUG [main] sftpapi.ftpClient (ftpClient.java:150) - user = ccmservice

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:151) - passwd = null

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:152) - localDir = null

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:153) - remoteDir = null

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:154) - secureFTP = true

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:155) - permKey = true

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:156) - Component Name : cdragent

2010-09-16 08:50:30,877 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:188) - After creating the ftpClient object

2010-09-16 08:50:30,877 DEBUG [main] sftpapi.ftpClient (ftpClient.java:178) - connect()-->

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.ftpClient (ftpClient.java:181) - connect(): connect for SFTP

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:89) - Inside the SFTPConnection Constructor

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:301) - checkConnection for Publisher using Key Authentication-->

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:302) - checkConnection: UserName : ccmservice

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:303) - checkConnection: hostname : /*removed*/

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:152) - getSshClient-->

2010-09-16 08:50:30,878 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:209) - getSshConnector-->

2010-09-16 08:50:30,880 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:164) - SSH Connection object obtained

2010-09-16 08:50:30,928 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:30,934 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:30,939 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:30,944 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:31,081 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:31,086 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:31,092 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:342) - Trying to authenticate using Private Key File : /home/ccmservice/.ssh/id_dsa

2010-09-16 08:50:31,098 ERROR [main] sftpapi.SFTPConnection (SFTPConnection.java:382) - Exception thrown while connecting to the publisher : com.maverick.ssh.SshException: Too many authentication failures for ccmservice [Unknown cause]

2010-09-16 08:50:31,098 DEBUG [main] sftpapi.SFTPConnection (SFTPConnection.java:390) - <--checkConnection for Publisher using Key Authentication

2010-09-16 08:50:31,098 DEBUG [main] sftpapi.ftpClient (ftpClient.java:193) - The status of checkConnection : false

2010-09-16 08:50:31,098 ERROR [main] sftpapi.ftpClient (ftpClient.java:220) - connect(): Failed connect to /*removed*/

2010-09-16 08:50:31,098 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:193) - After attempt to get ftpClient connection

2010-09-16 08:50:31,098 DEBUG [main] cdragent.cdrAgent (cdrAgent.java:206) - Sleeping for 60 seconds

Lakshmi Adapala · ‎09-23-2010

From logs seeing that sftp is initiating a connection with username "ccmservice". This is a known bug caused by incorrect permissions and you will need to open a TAC case as the authorized keys in the specific user's SSH directory need to be copied manually from root access.

CSCsz99841 CDR SSH user incorrect causing wrong file permissions in CUCM

View solution in original post

James Hawkins · ‎09-23-2010

Hi,

I am confused as to what you are trying to achieve.

If you are trying to transfer CDR records to a billing server then it should be a different physical machine rather than a different directory on the CUCMBE server.

thompsonr123 · ‎09-23-2010

Hi,

The problem being experienced means there is no data to view in CAR, nor is there any data to transfer to a billing server (not that we are using a billnig server, we just want to see info in CAR).

Lakshmi Adapala · ‎09-23-2010

From logs seeing that sftp is initiating a connection with username "ccmservice". This is a known bug caused by incorrect permissions and you will need to open a TAC case as the authorized keys in the specific user's SSH directory need to be copied manually from root access.

CSCsz99841 CDR SSH user incorrect causing wrong file permissions in CUCM

thompsonr123 · ‎09-23-2010

Hi,

Thanks for this info. Really appreciated!

I'll contact TAC.