Syslog Collector failure with third party SSL certificate

[UWE STEINHAU]

Hello,

We recently replaced our self-signed SSL certificates with certificates provided by our agency. After the change subscription attempts to the collector in [RME>Tools>Syslog>Syslog Collector Status] failed: SCLA0126: Could not subscribe to the Collector.

I believe the problem originates with the way the CSRs are handeled. An identification number rather than the actual FQDN must be provided in the common name field and this number is expected by the CA. A chain was built with multiple government CAs, and warnings received that the chain does not end in a trusted CA. My hands are bound by this policy - is there a way to make this work or any suggested workaround? Tried a DNS CNAME with the id number. No joy. I haven't tried renaming the host to the id number but I might if you think it might work and then will just cname the current hostname. We are running Solaris 10 systems. Here is the error from AnalyzerDebug.log:

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:150)
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:117)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1584)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:866)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1030)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:678)
        at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
        at java.io.ObjectInputStream$PeekInputStream.read(ObjectInputStream.java:2213)
        at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2226)
        at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2694)
        at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:761)
        at java.io.ObjectInputStream.<init>(ObjectInputStream.java:277)
        at com.cisco.nm.rmeng.fcss.common.FcssSyslogCollector.<init>(FcssSyslogCollector.java:95)
        at com.cisco.nm.rmeng.sa.SyslogAnalyzerEngine.notifySubscribers(SyslogAnalyzerEngine.java:975)
        at com.cisco.nm.rmeng.sa.SyslogAnalyzerEngine.start(SyslogAnalyzerEngine.java:1031)
        at com.cisco.nm.rmeng.sa.SyslogAnalyzerService.main(SyslogAnalyzerService.java:55)

Thanks....!!!

= Uwe =

[Joe Clarke]

Are you seeing any other errors relating to certificates?  Are you using Apache in SSL mode?  If so, is it working?  This problem may not be certificate-related.  If you have lsof installed, post the output of:

lsof -i :3333
lsof -i :4444

[UWE STEINHAU]

Thanks for the fast reply. Web access is working on port 443. The only problem that crept up is the syslog collector issue. Both collector and analyzer processes are running. I don't have lsof on the boxes but I'll try to get a version and install it today.

[UWE STEINHAU]

lsof -i :3333 and :4444 return nothing on either host.

[Joe Clarke]

Post the SyslogCollector.log.

[UWE STEINHAU]

The subscriber list is empty because we could not add the subscription after the swapping the certs. Sorry, was asked to obscure the host names - it shows host name only not FQDN.

SyslogCollector - [Thread: main] INFO , 14 Feb 2010 10:37:33,198, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 10:37:33,201, System Initialized.
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 10:37:36,694, Service started...
SyslogCollector - [Thread: Thread-9] WARN , 14 Feb 2010 10:42:04,383, Unable to add monitor for
SyslogCollector - [Thread: SyslogObjectForwarder] ERROR, 14 Feb 2010 11:07:42,369, Could not send syslogs, removing the subscriber...Connection refused
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 11:23:02,499, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 11:23:02,501, System Initialized.
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 11:23:02,850, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 11:23:06,047, Service started...

SyslogCollector - [Thread: main] INFO , 14 Feb 2010 23:59:33,732, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 23:59:33,735, System Initialized.
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 23:59:34,148, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 14 Feb 2010 23:59:37,352, Service started...
SyslogCollector - [Thread: main] INFO , 15 Feb 2010 23:59:34,112, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 15 Feb 2010 23:59:34,115, System Initialized.
SyslogCollector - [Thread: main] INFO , 15 Feb 2010 23:59:34,565, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 15 Feb 2010 23:59:38,168, Service started...
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 11:57:43,806, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 11:57:43,816, System Initialized.
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 11:57:44,220, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 11:57:47,493, Service started...
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 14:12:00,424, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 14:12:00,427, System Initialized.
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 14:12:00,781, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 14:12:04,007, Service started...
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 23:59:33,851, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 23:59:33,854, System Initialized.
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 23:59:34,303, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 16 Feb 2010 23:59:37,834, Service started...
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 16:42:51,156, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 16:42:51,166, System Initialized.
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 16:42:51,516, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 16:42:54,734, Service started...
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 23:59:33,673, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 23:59:33,676, System Initialized.
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 23:59:34,130, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 17 Feb 2010 23:59:37,759, Service started...
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:55:42,526, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:55:42,533, System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:55:42,886, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:55:46,111, Service started...
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:34,144, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:34,147, System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:34,604, Subscriber list is empty!
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:38,116, Service started...

Our secondary host shows a subscriber, however no syslog packets are seen. Also, this subscriber can not be unsubscribed (deleted).

SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:09:19,098, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:09:19,101, System Initialized.
SyslogCollector - [Thread: main] WARN , 18 Feb 2010 16:09:22,723, Unable to resurrect connection to a subscriber.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:09:22,770, Service started...
SyslogCollector - [Thread: Thread-11] WARN , 18 Feb 2010 16:14:07,828, Unable to add monitor for
SyslogCollector - [Thread: Thread-13] WARN , 18 Feb 2010 16:14:08,008, Unable to add monitor for
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:32:29,557, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:32:29,560, System Initialized.
SyslogCollector - [Thread: main] WARN , 18 Feb 2010 16:32:33,205, Unable to resurrect connection to a subscriber.
SyslogCollector - [Thread: main] WARN , 18 Feb 2010 16:32:33,263, Unable to resurrect connection to a subscriber.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 16:32:33,277, Service started...
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:33,728, Logging System Initialized.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:33,733, System Initialized.
SyslogCollector - [Thread: main] WARN , 18 Feb 2010 23:59:37,786, Unable to resurrect connection to a subscriber.
SyslogCollector - [Thread: main] WARN , 18 Feb 2010 23:59:37,857, Unable to resurrect connection to a subscriber.
SyslogCollector - [Thread: main] INFO , 18 Feb 2010 23:59:37,869, Service started...

[Joe Clarke]

Did you run lsof as root?  It needs to be run as root to get accurate data.

[UWE STEINHAU]

Yes, I did run it as root.  I could download a different version of lsof (currently lsof 4.80).

[Joe Clarke]

If you ran it as root, and tcp/4444 is not bound, then SyslogCollector did not start properly.  In that case, more debugging is needed from the Collector.  I recommend you open a TAC service request so they can walk you through turning on the debug, and verifying the current syslog setup.