Syslog issue

Answered Question
Aug 31st, 2009
User Badges:

I have installed a fresh copy of LMS3.0 on Solaris 10 but I don't get any syslog reports. When I go in var/log/syslog_info there are no messages in the file. Do I need to update anything else for syslog to get the messages in syslog_info?

Correct Answer by Joe Clarke about 7 years 7 months ago

Try forcing a restart of syslogd:


svcadm disable svc:/system/system-log


svcadm enable svc:/system/system-log

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Joe Clarke Mon, 08/31/2009 - 08:35
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

You need to make sure /etc/syslog.conf is properly configured so that messages from your devices are written to syslog_info. The default config line is:


local7.info /var/log/syslog_info


(Note: there are TABs NOT spaces between local7.info and /var/log/syslog_info.)


Once this file is updated, you must restart syslogd:


kill -HUP `cat /var/run/syslogd.pid`


The default assumes you are using local7 on your devices to send syslog messages. This, too, is the default for Cisco devices. If you are using a different facility, update syslog.conf accordingly.

nawas Fri, 09/04/2009 - 06:32
User Badges:

I checked syslog.conf file and it is configured as you have said. I only see the following logs in syslog_info but I don't see any other logs from my devices.

Sep 1 20:15:03 sip8a tsm_backup.pl[7555]: [ID 702911 local7.info] active, backing up

Sep 1 20:25:12 sip8a tsm_backup.pl[7555]: [ID 702911 local7.info] inspected=240037,backedup=723,transferred=617.42 MB,failed=0

Sep 1 20:25:12 sip8a tsm_backup.pl[7555]: [ID 702911 local7.info] backup complete RC=0

Sep 2 20:15:02 sip8a tsm_backup.pl[1225]: [ID 702911 local7.info] beginning detection of active host

Sep 2 20:15:02 sip8a tsm_backup.pl[1225]: [ID 702911 local7.info] Cannot execute /opt/scripts/ha/ha_control.pl ... Assuming no HA a

nd this host is active

Sep 2 20:15:02 sip8a tsm_backup.pl[1225]: [ID 702911 local7.info] active, backing up

Sep 2 20:24:02 sip8a tsm_backup.pl[1225]: [ID 702911 local7.info] inspected=240149,backedup=515,transferred=746.59 MB,failed=0

Sep 2 20:24:02 sip8a tsm_backup.pl[1225]: [ID 702911 local7.info] backup complete RC=0

Sep 3 20:15:03 sip8a tsm_backup.pl[29669]: [ID 702911 local7.info] beginning detection of active host

Sep 3 20:15:03 sip8a tsm_backup.pl[29669]: [ID 702911 local7.info] Cannot execute /opt/scripts/ha/ha_control.pl ... Assuming no HA

and this host is active

Sep 3 20:15:03 sip8a tsm_backup.pl[29669]: [ID 702911 local7.info] active, backing up

Sep 3 20:26:56 sip8a tsm_backup.pl[29669]: [ID 702911 local7.info] inspected=241879,backedup=10384,transferred=2.23 GB,failed=0

Sep 3 20:26:56 sip8a tsm_backup.pl[29669]: [ID 702911 local7.info] backup complete RC=0

I have attached a copy of syslog.conf



Attachment: 
Joe Clarke Fri, 09/04/2009 - 08:37
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This whole syslog.conf looks wrong as spaces are used instead of tabs. Perhaps you did some kind of conversion when you posted it. If it is really using spaces, fix it so all spaces are tabs.


Other than that, make sure your devices are sending syslogs using the local7 facility. If you see another facility configured (e.g. logging facility syslog), then either fix the device, or change the facility in syslog.conf.

nawas Tue, 09/08/2009 - 10:31
User Badges:

file check is OK, local7 is verified,

I see the following in the syslogcollector.log

SyslogCollector - [Thread: main] INFO , 04 Sep 2009 11:08:27,613, System Initialized.

SyslogCollector - [Thread: main] WARN , 04 Sep 2009 11:08:29,726, Unable to resurrect connection to a subscriber.

SyslogCollector - [Thread: main] INFO , 04 Sep 2009 11:08:29,753, Service started...

SyslogCollector - [Thread: main] INFO , 08 Sep 2009 09:45:45,337, Logging System Initialized.

SyslogCollector - [Thread: main] INFO , 08 Sep 2009 09:45:45,339, System Initialized.

SyslogCollector - [Thread: main] WARN , 08 Sep 2009 09:45:47,372, Unable to resurrect connection to a subscriber.

SyslogCollector - [Thread: main] INFO , 08 Sep 2009 09:45:47,398, Service started...

When I try to subscribe/unsubcrible syslog server I get the following:


SyslogCollector - [Thread: Thread-12] WARN , 08 Sep 2009 12:47:57,363, Unable to add monitor for cw2klms3

Joe Clarke Tue, 09/08/2009 - 10:56
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Trying to subscribe to a Collector is pointless if the messages are not arriving in the syslog_info file. Is this still the case?

nawas Tue, 09/08/2009 - 11:13
User Badges:

Yes, no messages arriving the syslog_info.I checked syslog_info and made sure it has the one TAB and no spaces between local7.info and /var.

I have attached an original copy from the server for your review.




Attachment: 
Joe Clarke Tue, 09/08/2009 - 11:21
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

If you're sure the devices are sending local7 messages, start a snoop on the server, then generate some messages from a test device:


snoop -o outfile -s 1518 udp port 514 and host IP


Where IP is the IP address of the device sending the messages. After you've collected enough packets, post the outfile. Of course, if no messages are captured, then this means that they are not arriving on the server, and something is blocking them in the network. Find out what is blocking udp/514, and fix it so those messages can make it to the server.

nawas Tue, 09/08/2009 - 11:31
User Badges:

Quick question before I do the snoop, I have three logging servers defined in my device, please see config


logging CiscoWorksServer1

logging WhatupServer

logging CiscoWorksServer2


syslog is working fine on server 1. syslog also works for what's up server but nothing works for CiscoWorksServer2

(server2 is a new install though)


1. Is there a limit that no of servers can be defined in one device?

2. Should I keep the server 2 on top and try?


When I do show log I see the following


Logging to CiscoWorksServer1, 192640 message lines logged, xml disabled,

filtering disabled

Logging to WhatupServer, 439 message lines logged, xml disabled,

filtering disabled

Logging to CiscoWorksServer2, 409 message lines logged, xml disabled,

filtering disabled


And over time number of messages logged increases for CiscoWorksServer2 which tells me that message are going somewhere in

CiscoWorksServer2 but not in the syslog_info file. Do you agree?

yjdabear Tue, 09/08/2009 - 11:41
User Badges:
  • Gold, 750 points or more

That's part of the problem: IOS can only log to two syslog servers.

Joe Clarke Tue, 09/08/2009 - 12:12
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

That's not true. You can have as many syslog servers as you want. We RECOMMEND you have no more than three.

yjdabear Wed, 09/09/2009 - 04:58
User Badges:
  • Gold, 750 points or more

I stand corrected. I think we saw high CPU util when having 3 or more syslogging destinations.


In a similar vein, how many "snmp-server enable traps" destinations can IOS handle?

Joe Clarke Tue, 09/08/2009 - 12:11
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Yes, the messages are being sent, but they're either being dropped in the network or on the server. The sniffer trace will help pinpoint which.

nawas Tue, 09/08/2009 - 13:06
User Badges:

Here is the snoop capture, I did the conf t and shut/no shut the interface. I generated 5 line and they show up in the capture.



Attachment: 
Joe Clarke Tue, 09/08/2009 - 13:11
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Try restarting syslogd on the server, and then regenerate the same messages to see if they show up in the syslog_info file:


kill -HUP `cat /var/run/syslogd.pid`

nawas Wed, 09/09/2009 - 07:01
User Badges:

Restarted syslogd, regenerated the message but stil it didn't make it to syslog_info. I have attached the snoop capture.



Attachment: 
Joe Clarke Wed, 09/09/2009 - 10:33
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Post the output of ps -efl and netstat -an.

Joe Clarke Wed, 09/09/2009 - 13:18
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Syslog is running, but not bound to udp/514. Post the output of:


svcprop svc:/system/system-log:default

Joe Clarke Thu, 09/10/2009 - 08:18
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This is what I thought. Your syslog service is not configured to allow remote message reception. Run these commands as root:


svccfg -s svc:/system/system-log setprop config/log_from_remote = true


svcadm refresh svc:/system/system-log


Then you should be receiving remote messages.

Correct Answer
Joe Clarke Thu, 09/10/2009 - 10:51
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Try forcing a restart of syslogd:


svcadm disable svc:/system/system-log


svcadm enable svc:/system/system-log

nawas Thu, 09/10/2009 - 12:23
User Badges:

That did it. It's working now.

Thanks a lot for all your help Joe. I knew you would resolve it.

Actions

This Discussion