RME not showing the old logs after the switch crashed

Unanswered Question
May 10th, 2009
User Badges:

Hi,

I have another problem. One of my 4500 crashed. After it came up again I am not able to see thelod logs to find out why it crashed. All the old logs has been gone.


Can I find the old logs in any other way?


Thanks

Sachi


  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Joe Clarke Sun, 05/10/2009 - 11:02
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

If the switch was sending syslog messages to RME, then RME should have recorded them in its database. You could run a standard or 24 hour syslog report for this switch under RME > Reports > Report Generator.


However, even if the switch was configured to send syslogs, the crash may not have generated any messages. The "show stack" command should show you the stack trace from the crash, however. That can be used by TAC to determine a cause for the crash.

sachidananda panda Sun, 05/10/2009 - 12:49
User Badges:

The switch was sending syslogs. But in RME database not showing any logs prior to the crash.The logs showing in report generator only after the switch again came up.


Thanks

Sachi

Joe Clarke Sun, 05/10/2009 - 12:57
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

The crash will have no bearing on messages which were already in the RME database. If the switch sent messages to the RME server prior to the crash, and those messages were processed from the syslog.log or syslog_info file, then they will still be in the database (unless they were purged due to your purge settings [7 days by default]).


Check the parameters you are using to run the syslog report. Make sure they cover the date range you want. Make sure that the date on which the messages were sent is not older than your purge interval (configured under RME > Admin > Syslog > Set Purge Policy).

sachidananda panda Sun, 05/10/2009 - 13:18
User Badges:

The purge setting is 7 days for all the devices. all other devices are showing the old logs before the crash date of this device. Only for this device only the old logs are not showing before the crash date.


Thanks

sachi

Joe Clarke Sun, 05/10/2009 - 13:21
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

If the messages never made it to the RME database, then there's nothing that can be done for them now. If the problem happens again (i.e. if you notice certain messages are not being archived), then troubleshooting can be done at that time.


For now, if the switch really was sending the messages (maybe the crash was related to some stuck subsystem which prevented any messages from being sent), they should be in the syslog message log on the RME server (NMSROOT\log\syslog.log on Windows and /var/log/syslog_info on Solaris). You can scan that file to get an idea of what was happening prior to the crash. However, as I said originally, the "show stack" will be more useful in isolating the cause of the crash.

sachidananda panda Sun, 05/10/2009 - 13:32
User Badges:

The syslog.log file is very big in size. I tried to open it but couldnt. Is there any way to open such a big file.


Another thing is before tha crash all the logs for that particular switch not showing when I generate a report. the crash happened only 3 days back. All other devices showing the logs for prior to that day except this one.


Thanks

sachi

Joe Clarke Sun, 05/10/2009 - 13:37
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

You may need to use a text viewing program other than Notepad to open. I know a lot of Windows users like Ultra Edit (http://www.ultraedit.com/downloads/ultraedit_download.html ). You can download an evaluation. there's also grep for Windows (http://gnuwin32.sourceforge.net/packages/grep.htm) which can be used to search the log for messages pertaining to this device.


My theory is that there was something wrong on the switch which prevented it from sending syslog messages. If the messages are now appearing in RME, and nothing was changed in RME, then that corroborates my theory.


However, if the problem happens again, then more troubleshooting can be done at that time.

Actions

This Discussion