Not getting syslog alerts.

Unanswered Question
Aug 2nd, 2010

Hi,

I am not getting syslog alerts from my CiscoWorks (LMS 2.6 & RME 4.0.5)
It was working fine beofore, but since 3 days it stopped to  work.
I checked the related services in server and all are looks fine (please find the attached file for the same).
Dont now what caused.

Can some one help ome on this please...


Thanks in advance.


Regards,
Naidu.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Joe Clarke Mon, 08/02/2010 - 16:58

Post he output of the pdshow command from the server as well as screenshots of RME > Tools > Syslog > Collector Status and Message Filters.

Latchum Naidu Mon, 08/02/2010 - 22:48

Hi Joe,

Please find the below output of pdshow command:

        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:28
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= EPMDbEngine
        State  = Program started - No mgt msgs received
        Pid    = 7632
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:32
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Application started by administrator request.

        Process= EPMDbMonitor
        State  = Running normally
        Pid    = 7836
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:35
        Stop   = Not applicable
        Core   = Not applicable
        Info   = DbMonitor Running Normally.

        Process= EPMServer
        State  = Administrator has shut down this server
        Pid    = 0
        RC     = 1
        Signo  = 0
        Start  = 02-08-2010 15:36:35
        Stop   = 02-08-2010 15:39:17
        Core   = Not applicable
        Info   = Server started by admin request

        Process= FHServer
        State  = Program started - No mgt msgs received
        Pid    = 7952
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:39
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= diskWatcher
        State  = Running normally
        Pid    = 7988
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:40
        Stop   = Not applicable
        Core   = Not applicable
        Info   = diskWatcher initialized OK.

        Process= DFMMultiProcLogger
        State  = Program started - No mgt msgs received
        Pid    = 7996
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:40
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= DFMLogServer
        State  = Program started - No mgt msgs received
        Pid    = 8004
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:40
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= DFMCTMStartup
        State  = Administrator has shut down this server
        Pid    = 0
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:40
        Stop   = 02-08-2010 15:39:17
        Core   = Not applicable
        Info   = Server started by admin request

        Process= DfmBroker
        State  = Running normally
        Pid    = 8140
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:42
        Stop   = Not applicable
        Core   = Not applicable
        Info   = DFM broker Running

        Process= DfmServer
        State  = Running normally
        Pid    = 8172
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:46
        Stop   = Not applicable
        Core   = Not applicable
        Info   = DFM server Running

        Process= DeviceDiscovery
        State  = Transient terminated
        Pid    = 0
        RC     = 0
        Signo  = 0
        Start  = 03-08-2010 07:00:01
        Stop   = 03-08-2010 07:00:03
        Core   = Not applicable
        Info   = Server started by admin request

        Process= CSRegistryServer
        State  = Running normally
        Pid    = 8180
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:46
        Stop   = Not applicable
        Core   = Not applicable
        Info   = CSRegistryServer is running in other process

        Process= CmfDbEngine
        State  = Program started - No mgt msgs received
        Pid    = 8188
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:48
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Application started by administrator request.

        Process= CmfDbMonitor
        State  = Running normally
        Pid    = 5972
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:51
        Stop   = Not applicable
        Core   = Not applicable
        Info   = DbMonitor Running Normally.

        Process= DCRServer
        State  = Running normally
        Pid    = 5984
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:36:52
        Stop   = Not applicable
        Core   = Not applicable
        Info   = DCRServer is up and running

        Process= CampusOGSServer
        State  = Program started - No mgt msgs received
        Pid    = 1300
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:25
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= CMFOGSServer
        State  = Program started - No mgt msgs received
        Pid    = 3392
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:25
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= ConfigMgmtServer
        State  = Program started - No mgt msgs received
        Pid    = 6572
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:25
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= EssentialsDM
        State  = Running normally
        Pid    = 920
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:25
        Stop   = Not applicable
        Core   = Not applicable
        Info   = RME Device Management started.

        Process= NCTemplateMgr
        State  = Running normally
        Pid    = 3644
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:29
        Stop   = Not applicable
        Core   = Not applicable
        Info   = NCTemplateMgr started.

        Process= NetShowMgr
        State  = Running normally
        Pid    = 6664
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:29
        Stop   = Not applicable
        Core   = Not applicable
        Info   = NetShowMgr started.

        Process= RMEOGSServer
        State  = Program started - No mgt msgs received
        Pid    = 6672
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:29
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= TISServer
        State  = Program started - No mgt msgs received
        Pid    = 6648
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:29
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= DFMOGSServer
        State  = Program started - No mgt msgs received
        Pid    = 7188
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:33
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= InventoryCollector
        State  = Running normally
        Pid    = 2600
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:36
        Stop   = Not applicable
        Core   = Not applicable
        Info   = InventoryCollector is up and running

        Process= Interactor
        State  = Program started - No mgt msgs received
        Pid    = 7880
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:41
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= NOSServer
        State  = Running normally
        Pid    = 8064
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:41
        Stop   = Not applicable
        Core   = Not applicable
        Info   = NOSServer is up and running

        Process= PTMServer
        State  = Running normally
        Pid    = 8128
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:43
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Started Successfully

        Process= EDS-GCF
        State  = Running normally
        Pid    = 6132
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:44
        Stop   = Not applicable
        Core   = Not applicable
        Info   = GCF Initialization complete at Mon Aug 02 15:39:05 CEST 2010

        Process= jrm
        State  = Running normally
        Pid    = 6364
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:37:46
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= 1004
        State  = Never started
        Pid    = 0
        RC     = 0
        Signo  = 0
        Start  = N/A
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Not applicable,

        Process= CTMJrmServer
        State  = Running normally
        Pid    = 9704
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:39:03
        Stop   = Not applicable
        Core   = Not applicable
        Info   = CTMJrmServer started.

        Process= ChangeAudit
        State  = Program started - No mgt msgs received
        Pid    = 9848
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:39:06
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= SyslogAnalyzer
        State  = Program started - No mgt msgs received
        Pid    = 4164
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 16:33:47
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request

        Process= DataPurge
        State  = Administrator has shut down this server
        Pid    = 0
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:39:06
        Stop   = 02-08-2010 15:39:38
        Core   = Not applicable
        Info   = DataPurge initialization completed at Mon Aug 02 15:39:38 CEST
2010.

        Process= ANIDbEngine
        State  = Program started - No mgt msgs received
        Pid    = 9872
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:39:08
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Application started by administrator request.

        Process= ANIServer
        State  = Running with busy flag set
        Pid    = 10852
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:39:45
        Stop   = Not applicable
        Core   = Not applicable
        Info   = ANIServer started.

        Process= UTMajorAcquisition
        State  = Never started
        Pid    = 0
        RC     = 0
        Signo  = 0
        Start  = N/A
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Not applicable,

        Process= AdapterServer
        State  = Program started - No mgt msgs received
        Pid    = 3692
        RC     = 0
        Signo  = 0
        Start  = 02-08-2010 15:39:17
        Stop   = Not applicable
        Core   = Not applicable
        Info   = Server started by admin request


C:\Documents and Settings\kwifii>

Regarding for Syslog Collector status, please find the attached files.

Regards,

Naidu.

Joe Clarke Tue, 08/03/2010 - 08:38

The pdshow is truncated.  You should run:

pdshow > pdshow.txt

Then post the pdshow.txt file.  However, things look to be working.  You have a rather large forwarded message count.  What messages are you receiving that you are not able to see in the RME reports?

Latchum Naidu Wed, 08/04/2010 - 03:47

Hi joe,

Please find the attached pdshow.txt file.

Also please find the attached syslog two emails whih I was getting previously from [email protected] but not now.

And I am able to see the device logs in Device Center-->Select Device-->Syslog Message Report.

I have configured syslog email notifications n Device Fault Manager-->Notification Services-->Email Notifications.

It was working before but not working since 1 week.

Regards,

Naidu.

Joe Clarke Wed, 08/04/2010 - 12:31

These emails have nothing to do with RME or syslog.  They are generated from DFM.  The reason you're not getting any more messages is that EPMServer is down (along with some other daemons).  Try restarting Daemon Manager on this server, then see if new DFM events generate email notifications.

jopetik09 Thu, 08/05/2010 - 02:40

I restrated the Daemon Manager service in LMS server Services but No luck.

Also started the three EPM process under Common Services-->Server-->Admin-->Processes. Please find the attached pic for the processes status.

Previously I had the same kind of issue and resolved when I creaded one new Notification Group in DFM, then I replaced the old group with new one.

I tried do the same thing now but when I try to open Notification Groups tab in DFM I am getting below error.

********************************

HTTP Status 500 - Internal Server Error


type Exception report

message Internal Server Error

description The server encountered an internal error (Internal Server Error) that prevented it from fulfilling this request.

exception

java.lang.NullPointerException
     at com.cisco.nm.trx.nos.action.NGroup.perform(NGroup.java:140)
     at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1786)
     at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1585)
     at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:491)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
     at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
     at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
     at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
     at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
     at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
     at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
     at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
     at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
     at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
     at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
     at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
     at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
     at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
     at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
     at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
     at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
     at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
     at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
     at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
     at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
     at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
     at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
     at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
     at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
     at org.apache.ajp.tomcat4.Ajp13Processor.process(Ajp13Processor.java:457)
     at org.apache.ajp.tomcat4.Ajp13Processor.run(Ajp13Processor.java:576)
     at java.lang.Thread.run(Thread.java:534)


Apache Tomcat/4.1.29

*************************************************

Please suggest me how it can be resolved.

Regards,

Naidu.

Attachment: 
Joe Clarke Thu, 08/05/2010 - 08:26

Post the new pdshow output.  Exactly what version of DFM do you have?

Latchum Naidu Thu, 08/05/2010 - 22:54

Please find the attached new pdshow output.

Also find the attached file for one device which is not showing Device alert identifier, Alert status, Alert Description.

Currently I am using 2.0.10

Regards,

Naidu.

Joe Clarke Fri, 08/06/2010 - 08:59

EPMServer is still down.  Post the EPMServer.log and all of the logs under NMSROOT/log/dfmLogs/EPM.

Latchum Naidu Sun, 08/08/2010 - 23:24

Hi Joe,

Please find the attached log files for your further investigation.

I checked the following logs files DfmBroker.log, DFMCTMStartup.log and DfmServer.log and they are empty.

Regards,

Naidu.

Latchum Naidu Sat, 08/14/2010 - 11:09

Hi Joe,

I didnt see any folders / named "dfmLogs" under present NMSROOT/log

I could see this folder in old log which i moved from NMSROOT/log to D drive (D:\log-1\dfmLogs\EPM) to make some free space. At that time I was trying to upgrade LMS from 2.6 to 3.0 as it needs 25 GB free space of 'C' drive (where LMS installed)

Do you want me to place back the old log folder (which i moved to D drive)?

Regards,

Naidu.

Joe Clarke Sat, 08/14/2010 - 12:14

You can't move things out of NMSROOT like that.  By doing so, you have broken DFM.  Move the directory back.  If you moved it away before upgrading to LMS 3.2, though, you will most likely have to uninstall DFM, then reinstall it.

Latchum Naidu Sat, 08/14/2010 - 14:07

Hi Joe,

Yes, it is my mistake. Actually the log file is increasingly constantly and the free disk space is getting slow, so I just thought to move the log file to safe location.

After I moved the log directory to D drive, there is one more new log directory has been created under C:\Program Files\CSCOpx  but no directories like conf, dfmlogs..etc.,

If i move back the old log directory to C:\Program Files\CSCOpx it will definetely ask like "do you want replace with existing folder" what you want me to do now?

Regards,

Naidu.

Joe Clarke Sat, 08/14/2010 - 16:48

The conf directory is the critical one.  Did you actually upgrade to LMS 3.x on this server?  If so, I strongly recommend you reinstall DFM, and that should restore DFM's functionality.

If not, then delete the NMSROOT/log/dfmLogs that is there now, and copy back the ENTIRE TREE that you had previously moved out.  This should be done with Daemon Manager shut down.  When complete, restart Daemon Manager, and DFM should work again.

Latchum Naidu Mon, 08/16/2010 - 02:55

Hi Joe,

I have not upgraded the LMS from 2.6 to 3.0

At present under C:\Program Files\CSCOpx I am seeing the folder named "log" and there is no such dfmLogs folder.

Can I delete the present "log" folder and move the actual old log folder (which have folders conf & dmflogs)

If I moved the actual log folder to NMSROOT, it will ask to replace with the exiting folder as it is same name.

suggest me how to proceed.

Regards,

Naidu.

Joe Clarke Mon, 08/16/2010 - 09:24

Do not remove the log directory.  Instead, copy into this directory the config and dfmLogs directories from the old log directory.

Latchum Naidu Tue, 08/17/2010 - 01:37

Hi Joe,

Thanks & Thanks for your continue support and excellet solutions.

I moved the conf & dmflogs to exiting log folder and DMF is working fine

I created the new group and subscribed this new group at Email Notification and Syslog Notification, but still I am not getting the email notifications like the attached one which I used to get.

Regards,
Naidu.

Actions

This Discussion