06-18-2007 04:15 PM
RME starts returning "running config unavailable" for a large number (hundreds) of IOS and PIX devices. "Compare Startup vs Running" or "Running vs Latest Archived" errors out:
Startup/Running config fetch failed. Cannot compare configurations.
When choosing "Two versions of the same Device", I can see two generations of running configs for a particular device (#10 and 11), but then I get
HTTP Status 500 - Internal Server Error
--------------------------------------------------------------------------------
type Exception report
message Internal Server Error
description The server encountered an internal error (Internal Server Error) that prevented it from fulfilling this request.
exception
java.lang.NullPointerException
at com.cisco.nm.rmeng.dcma.ui.helper.DCMADiffUIUtil.getConfigletDiff(DCMADiffUIUtil.java:176)
at com.cisco.nm.rmeng.dcma.ui.action.DCMAConfigDiffViewer.perform(DCMAConfigDiffViewer.java:540)
at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1786)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1585)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:491)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
at org.apache.ajp.tomcat4.Ajp13Processor.process(Ajp13Processor.java:457)
at org.apache.ajp.tomcat4.Ajp13Processor.run(Ajp13Processor.java:576)
at java.lang.Thread.run(Thread.java:534)
--------------------------------------------------------------------------------
Apache Tomcat/4.1.29
All these devices show up under Partially Successful for Config Archive purpose, but other Partially Successful ones have no problem with RME displaying their running configs.
Is there any way to address this situation without reinitializing the RME db?
06-18-2007 05:02 PM
It almost looks like someone went and cleaned out your archive on the file system. Enable ArchiveMgmt Client debugging under RME > Admin > System Preferences > Loglevel Settings, and reproduce this error. Then provide the dcmaclient.log.
06-25-2007 11:03 AM
Could you take a look at the dcmaclient.log attached to TAC Case 606284225, when you get a chance? Thanks.
06-25-2007 11:20 AM
Looks like you have some corrupt files on the file system. In particular, the config files under /var/adm/CSCOpx/files/rme/dcma/devfiles/532 appear to be corrupt. That appears to be device 10.100.20.112. You might try deleting this device from RME, then re-adding it to see if that corrects the problem you're seeing.
06-25-2007 11:28 AM
This is just one out of hundreds. Does that mean it has to be done against all of them? Any way to prevent what caused the corruption in the first place from recurring?
06-25-2007 11:42 AM
The only error in the log points to this one device. However, if other devices exhibit the same problem, then the same thing would apply to them as well. The only prevention against this type of error is to do regular backups.
06-25-2007 11:56 AM
So the cause of this corruption is supposed to be at the file system or hardware level, rather than a bug or defect of RME 4.x? Are the "backups" in your post referring to those produced by LMS 2.6, or by other means such as NetBackup? In the case of the former, how would I restore just the archives for those devices considered corrupt by RME, without setting back other unaffected devices as well? I do have "shadow" directory enabled, if that matters.
06-25-2007 11:59 AM
The corruption seen in the logs is most likely not caused by a bug in RME. The files appear to be truncated which is typical of out-of-space conditions or hardware problems.
I refer to an LMS backup. Of course, there is no way to partially restore these backups. In this case, though, since so many devices are affected, a good backup would be the way to go.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide