Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Community Member

Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Problem Details: I have 75 devices that fails when performing inventory collection. The

Message ID I am receiving is RICS0001 (Cannot successfully collect Inventory Information

for device) Internal Error. After trying to collect inventory again it happened again,

here is the information I was able to collect for one of the devices in the IC_Server.log

I have a TACService Request opened, but its been a couple of days since I heard anything. Just wondering has anyone else encountered issues with these devices. Wasn't a problem for me when I on LMS 2.5.1

Attached is the log I ran after setting the log level to fatal.

3 ACCEPTED SOLUTIONS

Accepted Solutions
Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Are all 75 failing devices 3550s? There was a new bug for in RME 4.1 that would account for IOS switches seeing this problem. The bug is CSCsk65355. There is a patch available by calling the TAC.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

You can always runs a sniffer trace. This is really the best way to diagnose inventory problems when the stack trace contains a MIB module name.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

We are constantly evolving our device packages. We may be polling for newer objects in newer device packages, and tickling some device-side bugs. As I said, the sniffer trace for a working and non-working device would give a more complete picture as to why one is failing. Something must be different in the ENTITY-MIB data one device is returning.

39 REPLIES
Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Are all 75 failing devices 3550s? There was a new bug for in RME 4.1 that would account for IOS switches seeing this problem. The bug is CSCsk65355. There is a patch available by calling the TAC.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Thanks for the update. I have a case currently opened Case #607070583. You are always such a good help when it comes to these types of questions.I guess I will keep waiting on them to send me the patch....

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

I just recently installed lms 3.0 in parallel to 2.6 and am having the exact same problem. It seems to only happen with the Cat3550 - 48port switches in the network. I tried to do an SNMP walk on it and i get a "No Such Object available on this agent at this OID"

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

I don't know on what object you are performing the SNMP walk, but as I said, there is a bug that affects inventory collection of 3550 switches in RME 4.1. You will need to call the TAC to get the patched SharedInventoryCatIOS.zip to fix the problem.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

That is correct. I have a current TAC request open as well and I have received the patch and install so far unsuccessful the developers are now looking into the logs I sent to determine the problem.

Here are the Steps they sent to install the patch.

* Stop the daemons

* Take backup of SharedInventoryCatIOS.zip present in the location NMSROOT\MDC\tomcat\webapps\rme\WEB-INF\lib\pkgs.

* Copy the attached zip file(SharedInventoryCatIOS.zip) to the location NMSROOT\MDC\tomcat\webapps\rme\WEB-INF\lib\pkgs.

* Start the daemons

* The affected devices need to be deleted and re-added.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

These instructions are slightly wrong. When you backup SharedInventoryCatIOS.zip, you must either put the backup in another location, or name the backup something that does not end in .zip or .jar (e.g. SharedInventoryCatIOS.zip.orig). If the .zip extension is preserved, the patch will not work.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Actually I moved the current SharedInventoryCatIOS.zip to the desktop on the server first and then copied over the patch they sent me to the directory that's in the instructions and started the daemons.

Deleted all the devices in re-added them and I still get the exact same number of failed devices. I'm actually going to request a webmeeting today to see if I can get this matter resolved.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Ah, if you did that, check the perms on the patched SharedInventoryCatIOS.zip. I'll bet casuser has no access to the new file.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

I just opened my TAC case... If you get a resolution, please post it. Thanks.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

I just checked and noticed the local admin user I have on the server is casuser, but when I checked the properties files on the zips in that directory its casusers so I manually added casuser full rights to the folder and now i'm starting the daemons now and I will delete and re-add the affected devices. I will follow up with you in a moment with the results.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Here is what I just noticed...in the local admin group I a casuser account but when I check the properties of the CSCOpx it looks like it using casusers account is being used for some reason will this have an affect on the perms?

The local admin group is casusers and the local acct is casuser.

I just ran an inventory collection once again with casuser and casusers full control of the file and still the same devices are failing.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

casuser and casusers should not be administrators on the system for security reasons. They must, however, have full control to all files and directories under NMSROOT.

For the devices that are failing, do you see the same errors in IC_Server.log?

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Thanks.....Attached is the IC_Server.log I ran the Inventory Collection and set the log level to debug mode as instructed from the person who is working my request.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Assuming one of the failing 3550s is 172.18.73.242, this looks like a different problem now. Please post the running config from this switch.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Please see attached.

Yes that is one of the peasants thats giving me issues.

The funny thing I have a number of 3550 48Switch devices at another location that I am not having issue with adding to Inventory collection.

If you want I can send you the running config from one of those devices as well.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

This could be an IOS bug. Take a look at one of the working 3550s, and compare the IOS version and config to see if there are any obvious differences. If there are none, you might try restarting a failing switch to see if that gets around the problem.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

I will have my Network Engineer look into the configs to see if he can determine what's going on. Will keep you updated.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

The code running on my 3550-48s is 3550-ipservices59-mz.122-25.SEB4. It is only affecting the 3550-48 port switches it looks.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

First, you need to get the patch for the known bug. That may resolve your problem (it should). If you still see failures, then additional troubleshooting can begin.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Will do. I am waiting on TAC to respond to the service request I opened. I will update once I get the patch and install it.

Thanks

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Good morning,

I compared the configs on the working and failed switches and the only thing i noticed was a access list permit statement for the CiscoWorks Server IP address the failed switched had the old CiscoWorks server IP address so I changed it and also changed the R/W community string.

The collection still failed but I was able to get this info from the log.

[ Tue Oct 30 15:35:00 EDT 2007 ],DEBUG,[Thread-22],com.cisco.nm.rmeng.inventory.ics.core.DevMgmtStatusUtil,462,QQQWQQ2: update Dev_Mgmt_Status set Sys_Object_Id = '.1.3.6.1.4.1.9.1.367',Dev_Status = -1,Inventory_Collection_Status = 3,AG_Collected = '',Failure_Reason = 'Device sensed, but collection failed ',Corrective_Action = 'Device side issue',Last_Updated_Time = '2007-10-30 15:35:00.238' where NetworkElementID = 4117

Device side issue concerns me....we will continue to investigate as to what is going on. I'm nervous about restarting the switches, this may have to be an after-hours project.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

What about the versions of code? The non-working switch is running an older 12.1 version. Is the working switch running a newer version?

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Attached are the show version outputs.

File sp00wsw.txt is the failed device

Files idjp3sw.txt is the successful device

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

So the working one is running a newer version of code. This may very well be a device side problem in the ENTITY-MIB. A sniffer trace of inventory collection from both devices should confirm the problem. Moral of the story will most likely be you need to upgrade the failing device.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

That is correct. When you say sniffer trace of inventory collection from both devices can I run one on the failed device even though it's failing collection in CiscoWorks.

Sounds like its a good time to start learning the Software Mgmt piece in RME to for Software Distribution.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

You can always runs a sniffer trace. This is really the best way to diagnose inventory problems when the stack trace contains a MIB module name.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

Thanks.....I will pass this info along to the Network Engineers.

Community Member

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

I have one last question regarding this issue. When we were on LMS 2.5.1 the same devices that are failing now we didn't experience issues with Inventory collection. Is Ciscoworks not backward compatible? I'm sure the patch works for the IOS bug, but from a device standpoint when I ran the WindowCL tool I was able to collect info. Sorry to be a pest but just trying to gather as much information as I can so I can pass along to the Engineers.

Cisco Employee

Re: Inventory Collection Failed on Cisco 3550 Switches LMS 3.0

We are constantly evolving our device packages. We may be polling for newer objects in newer device packages, and tickling some device-side bugs. As I said, the sniffer trace for a working and non-working device would give a more complete picture as to why one is failing. Something must be different in the ENTITY-MIB data one device is returning.

371
Views
14
Helpful
39
Replies
CreatePlease to create content