cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1114
Views
0
Helpful
10
Replies

Possible corruption in RME? Change Audit discrepancy.

stevek1
Level 1
Level 1

Hi All,

Running LMS 2.6. Looking at a Change Audit report, I see that the change poller has detected a change to a switch by user BloggsJ. Only thing is, BloggsJ did not even log on to this switch let alone make a change to the config.

Any ideas friends?

Cheers, SteveK.

1 Accepted Solution

Accepted Solutions

This is a known bug, CSCsi33887. The problem is that the original RME database lacks the schema changes necessary to implement this change. If you open a TAC service request, they can make a workaround available to you.

View solution in original post

10 Replies 10

miheg
Level 5
Level 5

Check the log of that machine. Or else the syslog.

If it also mentions BloggsJ then someone or something(ciscoworks) might be using his credentails.

Cheers,

Michel

Thanks Michel,

It wasn't actually Joe Bloggs (just an example name). It has been recording these changes against random members of the comms team on random devices that these guys didn't log in to. Interesting that you think maybe CW could be using their credentials. I am confident that people aren't sniffing and using other people's credentials - I am more concerned that the database may be corupted and what I can do to resolve.

I thought there was perhaps a reinit script for certain databases which may apply.

Any advice?

Cheers SteveK.

While you can certainly reinitialize the RME database, this may not be your problem. I've never seen this kind of corruption before. RME doesn't make up usernames. If Change Audit is reporting a config change was made by a certain user, either that user logged in to the device and made the change, they scheduled a job in RME that made the change, or RME used their credentials (perhaps using job-based credentials) to login and make a change to the device. The additional change details may help narrow down from which source the change occurred.

If you still want to reinitialize the RME database, you can run:

NMSROOT/bin/perl NMSROOT/bin/dbRestoreOrig dsn=rmeng dmprefix=RME

However, this will clear all inventory, configuration, syslog, and software image collections for all of your devices.

Thanks Joe,

I don't want to lose my historical data so at this stage I'm avoiding the reinit. I've run a validation check and it reported no errors. This particular issue only relates to switch changes and there are a lot of them. They are picked up by the Config poller when it runs overnight. For example, last night's job recorded changes to the VLAn config of 338 switches. These changes were randomly attributed to every member of the communications team I work in. NONE of us logged into any of these switches nor were there any scheduled cw2k jobs on them.

Is the poller simply noting a spanning-tree change and randomly assigning a user from the database to satisfy the required output of the change audit report? What do you think Joe?

SteveK.

VLAN config changes cannot be polled, so the VLAN config will always be fetched. Changes to the VLAN config are also not very useful. If you want, you can disable creating Change Audit records for the VLAN config by checking "Enable vlan Change Audit Filter" under RME > Admin > Change Audit > Config Change Filter.

As for where the username is coming from, you might try walking ccmHistoryEventTerminalUser (1.3.6.1.4.1.9.9.43.1.1.6.1.8) on these switches and see if the user shows up there.

Thanks Joe,

I tried to Enable vlan Change Audit Filter as directed but it didn't take effect.

I check the box to enable the filter, apply the change and receive a notification back that the new settings saved successfully. However, when I go back in, the filter is deselected again.

I tried it on my other cw2k server and it holds the new setting OK. For the one that didn't work, I tried logging in as admin and got the same results. I have also rebooted this box since and it still doesn't accept the change. Any ideas mate?

SteveK

This is a known bug, CSCsi33887. The problem is that the original RME database lacks the schema changes necessary to implement this change. If you open a TAC service request, they can make a workaround available to you.

Thanks Joe,

Once again you've come up trumps. Whatever they're paying you - it ain't enough.

SteveK.

Hi Joe,

Received and implemented the workaround script - works fine thankyou.

So now I can filter out the Vlan changes and my 24 hr Change Audit report has reduced from 20 pages to 2 pages. One thing I've noticed however, is that last night the change poller picked up 4 config changes (genuine changes of interest) and recorded random usernames against them. These are real data comms userids, but these users didn't login to the device nor schedule any cw2k jobs). This was one of the issues I noted originally with the Vlan changes.

SteveK.

See one of my earlier replies, and check ccmHistoryEventTerminalUser on these devices to see if the usernames appear there. Also, do a DCR export for these devices, and look at the credentials in the resulting CSV file. Perhaps those users used their credentials when adding the devices to DCR.

If none of that pans out, you will need to enable ChangeAudit and ArchiveMgmt Service debugging under RME > Admin > System Preferences > Loglevel Settings, reproduce the problem, then check the dcmaservice.log and cas.log to see where that username is coming from.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco