RME uses wrong IP addresses for telnet - LMS 2.5

Answered Question
Aug 20th, 2008
User Badges:

Hello,


RME 4.0 used wrong IP addresses for telnet connections. I changed the addresses in common services and tested the correct funtion in RME - Device Management - check device credentials.

The telnet connection was working well.

The next morning, the configuration save had failed with error:

TELNET: Failed to establish TELNET connection to 172.23.x.x - Cause: TELNET: Failed to establish TELNET connection to 172.23.x.x

and the IP address had changed to the wrong value.

Even deleting of the device and adding in common services did not help.

Where does RME get the wrong IP addresses which it should not use for telnet connections and how can I prevent it from doing so?


Thank you in advance for your help

Correct Answer by Joe Clarke about 8 years 7 months ago

I occurred to me that by router ID, you most likely mean loopback addresses. This, then, is almost certainly Campus Manager Discovery which is changing your IP address. You should try excluding these devices from Discovery under Campus Manager > Admin > Device Discovery.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.5 (4 ratings)
Loading.
Joe Clarke Wed, 08/20/2008 - 07:19
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

RME gets device address information from DCR. If you do not have the IP address configured in DCR, but rather just the hostname, then there is a bug where LMS daemons cache the successful lookup of that hostname indefinitely. This would require you to either enter the IP in DCR, or restart dmgtd every time you change the DNS record for a device. The bug is CSCse85033, and this was fixed in LMS 3.0

netzeatkw Thu, 08/21/2008 - 22:36
User Badges:

I changed the fields 'host name' and 'IP-address' under common services ... device properties to the correct IP address but it did not help. Where do I have to change it? Is it necessary to restart dmgtd when I change these fields?

Joe Clarke Fri, 08/22/2008 - 07:41
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

That is the location where you would need to change the IP address and hostname. When you changed those values, did you schedule a new job, or did you allow an existing job to run again?

netzeatkw Mon, 08/25/2008 - 22:43
User Badges:

After changing the values I did a dmgtd stop / start.

I did not schedule a new job but the existing archive update job ran afterwards and showed failed devices due to the wrong IP addresses.

Joe Clarke Mon, 08/25/2008 - 23:35
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Try creating a new job. The old job may have the IP address information cached.

netzeatkw Tue, 08/26/2008 - 07:13
User Badges:

I only know how to schedule the globel Config Collection under RME -> Admin -> Config Mgmt -> Archive Mgmt -> Collection Settings. How can I Create a Test Config Collection for the changed devices?

Joe Clarke Tue, 08/26/2008 - 09:16
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Go to RME > Config Mgmt > Archive Mgmt > Sync Archive. Select the problem device, and run the job.

netzeatkw Wed, 08/27/2008 - 05:28
User Badges:

That job was this time successful. What do I have to do that the daily RME Archive Mgmt Collection uses the correct values of hostname and IP address?

Joe Clarke Wed, 08/27/2008 - 09:04
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Disable the system polling and collection jobs, and apply those changes. Once the jobs are disabled, return to the Collection Settings interface, and re-enable them. You should get new job IDs, and the problem should go away.

netzeatkw Fri, 08/29/2008 - 01:10
User Badges:

I got new job IDs but the problem still remains the same. The scheduled archive poller and update jobs fail and the device properties are changed to the wrong IP addresses after running the job.

yjdabear Fri, 08/29/2008 - 06:27
User Badges:
  • Gold, 750 points or more

Do you mean the "wrong IP addresses" are populated into the device properties in DCR, after running the jobs? Is it one single "wrong" IP address, or multiple "wrong IP addresses"?

Joe Clarke Fri, 08/29/2008 - 08:40
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

I think I know what's happening here, but I'd like to see a screenshot of everywhere you see this wrong IP address.

netzeatkw Mon, 09/01/2008 - 01:24
User Badges:

There are multiple "wrong IP addresses" that are populated into the device properties in DCR, after running the jobs. Here are screenshots of the failing RME archive job and the changed CS device Properties.



Attachment: 
Joe Clarke Mon, 09/01/2008 - 02:29
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

I don't see the problem. The IP address in DCR is the same that is showing up in RME. If the .123 address is wrong, change it in DCR.

netzeatkw Tue, 09/02/2008 - 00:30
User Badges:

Both files show the wrong IP address. I changed it in DCR to the correct value (.225.23), tested it in RME - check device credentials and created a new archive poller and a new archive update job. After running that job, the IP address had changed to the wrong value.

Joe Clarke Tue, 09/02/2008 - 04:07
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

The IP address should not be changing in DCR on its own. After making the change in DCR, verify the change actually took effect. Select the device from the tree, then click the View button.

netzeatkw Tue, 09/02/2008 - 06:06
User Badges:

I deleted the old archive poller and archive update jobs in RME. Then I changed the IP address and hostname to the correct value and tested the correct function by a RME credential verification report (see attached file). I could start a "sync archive" job in RME archive management and it works well (2nd file). Then I scheduled new RMR archive poller and archive update jobs. They failed (no telnet connection) and the IP addresses had changed.

A colleague of mine found that cisco works tried to use the IP addresses of the router ID instead.



Joe Clarke Tue, 09/02/2008 - 09:03
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

RME will use the IP address which is configured in DCR (if an IP is configured). As to why that IP is changing to the incorrect one, I cannot say for certain. If you have Discovery enabled, the change could correspond to Discovery running. Other than that, the address in DCR should not be changing.


At this point, it might be quicker if you open a TAC service request, and have them walk you through the steps to isolate why the IP is changing in DCR.

Correct Answer
Joe Clarke Tue, 09/02/2008 - 14:38
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

I occurred to me that by router ID, you most likely mean loopback addresses. This, then, is almost certainly Campus Manager Discovery which is changing your IP address. You should try excluding these devices from Discovery under Campus Manager > Admin > Device Discovery.

netzeatkw Thu, 09/04/2008 - 00:08
User Badges:

This hint solved the problem. I had to remove the option "use loopback addresses" in the discovery options of the device discovery and now it works well.


Many thanks

Martin Ermel Thu, 09/04/2008 - 03:21
User Badges:
  • Blue, 1500 points or more

And it raises another option which I think is highly mandatory:

a checkbox in DCR that makes the mgmt IP address of a device permanent. It could be labled 'Retain Management IP' - and ptreserves the mgmt IP even when AutoDiscovery is enabled.

It should be available when going to

CS > Device and Credentials > Device Management > Edit Identity

Like the 'Display Name' that is treated like a string and can be customized there is the need of adjusting the 'IP Address' so that it matches the correct one (if the device has multiple IP addresses) - and prevent AutoDiscovery (i.e. the LMS name resolution mechanism) from overwriting it again (e.g when the choosen name resoltion mechanism does not fit for that device).

The reason for this is that I have never (since now) seen a network with consistent name resolution for ALL devices. There are always exceptions so it is impossible to find one name resoltion mechanism for all devices but only for the majority.

With this option the IP address of the exceptional devices can easily be adopted where as the name resolution mechanism can be choosen to fit the majority of devices.


@jclarke and all the others:

Does anybody see any helpful aspects for this option or see any points from the view of LMS that makes this impossible or hard to realize


- it is obvious that the mechanism that asserts if a found device (by AutoDiscovery) is yet in the DB must still exists - it should just not touch the IP Address if it is yet in the DB and the checkbox is enabled.



g.meerkoetter Thu, 09/04/2008 - 06:12
User Badges:

I'd rather see the algorithm discovery uses to pick the management address made configurable. So we could specify something like e.g.


1. use address from network_a/mask_a

2. else use iftype a or b address (covers loopback)

3. else use address from network_b/mask_b

etc.


In case there is more than one hit, let me choose whether highest or lowest address is preferred.


Or just make sure discovery never picks an unresponsive address! ;-)

Martin Ermel Thu, 09/04/2008 - 06:46
User Badges:
  • Blue, 1500 points or more

I also thought about the oportunity of defining a number of basic and distinct mechanisms for determining the management ip address (e.g like the one you mentioned - and some more) and then have a user interface in which you can choose which one to use and set the sequence order in which they should be applied until the first match.


You are right this would be great - but I think a feature like this won't came before LMS 5.x :-( because it would be a general redesign and this takes time at Cisco... (even if we would be more than 2 supporting this :-) )

- I've got the hope the checkbox could be realized more quickly (LMS 3.2)


g.meerkoetter Thu, 09/04/2008 - 07:15
User Badges:

Well, then I'd go for "Never pick an unresponsive address". I think the current algorithm results in at least undesired, if not buggy behaviour.


I've had a TAC SR (608672369) open for more than a month without getting an explanation or a decent workaround. Fortunately I found the necessary information in one of jclarke's highly appreciated posts.

Martin Ermel Thu, 09/04/2008 - 08:03
User Badges:
  • Blue, 1500 points or more

I agree and would count it as a bug if a management solution chooses an unreachable IP address as the management IP ...

- it somehow controvert the general purpose of a management software... ;-)


In general I think the current problem is that there is no in detail explanation or step-by-step description of the current algorithm - not on cisco.com - and I think not at Cisco internally...

I tried to summarize what I have found in this thread :

http://forum.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Network%20Infrastructure&topic=Network%20Management&topicID=.ee71a02&fromOutline=&CommCmd=MB%3Fcmd%3Ddisplay_location%26location%3D.2cc0c073

but there are still questions...


what I'll never understand is why they drop the sysName from DCR while it is catched by the discovery process. Go to

CS > Device and Credentials > Device Discovery

and open the list for 'Total Device Discovered' => here you have the sysName listed (and the Management IP is not yet determined - it is just listed the IP by which the device was found)

but in 'Devices Updated to DCR' the sysName is gone (and the Management IP and Display Name are determined)

that's why one cannot choose the sysName as a display name - they just drop it ...



Joe Clarke Thu, 09/04/2008 - 08:58
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Yes, picking an unreachable address as the management address could be treated as a bug. Having an option that says, "Do not use an unreachable address" is a bit ridiculous as no one would want to leave this unchecked.


I've summarized the Discovery process in a few of my presentations. My LMS Networkers lab went through this. I think your summary document with my additional comments pretty much sums everything up. The sysName can be used to determine a preferred management address IFF the sysName is resolvable to an IP address and that IP address is reverse resolvable to a hostname. That final hostname may or may not be the original sysName. This is why the update to DCR only includes the Display Name and management IP as that is what are determined once the algorithm runs.


I think in general, many users use Discovery incorrectly. Part of the blame rests with us as we changed terminology from release to release. Discovery is really only a means of populating DCR (i.e. finding new devices in the network). It is not a requirement to use LMS as there are other DCR population mechanisms (e.g. manual import, bulk import, etc.). In a stable network, Discovery would not need to be run much if at all.


If you do have to periodically run Discovery, you might add Discovery exclude filters or DCR exclude filters to prevent existing devices from being updated in DCR. This would serve as a workaround to the original problem in this thread as well as to others reporting a change in management IP.


As for adding new options to present Discovery from updating the management IP and hostname in DCR (as a Display Name update option already exists), I suggest following the usual PERS route. I'll handle the unreachable bug when I return from the UK.

g.meerkoetter Mon, 09/08/2008 - 00:24
User Badges:

Thanks Joe, filing this bug should help to improve the situation. Would you please post

the bug id when you get around to?


Joe Clarke Tue, 09/09/2008 - 15:11
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

I've filed CSCsu48696 for this. I wrote an experimental patch to test for management IP reachability. It is available from TAC for customers using LMS 3.1.

Actions

This Discussion