Device specified as unreachable is not queried by server running CS discovery. A single LMS 3.1 server discoviering via CDP, with seed devies, and two globaly defined community strings. Community strings verified through successful discovery of other devices sharing same community and manual mib browse from server using 3rd parth browser. Mib walk using Trouble Shooting>Device Center>MIB Walk fails. Device is eventual found and named on a different interface. Are there controls for directing discover to particular interfaces, and why would discover declare a device unreachable when no packets are sent to the IP address. Thanks
Please post your NMSROOT/conf/csdiscovery/CSDiscovery-config.xml. As for being able to direct Discovery on which management address to pick, this is decided based on the Preferred DCR Display Name option. For example, you can choose to use the highest loopback IP address as the management address.
I've tried the various global options with seemingly the same results. I just don't get why in this one device I'm focusing on I don't get any packets sent to the IP that's the DNS resolveable IP, and instead the device is discovered on another IP. See attached discover config.
This is how Discovery is configured. You've setup an exclude filter such that everything will be discovered EXCEPT devices which are found to have an interface in 172.*.*.*. You do NOT want to cross any router boundaries. You have a few seed devices, but nothing connected to these devices will be discovered that is more than 2 CDP hops away.
That said, what is the device on which you are focusing? What IP addresses does it have? How is it connected to your seed devices?
It is one of the seed devices 188.8.131.52. In previous attemps to discover it I used it's DNS resolvable address 184.108.40.206. It also has addresses of 220.127.116.11, 18.104.22.168 and 22.214.171.124. - I get the same behavior if I use the DNS IP 61.2 for the seed device, that is unreachable device with no packets sent, and then the device found on 61.114.
Go to Common Services > Device and Credentials > Device Discovery > Discovery Logging Configuration, and enable debugging for the System Module, Neighbor Module, Discovery Framework, and Discovery Deviceinfo, then run a new discovery, and post the ngdiscovery.log.
This is what I see:
c6509r-srv.syr.edu/126.96.36.199 is filtered since it has the IP address 172.30.4.1 configured.
backbone.syr.edu/188.8.131.52 is properly discovered.
184.108.40.206 cannot be discovered due to SNMP timeout on system.
220.127.116.11 cannot be discovered due to SNMP timeout on system.
18.104.22.168 is filtered because it has the IP address 172.20.38.3.
22.214.171.124 is filtered because it has the IP address 172.18.17.1.
126.96.36.199 is filtered because it has the IP address 172.16.19.1.
188.8.131.52 cannot be discovered due to SNMP timeout on system.
184.108.40.206 cannot be discovered due to SNMP timeout on system.
220.127.116.11 cannot be discovered due to SNMP timeout on system.
18.104.22.168 is filtered since it has the IP address 172.20.2.1.
backboneb.syr.edu/22.214.171.124 is properly discovered.
126.96.36.199 is filtered since it contains the IP address 172.20.38.3.
CMbone.syr.edu/188.8.131.52 is properly discovered.
184.108.40.206 is filtered since it has the IP address 172.16.19.1.
220.127.116.11 is properly discovered.
18.104.22.168 cannot be discovered due to SNMP timeout on system.
humerus2.syr.edu/22.214.171.124 is properly discovered.
126.96.36.199 cannot be discovered due to SNMP timeout on system.
188.8.131.52 cannot be discovered due to SNMP timeout on system.
184.108.40.206 cannot be discovered due to SNMP timeout on system.
220.127.116.11 cannot be discovered due to SNMP timeout on system.
18.104.22.168 cannot be discovered due to SNMP timeout on system.
22.214.171.124 cannot be discovered due to SNMP timeout on system.
126.96.36.199 cannot be discovered due to SNMP timeout on system.
10.21.0.137 cannot be discovered due to SNMP timeout on system.
10.21.1.107 cannot be discovered due to SNMP timeout on system.
10.21.8.37 cannot be discovered due to SNMP timeout on system.
10.21.1.169 cannot be discovered due to SNMP timeout on system.
10.21.1.170 cannot be discovered due to SNMP timeout on system.
Your test device, backboneb/188.8.131.52 is properly discovered, and 184.108.40.206 is used as the management IP. This device should now be in DCR.
Yes it is discovered, but it is initially reported as being unreachable without having sent a packet out. It appears to be timing out within the system. This isn't a problem for devices that have multiple interfaces and for whatever reason are eventually discovered. However the concern is what will happen when discovering other single IP addressed devices? It's a reliability concern ya know what I mean?
I don't see any indication of that in the logs. This device is properly discovered with no hint of unreachability. If you can reproduce this, capture a screenshot illustrating what you are seeing with a new ngdiscovery.log.
I should also add that this backbone device is added as backboneb.syr.edu/220.127.116.11 based on the preferred management address scheme. Future management operations will use that IP address.
I run a Wireshark capture on all the addresses on this switch, and get no packets. I see the Rimoute when quering system talbe from deivce, msg in the log, so does that usually mean a packet has been sent? FWIW I ran the discover with only the 61.2 device as a seed and the discovery failed. log attached
I see an SNMP timeout on 61.2. This would indicate that Discovery sent an SNMP request for the system branch. If you're not seeing any packets in wireshark, my guess would be you're looking at the wrong interface, or the filter is too strict. Exactly what filter did you use?
I suppose another explanation would be that this server doesn't have a route to 61.2, and thus the server cannot even encapsulate a packet to that address.
My concerns exactly so I ran a mib browse to each interface from the server and captured all the packets to and from. Now there are two interfaces on the server, but only one is linked. Since the mib browser traffic is captured the assumption is discovery would use the same one and in the earlier attempts with multiple seeds it did find the device. Something is flakey.
I'd like to see your wireshark rules because the log gives every indication that SNMP is timing out (which implies at least one packet is sent to the device).
"ip.addr == 18.104.22.168 || ip.addr == 22.214.171.124 || ip.addr ..." etc.
The rule works for every address specificed when I MIB browse the target device from the CS server. I don't think it's the server, ACL, firewalls, etc. But I will say that in my experience when something gets this arcane the problem is usually something simple. I'm just not seeing it.
You could connect to the device in question, and enable debug snmp packet. This would indicate whether or not any SNMP is arriving from the server. Also, do you have Cisco Security Agent installed on this server?
What about the basics? Can you ping this IP from the server? In your first post, you mentioned SNMP Walk from Device Center is not working. Is this still the case?
There is some additional debugging you can enable which may also help narrow down the problem. Enable Data Collector debugging, and run a new discovery for this device. Post the new ngdiscovery.log.
Ran it again, for just this one device and after having tried the defined community strings, and failing the log has this line:
"ug 21 12:42:45] INFO [LoggingOutputStream : Thread-2 flush] : Io error while closing SnmpSocket : Cannot assign requested address: Datagram send failed"
Does this square with not seeing any packets leaving the NIC?
This is a standard error everyone typically sees at the end of Discovery. It doesn't point to any specific problem. However, the whole log may be useful. In particular, a timeout error will be thrown for more problems than just a timeout. The extra debugging should pinpoint exactly why the timeout is seen.
The error points to a standard SNMP timeout, and not any other exotic communication problem. At least six packets should have been sent (three for each Discovery community string). The only thing that would prevent them from going out is a missing route on the server (and no default route defined) or something like the Windows Firewall Service blocking the packet.
I figured it out. Turns out, you have a service request opened for this issue which I was working on. You mistyped your device address in Discovery. The device you want is 126.96.36.199. The device you added is 188.8.131.52. You're missing a 0. That explains why your sniffer trace shows no packets, and why the device is unreachable.
It wasn't until I actually saw your sniffer trace that I noticed this.
Thats it! Not only had I mis-typed that address, but there were other devices in the seed file simularly mis-typed. When the problem gets arcane, it's usually the simple. Thanks for your help on this