LMS 4.0.1 and User tracking with SNMP v3

[csireg000]

Hi! (again )

I've another problem with our new LMS 4.0.1.

We manage our devices with SNMP v3 but the user tracking don't want to work flawlessly.

I've attached an example from our SNMP configuration. Basicly it's the same in our devices.

1st the problem was that no matter what I did the User tracking didn't want to find any host. I left it and worked on something else. After 2 weeks suddenly appeard couple of thousand end host.

As earlier (LMS 2.6 or 3.2 with snmp v2) it is the same that LMS cannot differentiate normal end host and IP Phones although we have several thousand from both. But this is only one problem.

The other is that there are switches with the same IOS and SNMP configuration and from one I get the UT data and from another one I didn't get anything. Only from some 4506 (aprox. 12-15) and 6506 (2) works and we have 20+ 4506 and 10+ 6506. Not to mention the other switches (couple of houndred 2960 and 3750).

I'll be grateful if somebody could advice what to do.

Thanks

Gabor

[Michel Hegeraat]

First, you can only see end host MAC addresses in these VLAN's

snmp-server group CWK v3 auth context vlan-2101

snmp-server group CWK v3 auth context vlan-2110

snmp-server group CWK v3 auth context vlan-2202

snmp-server group CWK v3 auth context vlan-2801

snmp-server group CWK v3 auth context vlan-2802

snmp-server group CWK v3 auth context vlan-2901

I assume that is OK.

Next if any IP address needs to be connected to those MAC addresses then the default gateway needs to be a managed cisco router.

Next if any hostname needs to be connected to those IP addresses then the LMS server has to be able to resolve those IP's. Use CSCOpx/bin/resolver.pl to see if it can.

So if you don't get any MAC address, problem is probably on the switch.

If you don't get any IP addresses, problem is probably with the default gateway.

If you don't get any hostnames, problem is probably the DNS of the LMS.

Cheers,

Michel

[csireg000]

Thanks for the response, Michel.

I think it's better if I separate the things. Take we look at only forexample 4506 switches.

I have switches with working UT (MAC, IP as well as DNS names) and switches where UT not works (not even MACs). Every 4506 switch has almost the same config (differences are: IP subnets, VLAN numbering, hostnames), acting as L3 switches so they are default gateways also.

So based on your post I think I have problem somewhere around the switch but I just can't figure it out what.

[Michel Hegeraat]

Best focus on one host on that switch Gabor.

Its VLAN is in the context?

snmp-server group CWK v3 auth context vlan-

leave a ping runing to ping every 4 min.

That way the host is always in the show mac-address-table !  (UT is not finding silent hosts, but most devices are anouncing themselves these days)

What is the default gateway of the host?

Run a show arp on the default gateway.

Next step would be to obtain these same tables via SNMP. but I don't recall the OID's right now

LMS has a cli to do snmpwalks as well in CSCSpx\objects\jt\bin

Cheers,

Michel

[csireg000]

VLAN context is ok.

On the switch I choose we have this config:

snmp-server group CWK v3 auth context vlan-1571

snmp-server group CWK v3 auth context vlan-1771

snmp-server group CWK v3 auth context vlan-2831

Forexample the VLAN1571 contains aprox. 200 hosts. The "sh mac-address-table" in the switch shows every host. The "sh arp" (sh ip arp vrf xxx) shows all of the IP addresses from this VLAN.

The hosts have this VLAN IP address as default gateway.

I'll check the SNMP table later, I have to do some research to find the correct OID.

Thanks for the help so far.

[Michel Hegeraat]

FYI, I thinik it is this

Arp table  .1.3.6.1.2.1.3.1

Mac table .1.3.6.1.2.1.17.4.3.1.1

Don't forget for the later you need to give the vlan-id as context.

Also you can try to debug the UT:

Under: Admin  > System >  Debug Settings > User Tracking Server  select module "usertracking" and zoom in on one device by specifying its IP address.

Cheers,

Michel

[csireg000]

Now I have checked the OIDs you wrote.

ARP table is ok, MAC table says "No such instance" and not only in this device, but in all of our devices.

What makes everything more interesting is that today morning I looked at the User Tracking Summary window and saw that nothing has changed since last friday. Ok then let's try to debug. I ran an acquisition on the device i picked last friday. After 5-10 minutes i saw that the host entries number has increased. (Remember, all the devices were in the inventory 4-6 weeks now and nothing on the devices or in the ciscoworks was changed) Okay, let's have a look to the details. Now on half of the new entries are from the switch I ran the acquisition and the other half is from another switch. Weird.

But still there are hundreds of switches from I don't get anything. (Have all the same snmp config, correct vlan-id context, etc.)

Oh yes one other thing. The older entries in the UT didn't get any update. Once they are in, the last seen column does not change. It's like they never been refreshed.

So in summary I still have the following problems:

- UT doesn't provide infromation from all switches.

- Last seen information doesn't get any update.

- IP Phone and normal end host isn't differentiated.

[Michel Hegeraat]

There is a utility that should help debuging these issue.

Check this out.

http://www.cisco.com/en/US/partner/docs/net_mgmt/ciscoworks_lan_management_solution/4.0/user/guide/admin/appendixcli.html#wp1041653

Cheers,

Michel

[csireg000]

Unfortunately I don't have rights to this link. Probably because we don't have smartnet for CWK only shared support.

But I'll try to ask somebody who have rights to it.

[Michel Hegeraat]

Understanding Debugger Utility

The utility displays a report on the reasons why User Tracking failed to discover end hosts on specific ports.

In many cases, User Tracking may not perform as expected. This may be  because of problems in other LMS applications. For instance LMS Server  may have devices that are not discovered or inadequate VLAN discovery in  Topology Services.

You can run the utility to troubleshoot problems, or provide the report  and log generated by the utility when you contact TAC for help in  diagnosing problems.

The debugger utility uses the data collected by LMS Server and reports the reasons for the missing ports in User Tracking.

This tool also has an SNMP component embedded which runs an SNMP query  for the table as a part of verification for SNMP failure. For example,  SNMP bugs in Catalyst operating system because of which User Tracking  may fail to discover devices.

This generates an Action Report that you can use to analyze the data.

The Debugger Utility:

1. Checks the switch ports in a sequential order.

2. Reports violation of basic rules for each of the missing ports such as link ports and trunk ports.

3. Checks for SNMP retrieval of data, if the ports pass the validity check.

4. Generates an Action Report suggesting possible remedial actions to retrieve the valid missing ports.

Using Debugger Utility

The Debugger Utility is available at $NMSROOT/campus/bin/ (where $NMSROOT is the directory where you have installed CiscoWorks).

To run the Debugger Utility, run the command:

utdebug -switch switch-ip -port port1[,port2 ...] [-export filename]

where,

switch is the switch to which the end hosts are connected.

ports are the ports on the switch which have missing end hosts User Tracking.

-export filename specifies  that the debug messages be stored in the file specified. If this option  is not used, the messages are displayed on the console.

For example,

utdebug -switch 10.29.6.12 -port 5/12

utdebug -switch 10.29.100.10 -port Fa0/10

utdebug -switch 10.29.6.14 -port Gi6

Pretty sure you will find this and perhaps more in the build in help of LMS

Cheers,

Michel

[csireg000]

Hi Michel,

I didn't had time lately but now I was able to check UT again.

I found that some 6506 device provide and some not UT.

SNMP configuration, IOS version, Supervisor Engine are the same.

On the switches from which I don't receive UT data I ran the Debugger Utility which says from the ports (standard access ports) that they are routed ports. But they aren't. Standard access ports from I can collect MAC and IP information through CLI with the commands: sh mac-address-table and sh ip arp.

I don't have any idea why the ports are in this state. Rediscovery or complete delete and re-add the device doesn't help.