Port-Security Email Notifications LMS 4.0?

[jsconners72]

I have been reading as much as I can get my hands on regarding setup of LMS 4.0 to get SNMP Traps or Syslog messages to my LAN admins via email...I am still stumped!

All I want is a clear email notification that device X Port XX was tripped.

I'm confused about just how LMS handles SNMP traps or Syslog messages sent from a client switch (for instance a 2960/48 runnning 12.2(25)SED)

Here is what I have done on the switch based on Cisco LMS documentation found here:

http://www.cisco.com/en/US/docs/net_mgmt/ciscoworks_lan_management_solution/4.0/user/guide/admin/useNotif.html#wp1073644

I setup port security successfully on a test switch and confirmed it works. At first I just put "snmp-server enable traps" and this populated my switch config with all of the default traps which was not what I was after, but I configured the email notification on the LMS server and succesfully generated 2 cryptic trap messages from the switch which apparently had nothing to do with port-security directly....but at least it proved that the switch was sending traps to the server and the server was sending emails to me. I then tried to tune it to only send port-security traps using the below commands (ip address obscured):

snmp-server enable traps port-security

snmp-server community ****** RW

snmp-server community ****** RO

snmp-server host X.X.X.X ******

I researched the "port-security" part of this snmp-server command and some some discussion that it had been rescinded??? I also checked the Cisco Command Lookup for this option and it didn't appear valid????:

https://tools.cisco.com/Support/CLILookup/cltSearchAction.do

That didn't generate anything when we reset the port and tripped it again. I then saw some advice saying that the syslog server in the switch should generate a message specific to port security and tried changing to this command (see below).

snmp-server enable traps syslog

Again...that didn't generate anything when we reset the port and tripped it again. I really don't get how Syslog is handled in LMS anyway...I see the parts about receiving SNMP messages from client switches and sending emails but not syslog...it appears to only work if you setup syslog "polling" which is pull and not push (what we need to be proactive).

I have seen several post asking the same questions, most are abandoned or get back answers like "go to this link and read this 1000 page manual to learn about protocol x"

Does someone have a simplified method to accomplish port-security to email notification in LMS 4.0?

[Michel Hegeraat]

LMS does do much with traps. Only the DFM part takes some traps but it won't do what you want here..

With syslog you can do a little more.

Under admin -> Netwotk -> Notification and Action -> Syslog automated actions, you can create an action that listens for a specific message.

if the description contains '*err-disable*' you can raise an action where it sends a mail

Cheers,

Michel

[jsconners72]

Thanks Michel,

I have a test switch logging syslog to the LMS server. I confirmed receipt of messages in the NMSRoot\...syslog.log file. However...I've tried looking everywhere and syslog doesn't display in the LMS web interface...does this mean there is additional syslog config needing to be done on the LMS server first?

I think I understand that "polling" is usually how the syslog entries get populated in the LMS syslog sections by device...what about client pushed logging entries?

I did "Create -> Choose device -> Named the Action ->"...now it wants me to "Define Message Type", where does "err-disable" go? The options are Facility,Sub-Facility,Severity,Mnemonic and Description in the dialogue box.

Is there someplace with clear examples of how to decode syslog messages for use in these fields?

Thanks again!

[Michel Hegeraat]

What is relevant is that the switch sends the syslog with the same IP LMS uses to manage the switch.

And of course LMS looks only at syslog from managed devices.

There is no polling, LMS will act when messages are received.

The "err-disable" goes in the description, the rest is *

Cheers

Michel

[jsconners72]

Thanks again Michel!

I set this up as specified...I only placed the text "err-disable" (without quotes or wildcards) into the "description" field, tripped the port but I did not get an alert email.

So if I am seeing entries in the LMS servers's NMSRoot/...syslog.log file then it is configured properly? Why can't I see these in the web interface?...I can only see them in the raw log file.

I think I'm still missing some local server syslog configuration somehow...I would assume I would see these in the web interface no?

[Michel Hegeraat]

Where do you look in the GUI?

did you look in http://didata:1741/dfm/eventsbrowser/Main/eventmon_wrapper.jsp?navid=EventMonitor under syslog?

Cheers,

Michel

[jsconners72]

Thanks again!...

Yes I did look there. it says "No syslogs are available" in the Syslogs details area, even though I can use Notepad to see the entries in NMSRoot\log\syslog.log...do I need to activate something for collection from the local host? I also selected "all devices" in the "Device Selector" and clicked "View" button then chose "Show Events"-> "Last 1 day" and refreshed the Sylog pane, nothing at all shows up.

I also tried running a 5 day Syslog report from this screen with all devices selected with no results.

[Michel Hegeraat]

Assuming the IP addresses used to send the syslog, matches the management IP address in LMS, LMS should act upon these messages, so if you syslog reports remain empty then something is not working right.

I think under Admin -> System  you find a portlet call syslog collect status. This should tell you how many messages were received an how many LMS could take in for processing. Can you post what it says there

Cheers,

Michel

[jsconners72]

Michel thanks again!

Here is a copy of the "Collector Status" from Admin->Collection Settings->Syslog->Syslog Collector Status:

Name   Forwarded   Invalid   Filtered   Dropped   Received   Up Time   Update Time 

CWSERVER01 0 2304 33 69 2407 Mar 22 2012 09:56:18 British Summer Time(GMT +01:00:00) Mar 27 2012 09:02:03 British Summer Time(GMT +01:00:00)

CWSERVER01 is the server name where LMS is installed.

I ran the option "Test Collector Subscription" and I got this:

Test Collector Subscription Status

SSL certificate status   SSL certificates are valid and properly imported

Collector status   Collector CWSERVER01 is up and reachable.

I just ran a report from

Monitor> Monitoring Tools> Event Monitor and I still don't see any logs from the switch there.

These are the latest switch entries from \\CWSERVER01\c$\Program Files (x86)\CSCOpx\log\syslog.log proving that the switch (X.X.X.X) is shipping the logs to the server:

...

Mar 27 01:01:31 X.X.X.X 3955: Mar 27 00:01:23: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down

Mar 27 01:01:31 X.X.X.X 3956: Mar 27 00:01:24: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to down

Mar 27 01:01:31 X.X.X.X 3957: Mar 27 00:01:27: %LINK-3-UPDOWN: Interface FastEthernet0/2, changed state to up

Mar 27 01:01:31 X.X.X.X 3958: Mar 27 00:01:28: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

Mar 27 01:01:52 X.X.X.X 3959: Mar 27 00:01:46: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to down

Mar 27 01:01:52 X.X.X.X 3960: Mar 27 00:01:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/2, changed state to up

....

There are other entries from managed devices as well but this is the only switch I am sending syslogs from directly to the server.

...I'm stumped!

[Michel Hegeraat]

if you now export you DCR devices to CSV text, and then find the name of the switch in that file.

Is the switch IP address, the X.X.X.X you see in the syslog file?

For instance I see thi in my syslog file:

Mar 27 13:07:30 10.170.46.253 103460: 5w0d: %CDP-4-NATIVE_VLAN_MISMATCH: Native VLAN mismatch  discovered on FastEthernet0/19 (46), with AS3 FastEthernet0/10 (555).

And 10.170.46.253 is the management IP of this switch so RME knows about it.

It is vital that it is the same IP address, if not  then the some IP is sending syslogs but LMS doesn't know who that is and  drops the messages.

Can you confirm this?

Cheers,

Michel

[jsconners72]

OK thanks Michel...I used the DCR Export utility to output a .csv report. the "management_ip_address" matches the IP of the switch in my syslog. "host_name" is correct, displayname is correct and everything else seems correct.

Not sure where to go from here

Joe

[Michel Hegeraat]

go here :Admin > System > Debug Settings > Config and Image Management Debugging settings

Enable debug setting for Syslog Analyzer to debug

AnalyzerDebug.logshould tell you what goes wrong

when it works it looks like this.....

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],Preparing to process the PDU from the collector

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],PDU is of type: REQUEST TO PROCESS NEW SYSLOGS

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],Preparing to add new syslogs for processing

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],Added a syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],Added a syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],Added 2 syslog for further processing by SA

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[Thread-13],Added new syslogs to the action processor

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to process new syslogs from the collector

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to find the device id

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to find device id by assuming 10.170.46.253 as address

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Finding device id in the cache

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Device id found in the cache itself as 18

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Found the device id as 18

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to insert the syslog into database

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to find device id by assuming 10.170.46.253 as address

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Finding device id in the cache

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Device id found in the cache itself as 18

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Found the device id as 18

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to insert the syslog into database

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to hand of syslog to the database handler

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Syslog length=2

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4], Time stamp of the syslog received is : Tue Mar 27 16:14:47 CEST 2012 GMT 27 Mar 2012 14:14:47 GMT

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4], Time stamp of the syslog received is : Tue Mar 27 16:15:32 CEST 2012 GMT 27 Mar 2012 14:15:32 GMT

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Inside execute mothod

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Insert into SYSLOG_20120227(Syslog_Device_Id,Syslog_Device_Name,Syslog_TimeStamp,Syslog_Facility,Syslog_SubFacility,Syslog_Severity, Syslog_Mnemonic,Syslog_Description )values(?,?,?,?,?,?,?,?)

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Inside Retry count

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Connection is now false

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Recreated the statement object

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Row count 2

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Added syslog to the database handler

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Insertion of syslog into database is done

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to find interested actions, bypassing

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Device List 1* syslog is CDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/18(46),withGOLDLAB_IPT_3524FastEthernet0/24(1).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 2

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 7

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],message heheCDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/18(46),withGOLDLAB_IPT_3524FastEthernet0/24(1).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 1

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Device List 1* syslog is CDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/18(46),withGOLDLAB_IPT_3524FastEthernet0/24(1).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 2

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 7

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],message heheCDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/18(46),withGOLDLAB_IPT_3524FastEthernet0/24(1).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Found 3 actions on the syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to check with DM to invoke action on the device 18

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],The device already granted by DM

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Statemanagement allowed actions on the syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to hand off the syslog to action handlers

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to hand of syslog to the action processor

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Not a valid action

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Not a valid action

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Not a valid action

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],No actions defined for the syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Syslog handed off to action handlers

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Device List 1* syslog is CDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/19(46),withAS3FastEthernet0/10(555).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 2

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 7

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],message heheCDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/19(46),withAS3FastEthernet0/10(555).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 1

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Device List 1* syslog is CDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/19(46),withAS3FastEthernet0/10(555).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 2

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],inside 7

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],message heheCDP-4-NATIVE_VLAN_MISMATCH:NativeVLANmismatchdiscoveredonFastEthernet0/19(46),withAS3FastEthernet0/10(555).

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Found 3 actions on the syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to check with DM to invoke action on the device 18

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],The device already granted by DM

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Statemanagement allowed actions on the syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Attempting to hand off the syslog to action handlers

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Preparing to hand of syslog to the action processor

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Not a valid action

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Not a valid action

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Not a valid action

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],No actions defined for the syslog

[ Tue Mar 27  16:15:42 CEST 2012 ],DEBUG,[ActionThread4],Syslog handed off to action handlers

[jsconners72]

OK Thank again!...I set both "Syslog Analyzer" and "Syslog Analyzer User Interface" to "Logging Level"> "Debug" ("Info" was the default)

Where do I get this output in syslog.log?

[Michel Hegeraat]

In CSCOpx\logs\AnalyzerDebug.log

Cheers,

Michel

[jsconners72]

Thanks Michel...

Interesting! I see a bunch of these starting 21/3:

[ Wed Mar 21  16:21:09 GMT 2012 ],ERROR,[Thread-14],java.sql.SQLException: JZ0S2: Statement object has already been closed.

[ Wed Mar 21  16:21:10 GMT 2012 ],ERROR,[Thread-14], Unable to initialize logging infrastructure for localization

[ Wed Mar 21  16:21:10 GMT 2012 ],ERROR,[Thread-14],No resource is associated with key "Unable to initialize logging infrastructure for localization".

[ Wed Mar 21  16:21:10 GMT 2012 ],ERROR,[Thread-14],Unable to initialize logging infrastructure for localization

and these with today's date:

[ Wed Mar 28  01:00:02 BST 2012 ],DEBUG,[Timer-4],Issue while calling db view procedure

com.sybase.jdbc2.jdbc.SybSQLException: SQL Anywhere Error -141: Table 'SYSLOG_TODAY' not found

at com.sybase.jdbc2.tds.Tds.processEed(Tds.java:2884)

at com.sybase.jdbc2.tds.Tds.nextResult(Tds.java:2206)

at com.sybase.jdbc2.jdbc.ResultGetter.nextResult(ResultGetter.java:69)

at com.sybase.jdbc2.jdbc.SybStatement.nextResult(SybStatement.java:220)

at com.sybase.jdbc2.jdbc.SybStatement.nextResult(SybStatement.java:203)

at com.sybase.jdbc2.jdbc.SybStatement.executeLoop(SybStatement.java:1766)

at com.sybase.jdbc2.jdbc.SybStatement.execute(SybStatement.java:1758)

at com.sybase.jdbc2.jdbc.SybStatement.execute(SybStatement.java:815)

at com.cisco.nm.rmeng.sa.db.RmeSaDbHandler.createTable(RmeSaDbHandler.java:417)

at com.cisco.nm.rmeng.sa.db.RmeSaTableGenerator.run(RmeSaTableGenerator.java:153)

at java.util.TimerThread.mainLoop(Timer.java:512)

at java.util.TimerThread.run(Timer.java:462)

[ Wed Mar 28  01:00:02 BST 2012 ],DEBUG,[Timer-0],SYSLOG_20120228 could not be createdSQL Anywhere Error -110: Item 'SYSLOG_20120228' already exists Error code 12006 SQL state 52010

com.sybase.jdbc2.jdbc.SybSQLException: SQL Anywhere Error -110: Item 'SYSLOG_20120228' already exists

at com.sybase.jdbc2.tds.Tds.processEed(Tds.java:2884)

at com.sybase.jdbc2.tds.Tds.nextResult(Tds.java:2206)

at com.sybase.jdbc2.jdbc.ResultGetter.nextResult(ResultGetter.java:69)

at com.sybase.jdbc2.jdbc.SybStatement.nextResult(SybStatement.java:220)

at com.sybase.jdbc2.jdbc.SybStatement.nextResult(SybStatement.java:203)

at com.sybase.jdbc2.jdbc.SybStatement.executeLoop(SybStatement.java:1766)

at com.sybase.jdbc2.jdbc.SybStatement.execute(SybStatement.java:1758)

at com.sybase.jdbc2.jdbc.SybStatement.execute(SybStatement.java:815)

at com.cisco.nm.rmeng.sa.db.RmeSaDbHandler.createTable(RmeSaDbHandler.java:385)

at com.cisco.nm.rmeng.sa.db.RmeSaTableGenerator.run(RmeSaTableGenerator.java:153)

at java.util.TimerThread.mainLoop(Timer.java:512)

at java.util.TimerThread.run(Timer.java:462)