Getting Ciscoworks alerts for FCS errors and high Collision Rate %

Unanswered Question
Apr 7th, 2010
User Badges:

I was wondering if it was possible to set up Ciscoworks to notify me via email whenever a port has a high FCS error rate and/or a high collision rate % (over 5% ideally).


I'm currently running LMS2.6.



If this is possible, please explain in detail- I'm afraid I'm rather clueless with how SNMP works.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
jpeterson6 Wed, 04/07/2010 - 12:29
User Badges:

By the way, here is my current configuration on my Cisco switch IOS CLI for SNMP:


logging xxx.xxx.2.240
snmp-server community ******** RO
snmp-server community ******** RW
snmp-server location 2C19A
snmp-server contact TSC (x9149)
snmp-server system-shutdown


Would this be enough for Ciscoworks to get the information it needs? The logging server is the Ciscoworks server IP address.

Joe Clarke Wed, 04/07/2010 - 23:26
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

This is doable by default in DFM.  DFM already has events for high collision rate and high error rate.  As long as your devices are Known in DFM, you should see alerts relating to interface problems pop up under DFM > Alerts and Activities > Alerts and Activities.  When you click on the alert IDs, you will see individual events relating to those error and collision problems.  If you see those, you can configure an email notification subscription under DFM > Notification Services to send you an email when those events are generated.


--


Please support CSC Helps Haiti


https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

jpeterson6 Mon, 04/12/2010 - 09:27
User Badges:

Hi Joe,


Thanks for the reply, but unfortunately I'm not seeing any alerts in DFM for a high collision or error rate. The only alerts I'm seeing involve Unresponsive or OperationallyDown (If I configure DFM to watch specific interfaces).


Could that have something to do with my SNMP settings on the switch side? I know that the high collision and error rates are happening from the CLI displays, but Ciscoworks doesn't seem to see them.

Joe Clarke Mon, 04/12/2010 - 15:50
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Are the ports experiencing these problems managed by DFM?  You can check this under DFM > Device Management > Device Details.  You want the management state of each port to be true.  If you have to change any of the ports, make sure you go to DFM > Configuration > Polling and Thresholds > Apply Changes.


Beyond that, the device would need to be reporting ifInErrors and ifOutError increases as well as possibly dot3StatsFCSErrors, dot3StatsInternalMacReceiveErrors, dot3StatsAlignmentErrors, and dot3StatsFrameTooLongs.  You would also need to make sure the the error rate is greater than the default threshold of 10% (based on the current overall utilization).  If you need to modify this threshold, that can be done under DFM > Configuration > Polling and Thresholds > Managing Thresholds.

jpeterson6 Tue, 04/13/2010 - 07:21
User Badges:

So just to clarify; In order to see any error alerts via email from DFM, I need to manually set every port to 'True' that I manage within the campus under Device Details? That seems terribly inefficient considering I manage over 4000 physical switchports. There has to be a better way, or else I'm just not understanding clearly. Is this the only way for a switch to use the defined thresholds for error rate/collision/etc, and then ultimately sending me an email whenever the thresholds are triggered?


Regarding ifInErrors, ifOutError,  dot3StatsFCSErrors, dot3StatsInternalMacReceiveErrors,  dot3StatsAlignmentErrors, and dot3StatsFrameTooLongs. Are those SNMP messages? If so, how do I get the switch to report them (If they aren't already, based on my SNMP config a couple posts up)?


Lastly, regarding the threshold settings. If I make a change to the thresholds on 10-100MB Interface Groups, will that apply to Access and Trunk groups as well, or are the three independant? If they are, what does an Interface Group represent if not Access or Trunk ports?


Sorry for the barrage of questions, this is just turning into a more complex issue than I realized at first.

Joe Clarke Tue, 04/13/2010 - 22:34
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Yes, each port must be managed by DFM.  All trunk ports (in DFM parlance, a trunk port is a switch port connected to another DFM-managed device) will be managed by default (i.e. the management state will be True).  For access ports (those switch ports not connected to another DFM-managed device), you will need to manually manage them.  Interfaces are always managed by default.


You can use the DFM bulk management scripts found under NMSROOT/objects/smarts/utils.  Essentially, you'll run the getStateIntPort script to export the current port states.  Then, edit the export file, and set the management state to "EXPLICITLY_MANAGED".  then run the setStateIntPort script to re-read the file, and update the management states.  Finally, run the apply script to save the changes.


The ifInErrors, etc. are SNMP objects which DFM will poll to determine the port error statistics.  You can poll them with the Device Center SNMP Walk tool to test that they are incrementing.


If you want to change the threshold for your ports, you will need to edit the Access Port and/or Trunk Port group.  Interfaces are separate entities.  An interface has a layer 3 address where as ports are layer 2 only.


--


Please support CSC Helps Haiti


https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

jpeterson6 Wed, 04/14/2010 - 07:26
User Badges:

Thanks for clarifying. I have a pretty good idea of what's going on now (I hope).


There's one problem though; I'm not seeing those scripts you mentioned. There's no 'utils' folder under NMSROOT/objects/smarts. I tried running a search for the script you listed but wasn't able to find it. Do I need to download it from somewhere?

Joe Clarke Wed, 04/14/2010 - 09:31
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

Make sure you are on a recent version of DFM 2.0.  I have 2.0.13 on my box, and they exist.  I seem to recall they were added in 2.0.11.

jpeterson6 Wed, 04/14/2010 - 10:13
User Badges:

I thought as much.


I seem to have an issue connecting to the CCO via the Software Update in Common Services, and Cisco doesn't appear to have 2.0 versions of DFM listed on their website anymore.


We've recently ordered LMS3.2 though, so I suppose I can safely assume the script program you mentioned will exist in the 3.0 versions of DFM. I'll just have to wait till then to get things working properly.

jpeterson6 Thu, 04/29/2010 - 12:55
User Badges:

Sorry for the big delay on the response for this. I've been through some interesting issues ever since installing LMS3.2 but I think I'm nearly ready to get back to the original problem now.


I ran the script and that did seem to work.


Unfortunately it seems there's one more issue to sift through. I'm unable to verify that I'm getting the email alerts I need as I'm currently getting alerts every time one of the switchports gets disabled. Normally this would be fine except that when I get over 2000 emails every day just from people shutting down their PCs, it makes me hard to sift through and find actual issues (like the FCS errors and Collision stuff).


I tried creating two Event Sets, one for the switches that have important servers on (where I'd WANT to know if a switchport turns off), and one where I tried disabling the OperationallyDown alert in DFM->Notification Services->Event Sets and assigning every switch to the notification group that uses it (except the mission critical stuff). It seemed like an easy solution, but this didn't seem to solve the issue; I'm still getting OperationallyDown alerts and I'm not sure why.


To try and simplify the problem:


- Event Set A is for mission critical switches. I want to know when these switchports are down obviously. They have their own notification group assigned to those switches to use this Event Set. This one seems to be working the way I want it to, so far.


- Event Set B is for the rest of the switches in the network. I have OperationallyDown unchecked yet I'm still getting alerts everytime one of the access switchports shuts down (not the trunks). I had to disable the email notification for this notification group because my inbox was getting spammed so hard.


Any thoughts? I've also attached two screenshots to verify the configs.

Attachment: 
wildemasth Mon, 05/10/2010 - 21:58
User Badges:

Clear the ticks in Alert Severity and Alert Status. Then it will send the emails according to your event sets.

Joe Clarke Mon, 05/10/2010 - 22:24
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

You're absolutely right.  Thanks for spotting the reply.  I missed the follow-up post.

jpeterson6 Tue, 05/11/2010 - 07:34
User Badges:

Hmm, so those two checkboxes are actually overriding the Event Set rules? That's interesting. I'll check it out and let you know how it goes

Joe Clarke Tue, 05/11/2010 - 21:37
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

The Event Sets only apply to events.  There is no way to filter alerts.  If you check the boxes for alerts, then each alert change will generate a new notificiation regardless of the event set enabled.

Actions

This Discussion