This discussion is locked

ASK THE EXPERTS - INTRUSION PREVENTION SYSTEMS

Unanswered Question
Sep 10th, 2010

Welcome to the Cisco Networking  Professionals Ask the Expert conversation. This is an opportunity to get an update on Intrusion Prevention System with Scott Fringer. Scott Fringer is a Technical Assistance Center engineer on the intrusion detection system team in Research Triangle Park, North Carolina. His team supports Cisco's various intrusion detection/prevention sensors, the Cisco IOS IPS feature set, Cisco Security MARS, Cisco Security Manager, Cisco Security Agent, and the Cisco Anomaly Detector/Guard products. Fringer has represented the Technical Assistance Center at previous Networkers conferences and currently holds CCSP certification.

Remember to use the rating system to let Scott know if you have received an adequate response.

Scott might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered   questions in other discussion forums shortly after the event. This  event  lasts through September 24, 2010. Visit this forum often to view  responses  to your questions and the questions of other community  members.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (26 ratings)
Loading.
j.miller_32 Mon, 09/13/2010 - 09:30

My IPS sensor is configured for automatically updating its signatures.  Recently the updates stopped occurring, after working successfully.  How can I correct this?

Scott Fringer Mon, 09/13/2010 - 10:25

Hello;

  There are multiple causes for automatic IPS signature updates to stop functioning.  Two quick CLI commands can help narrow the troubleshooting process.  From the CLI of your sensor, issue the following command:

show version

  This command will allow you to check that the IPS sensor has a valid IPS services license.  Specifically, you will note output similar to the following:

Cisco Intrusion Prevention System, Version 7.0(2)E3

Host:                                                        

    Realm Keys          key1.0                               

Signature Definition:                                        

    Signature Update    S478.0                   2010-05-20  

OS Version:             2.4.30-IDS-smp-bigphys               

Platform:               IPS-4240-K9                          

Serial Number:          JMX00000NS                          

Licensed, expired:      24-May-2010 UTC

  If the license is expired (red text), you will need to work with your Cisco account team or partner to renew the IPS signature support for the IPS sensor.  This will allow you to receive a new license key, which should restore signature update functionality.

  One other culprit that can be verified from this output is that you are running the most recent analysis engine software for the IPS sensor (green text).  Cisco's signature development team writes signatures to the current version of analysis engine.  These signatures will not be compatible with older analysis engine releases.  If you are not at the most recent analysis engine release, you can upgrade the IPS software to correct this issue.

  If the license is not expired and you are running the current analysis engine release, the next command will help determine if there is a credential issue or potential connectivity issue:

show statistics host

  At the very end of the output of this command is a section titled,"Auto Update Statistics", you will be able to see the results of the most recent automatic signature update attempt.  This output may clearly indicate the credentials are invalid, there was a communication issue or there was not currently an update available (confusingly the output is "Success: No installable auto update package found on server").  Corrective action will need to be tailored to this output.

Scott

ROBERTO TACCON Mon, 09/13/2010 - 09:31

Hello Scott,

may I ask you:

1) if the IPS appliances (as other IPS solutions MCAFEE / TIPPING POINT/ ISS )...can drop and/or block the intruders IP without using "external cisco features products" (VLAN maps / ACLs/ shun /...) ?

2) when does tha IPS module for the ASA5505 will support the version 7.x ?

3) when does the IOS IPS will support the IPv6 IPS feature ?

Thanks in advance

Roberto

Scott Fringer Mon, 09/13/2010 - 10:05

Roberto;

  To answer your questions:

1) Yes, Cisco's IPS appliances (and modules) can perform traffic denial actions directly on the sensor when configured for inline operation.  These actions can deny a single packet, just the attacker, the attacker/victim pair, or the connection.

2) Current plans are in place for the AIP-SSC-5 to be supported in the 7.0(5) release of IPS software.

3) I am not currently aware of a time-frame to bring IPv6 support to the IOS IPS feature set.

Thanks,

Scott

mikecrowe4ICS_2 Mon, 09/13/2010 - 17:48

Cisco just recently added AAA authentication support for the IPS.  While this is a good start, it's limited to the CLI and IDM, and only supports RADIUS.

    1. Is there any plan to add TACACS+ support in the future?
    2. Is there any plan to add support for AAA authentication (even RADIUS) for IME?

Thanks for all your help here on the forums, Scott!

Scott Fringer Tue, 09/14/2010 - 12:05

Michael;

  Apologies, it appears my earlier reply via email did not post as expected.

  At this time, I do no have any insight into the planning for implementation of either feature you mention.

Thanks,

Scott

MaseBarnes Tue, 09/14/2010 - 04:50

Why aren't there any plans to support the CSC AND the IPS module for ASA?

I need a complete UTM solution, comparably to Astaro, Watchguard and so on ...

Scott Fringer Tue, 09/14/2010 - 05:20

I cannot provide insight into the decisions made from a product

development standpoint. My role is that of product support.

Scott

hariprasad_n Tue, 09/14/2010 - 10:54

Hello Scott,

Thanks for doing this. My question is related to Global Correlation feature in IPS ver 7.x.

1. Is there a way to tell how many packets/sessions were actually dropped by this feature in say for example last 24 hrs?

2. Identify the related events generated so I can for example find out which internal machine tried to contact a botnet internet IP?

3. Any other reporting function which would actually indicate global correlation is playing a role in dropping malicious traffic?

The only place I see global correlation info is in the actual event generated but I am looking to see if there is a more generic reporting feature.

thanks,

-Hari

Scott Fringer Tue, 09/14/2010 - 11:48

Hari;

  Global correlation brings two methods for responding to potential malicious activity:

  1. global correlation inspection
  2. reputation filtering

  When GC inspection is utilized, the IPS sensor will adjust the risk rating of a firing signature event based on the reputation score of the attacker IP address.  When this action is taken, the details are included in the signature event details.  So, you should be able to discern from the signature event both the GC inspection changes and actions taken by the sensor.  This will be reported on a per-signature event basis.

  When reputation filtering is utilized, there is no corresponding signature event fired when an attacker is denied; the sensor simply

denies the traffic.  You can track the outcome of this activity from the sensor CLI by issuing:

show statistics analysis-engine

  The last section of the analysis engine statistics covers global correlation activity.  It is titled,"GlobalCorrelationStats" and will provide event counts and hosts that were determined as potentially malicious.

  Within the IPS Device Manager GUI (IDM) you can add a gadget to the dashboard which provides a graph/table of the percentage of packets denied due to global correlation.  It will present a segment for "Traditional IPS Detection", "Global Correlation Inspection" and "Reputation Filtering".

Scott

bibhuthi79 Tue, 09/14/2010 - 14:07

Hi Sir,

I have a 3750 series switch. WS-C3750-48TS-E

Wanted to know, does it support routing. Could you please explain the way we can differntiate the different 3750 series switches to support routing.

Could you please provide me the URL to know much about 3750 series switches.

Thanks,

Bibhuthi

Scott Fringer Tue, 09/14/2010 - 14:20

Bibhuthi;

  Unfortunately, the Catalyst 3750 is not my area of expertise.  You can find out all about the Catalyst 3750 series switches at the following link:

http://www.cisco.com/go/3750

  From the initial details on that page, it does appear the Catalyst 3750 supports various IP routing options.

Scott

jzarifyar Wed, 09/15/2010 - 16:59

I need Ip cache flow source and destination. What command would get an output like this on a switch or router:    Source.       Destination.       Packets.       Bytes 10..x.x.x      10..x.x.x              2                  76    Thank you in advance Jay

Posted from my mobile device.

Scott Fringer Thu, 09/16/2010 - 03:40

Jay;

  This is not a topic related to Cisco's IPS devices (my area of focus), and is not a question for which I can provide an answer.

Scott

partner.bkme_2 Thu, 09/16/2010 - 03:02

Hi Scott,

Our team is looking at mitigating the risk pertaining to the TLS cipher renegotiation (http://isc.sans.org/diary.html?storyid=7534 ) through a Cisco IPS (7.0(2) E4 on 4240/4275). Would it be possible for you to shed some light on this subject, is there any signature in particular that would do this job.

We are looking at mitigating this risk on any inbound traffic from the internet to our environment.

Thank You

Scott Fringer Thu, 09/16/2010 - 04:38

Ali;

  There is not a specific signature for Cisco's IPS sensors to detect an exploit of this vulnerability.  Cisco's IntelliShield site (http://www.cisco.com/security) does have a security alert regarding this vulnerability:

http://tools.cisco.com/security/center/viewAlert.x?alertId=19361

  To determine if a custom signature could be created would require capturing network traffic of the vulnerability being exploited and reviewing the captures to determine if there is any indentifiable/recurrent patterns.

  At this time, the best mitigation looks to be implementing the available patches from each vendor.

Scott

lchance Thu, 09/16/2010 - 07:30

Hello Scott,

I’m new to IPS and have a question (it may be dumb, so forgive me).

Is there any way to tie together multiple signatures as a type of compilation event? That is, when I see two separate signatures fire, 5606/0 and 16297/1, which turns out to be when an internal user gets prompted to log in at a DMZ system which is not part of the Windows Domain. I hope that makes sense enough to warrant an answer.

When I get some training under my belt I'll be dangerous...

Thanks for helping!

LC

Scott Fringer Thu, 09/16/2010 - 07:55

LC;

  Not a dumb question at all; and they all warrant an answer.

  It is certainly possible to create a signature event based on the occurrence of two separate signatures firing in a specific order.  To accomplish this, you will need to make use of the meta signature engine.  This engine can combine multiple signatures (meta-components) into a single event when the component signatures fire.  You can find out more about the meta engine here:

http://www.cisco.com/en/US/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html#wp1014660

  So, you can create a new custom signature that is based on the meta engine.  Within the signature definition you would add signatures 5606/0 and 16297/1 as components.  You would then set various requirements such as component ordering, component count, etc to tune the meta signature to fire based on your requirements.

  Good luck with your training and learning - and feel free to come back to the Cisco Support Community with any other questions you may have.

Scott

trippi Fri, 09/17/2010 - 13:53

I want to filter the src IP address on a signature.

Is there a way to say 'not equal to a value'?  such as !10.10.0.0/16.

Thanks

Scott Fringer Fri, 09/17/2010 - 17:46

While you cannot configure an exclusion via negation in the signature definition, you can create an event action filter (EAF) for that specific signature and that specific source (attacker) IP address.  You can then set the actions to subtract for that EAF to not produce an alert, or deny traffic or any other available action.

You can read more about event action filters here:

http://www.cisco.com/en/US/partner/docs/security/ips/7.0/configuration/guide/idm/idm_event_action_rules.html#wp2034816

This should allow you to achieve the same result of ignoring a specific source IP address for the given signature.


Scott

pcoughlin01 Mon, 09/20/2010 - 12:38

Hi, I'm looking for some guidelines on tuning the signatures available on an IPS4240 installed inline on an Internet-edge connection.  I'm seeing that many signatures are enabled by default, and others are not.  Short of looking through each and every signature, are there any guidelines for controlling these, or are they already optimized as received from Cisco?  For example, in sig0 there are a whole bunch of "IOS IPS" signatures enabled.  Are these just for IPS on IOS devices, or are these specific signatures protecting IOS devices.  If just for IOS IPS, should I disable them all?

Thanks,

Pat

Scott Fringer Mon, 09/20/2010 - 12:48

Pat;

  As signature updates are made available, the signature developers work to ensure that currently active threats are able to be detected by the IPS.  To that end, the defaultly enabled signatures are usually sufficient to meet those needs, and stepping through each and every signature may not be necessary.  Though familiarizing yourself with those signatures that are available can be beneficial should you have legacy systems still running in your environment - this would allow you to enable signatures that may exist to detect any legacy exploits.

  In regard to your question regarding noticing a "whole bunch" of IOS IPS signatures; if these were noted by selecting the "IOS IPS" category within IDM - this is simply a logical grouping of signatures.  These "IOS IPS" signatures are those that are able to be implemented on a Cisco router running a supported release of IOS.  These signatures are not limited to use only by IOS IPS, and disabling them will reduce the protection afforded by your IPS sensor.  Signatures can belong to multiple categories.

Thanks,

Scott

j.delossantos Mon, 09/20/2010 - 15:51

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

Hello.

We have a 4240 running inline with software 7.0(4)E4 on our edge.

There have been a few occasions where the IPS has reached 100% Inspection Load and whenever this happens it just drops packets. The interfaces themselves are not saturated because we only have 200M from our ISP.

What troubleshooting tips would you advice so I could drill down on the actual cause of the Inspection Load reaching 100%? When this occurs it usually stays at 100% for 5to10minutes.

Thank you.

-

Jun

Scott Fringer Mon, 09/20/2010 - 16:53

Jun;

  The first thing to keep in mind is that the IPS-4240 is rated up to 250 Mbps throughput - traffic composition and sensor configuration can lower this rate.  A potential configuration impact to sensor performance can come from excessive signature tuning/enabling of all signatures.

  While the interfaces may not be processing at line rate, you can see if the sensor is over-subscribed by checking the interface statistics:

sh interface

   The key indicators to check would be on the interfaces assigned to your inline pair; of interest are:

Total Receive Errors = 4215

Total Receive FIFO Overruns = 4240

  If these values are non-zero, an issue has occurred where the sensor could not process packets as quickly as they were arriving, and in turn those packets were dropped.  Rapid traffic spikes while the sensor is already at a high load can also lead to this behavior.
  There is also a higher processing price for UDP packets as compared to TCP packets.  If your traffic mix is more frequently UDP-based, this could cause an issue.  You can see the rate of TCP and UDP packets processed by the sensor by reviewing the virtual sensor statistics:
sh stat virt 
  The two counters to look for are:
Total TCP packets processed since reset = 1297446
Total UDP packets processed since reset = 5534273
  You will want to take these values in the context of total traffic processed by the sensor and how long it has been since the sensor statistics were cleared.  Again, from the same output, look for:
Total packets processed since reset = 18898040
Total IP packets processed since reset = 10958049
and:
Number of seconds since a reset of the statistics = 509614
  You can see if there is excessive alerting due to signature tuning by reviewing the event store statistics:
sh stat event-store
  You want to look for the number of times the event store has wrapped:
The number of times the event store circular buffer has wrapped = 5
  You will want to key on whether the rapid event rate is due to error events (sensor based issues) or alert events (firing signatures).  If the rate is high due to error events, you will want to ensure configuration of items such as blocking hosts, global correlation and IPS automatic signature updates are correct.  If the rate is high due to alert events, you will want to review any changes you have made to signatures from defaults.  You can narrow in on offending signatures by again reviewing the virtual sensor statistics, this time keying on the section, which will look similar to:
Per-Signature SigEvent count since reset
    Sig 1101.0 = 6
    Sig 1306.0 = 10
    Sig 2000.0 = 5041
    Sig 2004.0 = 24999
  Signatures with high event counts should be checked.
  You can also review the logs provided in the output of the 'sh tech' command to see how frequently the event store is wrapping.  You will want to search for messages similar to:
17Sep2010 10:11:36.389 18.139 sensorApp[841] IdsEventStore/W errWarning - the event store wrapped around [IdsEventStore::writeEvent(), index = 58590]
  These are just a few tell-tale signs of sensor over-subscription that can result in dropped packets when the sensor is configured for inline operation.
Scott
j.delossantos Tue, 09/21/2010 - 08:43

Thank you!

That was very helpful. The event store has wrapped 3 times in 17days. It seems like I need to do a lot more tuning or maybe disable some of the low risk signature.

I do have a TAC case open on this. The engineer advised to clear the stats while the issue is occurring and collect the stats after.

I'll post an update.

Thanks!

pcoughlin01 Tue, 09/21/2010 - 07:29

Hi Scott, can you please explain the data archiving functionailty within IME.  I'm interested in details about maintaining past events, disk space management, and historical log file retention.  Can the archive files be moved off the appliance, and if so, can they be re-imported at a later date, say for running reports?  Any information about working with the events archive is appreciated, similar to how you would maintain old syslogs in case you needed to go back in time and research something.  Any commands used to monitor disk space/log file size is appreciated too.

Thanks,

Pat

Scott Fringer Tue, 09/21/2010 - 10:05

Pat;

  The data archiving facility within IPS Manager Express (IME) is provided as a method to maintain performance of the IME system.  IME stores events in a local MySQL database.  The default storage is to store 1,000,000 events in 100 data files.  You can adjust these values to better meet your storage needs; events per file can be configured to store between 1,000 and 1,000,000 events and the number of data files can range from 10 to 400.

  There is no direct space monitoring/management functionality in the IME GUI.  You can monitor system disk space via standard Windows tools.  The data files are stored in:

\Program Files\Cisco Systems\Cisco IPS Manager Express\MYSQL\data\alarmDB

  You can monitor this directly to see the direct impact of IME on the system's disk space.  Standard Windows Explorer detail view should allow you to see the current file size of each archive file.

  There is no functionality present to support moving the archive data from the IME system and restoring it at a later time.  As long as the data files remain on the IME server, you can query for any date range that is currently stored in those files.  If you need further protection for accessing data at a date that may be removed from the system, you may want to consider backing up the IME installation on a regular basis for restore to another system for archive review.  Please note that this is only a possible solution, not fully tested.

Scott

What is the best way to configure IOS IPS to take action on the numerous viruses/malware/Trojans that are prevalent on any network at any given time?  I have my routers configured with the IOS IPS to notify SDEE but how can I know that the signatures that are enabled are applicable for worms, Trojans, etc?  How do I verify that I truly have the signatures enabled to alert me or any event relating to the IOS IPS?

Scott Fringer Tue, 09/21/2010 - 10:54

David;

  The most intuitive method for managing IPS signatures on a Cisco router is through the use of the Cisco Configuration Professional (CCP).  This free GUI application allows you to fully manage and tune individual IPS signatures that are enabled on your IOS-based router (along with other router configuration options).  CCP should be able to provide you a view of the installed signatures grouped by categories such as attack, dos, ddos, etc.  You can find out more about CCP and download it here:

http://www.cisco.com/go/ccp

  You can also make use of similar commands from the router CLI to see details of active signatures and other operational states.  Some commands of interest would be:

show ip ips signature-category

show ip ips signatures

  Based on the output of this command, you should be able to determine the enabled signatures for each category.

Scott

exploit_haxor Tue, 09/21/2010 - 11:28

Hi Scott,

              I am new to ips and i want to write some custom signatures, i was looking for some beginner (easy to understand) document which can tell more about the signature engine and regex?.........is there any reading material or documentation that you can point me to so that i can understand the signature engine and write custom signatures?.....any suggestion would be helpful

Thanks

Scott Fringer Tue, 09/21/2010 - 11:40

The best place to start with understanding custom signature development for Cisco's IPS platform is to understand the system's architecture.  There is a good overview available at this link:

http://www.cisco.com/en/US/docs/security/ips/7.0/configuration/guide/cli/cli_system_architecture.html

The next piece to understand is the functionality the various signature engines available provide.  This is outlined at this link:

http://www.cisco.com/en/US/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html

With the underlying technology in place, you can start with custom signature creation as discussed at this link:

http://www.cisco.com/en/US/docs/security/ips/7.0/configuration/guide/cli/cli_signature_definitions.html#wp1042406

(Or for the IDM GUI):

http://www.cisco.com/en/US/docs/security/ips/7.0/configuration/guide/idm/idm_signature_wizard.html#wp2117436

As for references on regular expressions, there are a multitude of resources available on the Internet.  There is a brief overview within the IPS user guide at this link:

http://www.cisco.com/en/US/docs/security/ips/7.0/configuration/guide/cli/cli_signature_engines.html#wpmkr1215311

Scott

exploit_haxor Wed, 09/22/2010 - 02:21

Hi Scott,

               Thanks a lot for your help....i had another question,  in the service http engine if  i specify "uri regex"  and  "header regex".....will the signature trigger only when it matches both uri and header regex or will it trigger if either one of them match?

Scott Fringer Wed, 09/22/2010 - 04:32

Within a signature all enabled and defined regex values must match for the signature to match and in turn take action.  So, if you want a signature to fire whether either the header regex or the URL regex matches, you would want to create two separate signatures; one for each specific match.

Scott

exploit_haxor Wed, 09/22/2010 - 11:23

Hi Scott,

                we have around 15 cisco ips devices running IPS 7.0(4)E4, sometimes the "analysis engine" of some of the sensors stops running (i.e anlaysis engine NOT running)......what is the cause of this? and what are the steps or solution to solve this problem so that analysis engine starts running normally?

Thanks

Scott Fringer Wed, 09/22/2010 - 11:29

Unfortunately there is not a single cause or corrective action for times when the analysis engine stops running.  When this occurs, the best thing you can do is gather a 'show tech' from the affected sensor and open a service request with TAC.  It will take specific investigation on a case-by-case basis.

Scott

Actions

This Discussion

Related Content