Welcome to the Cisco Networking Professionals Ask the Expert conversation. This is an opportunity to get more information on troubleshooting IPS. with Cisco expert Srinavas Mallu. Srinivas is a senior customer support engineer in High Touch Technical Support (HTTS) within the Technical Assistance Center (TAC). He has a double CCIE in Routing & Switching and Security (CCIE# 8914). Mallu has been in TAC for the past eight years supporting security related products such as PIX, ASA, FWSM, Security on IOS, IPSec, ACS and IDS. He also trains people on his team on security technologies.
Remember to use the rating system to let Srinavas know if you have received an adequate response.
Srinivas might not be able to answer each question due to the volume expected during this event. Our moderators will post many of the unanswered questions in other discussion forums shortly after the event. This event lasts through September 25, 2009. Visit this forum often to view responses to your questions and the questions of other community members.
I want to know the bugs realated to IOS c1841-advipservicesk9-mz.124-12c.bin
As i am facinf some problem within VPN tunnels
There are several bugs, and to narrow down the search, I need a clear cut criteria. What exactly is the problem you are running into. What is the peer routers IOS version?
Is this a VPN issue, or an IPS issue?
Hello, we have ASA 5540 with aip ssm module and its version is 6.1(1)E1. We want to upgrade it to 6.1(1)E3 and
get all of the new signatures which are belong to E3. In order to upgrade, there are three titles in your site which are called
"system upgrades", "system software" and "signature updates".
In system upgrades page, there are two title, "IPS-K9-6.1-1-E3.pkg" and "IPS-engine-E3-req-6.1-1.pkg"
In System software page, there is one title, "IPS-SSM_40-K9-sys-1.1-a-6.1-1-E3.img".
In signature updates page, there are many signatures up to S433.
What is the correct order to upgrade 6.1.1(E1) to 6.1.1(E3) and get the last signature updates?
Now our device's signature version is S329 in E1. How can we go to S433 in E3?
To upgrade your IPS version on the ASA 5540, you need to go to the System upgrades page, and download the "IPS-K9-6.1-1-E3.pkg" file and upgrade the version. The system software page is for the SSM image itself, that goes into the SSM module. The signature update page, has signature updates for various IOS levels, if you need to be on the latest signature update, you can download the latest update. By downloading it, it patches the IOS level on the ASA with the latest signature.
Hope this helps!
Once you download and upgrade to "IPS-K9-6.1-1-E3.pkg", you can then go to the signature update page, and then go to S433 in E3 and download the "IPS-sig-S433-req-E3.pkg" and apply the patch.
When tuning the IPS on our 5520, what is a good recommendation for active rules? Is there baseline setup of rules that should be enabled, ie SIG2158/0 set to alert. Do people find some rules better set to "Deny Attacker Inline" or just log the information?
It really depends on the risk rating for the signature. Here is a good document that talks about the recommended action based on the risk rating and how the risk rating is computed.
Hope this helps!
Forgot to give you the URL. Here it is;
Risk Rating and Threat Rating: Simplify IPS Policy Management
I have following questions
1.i have recently enabled global correlation feature on AIP-SSM , i can able see the information on IME regarding the number of packet blocked by this feature, how can i get more details like attacker ip , victim ip , nature of attack etc..
2. Its abt cisco secuirty manager ,we planning to install security manager 3.3.0 with standard edition license
2.1 Does this verison support IDSM2 and FWSM
2.2 Does it support the ips 7.x Engine
3. When configuring IDSM2 on 6513 with following config sup720,IOS 12.2(18)sfx16 on promiscuous mode how many monitor session (SPAN)can we configure on this
I'll try my best to answer all your questions.
1. Have you configured Network participation, in 'full' mode, which is a requirement to collect the data you are looking for. Here is more information on this;
2.1 Yes. This version supports both IDSM2 and FWSM
2.2 Yes again. CSM 3.3 supports the latest IPS 7.x
3. You can have only two span sessions no matter what the IOS is or what sup you use.
Hope this helps!
Now IPS is configured in partial mode, i will enable full mode and check this out.
Another issue regarding IDSM
Getting lots of syslog message displaying following error
description: Note: /etc/modules.conf is more recent than /lib/modules/2.4.30-IDS-smp-bigphys/modules.dep"
Appname : modprobe
i think this error is due to timestamp issue of linux kernel module configuration file.
how can i correct this.
This is a warning that the modules.conf file, which contains a list of
kernel modules and their command-line arguments, has been modified and
therefore depmod may need to be run if the changes to modules.conf
affect any dependencies between the modules or the location of the
module's binary. Modules.conf can be modified at boot time by the boot
scripts, which updates its timestamp but it will not alter the
dependencies or location of the modules.
To fix this, run depmod in the service account (su to root first).
Hope this helps!
As you told , i have enabled full network participation mode.
but still not getting any details about denied packets.
I can see how many packets are denied due to GC on IME . But not getting any incident detail either on IME or MARS.
please clarify ,what will happen if following two conditions occur.
1. When packets are dropped due to reputation filtering.
2. when packets are dropped due to increase in RR.
You can place the sensor in a global correlation audit mode which will result in events being generated for these instances where the deny actions are currently being performed. To enable this audit mode, just enter the following commands:
As another option, Global Correlation works by adding risk to the event based on the attacker's reputation. Purely as a debugging step, you can
enable logging to display the packets for the HIGHRISK events. In general, having logging enabled impacts performance so you wouldn't want to keep logging enabled by default.
The sh stat ana command will display the global correlation stats on the actions being performed.
Please let me know if this helps in identifying the reputation filtered
Here is a good read;
Hope this helps!
For packets dropped due to reputation filtering try executing "show statistics analysis-engine".
At the bottom of the statistics should be a section on Malicious Site Deny Hit Counts. I think this section will tell you either the IPs or at least Networks being denied by Reputation Filtering and packet counts denied for each.
NOTE: The Reputation Filter denies do not cause alerts to be created. So the only way to get the IPs is to use the command above.
As for when packets are dropped due to an increase in RR by reputation.
There are 2 ways this can happen.
Either the reputation brings the RR up to 90 or higher (would have been below 90 without reputation), and because it is 90 or higher it gets denied by the default event action override for deny-packet-inline.
OR reputation raises the RR (maybe not as high as 90), but high enough that internal to the reputation code it matches a reputation override that adds deny-packet-inline.
The reputation override works similar to the configurable event action overrides.
Except that instead of a default 90-100 for the deny-packet-inline override it has a lower range.
The range is determined by the global-correlation-inspection-influence.
Permissive mode will likely prevent reputation itself from adding a deny (the default override might, but the reputation code itself is less likely to).
Standard mode which is the default and where quite a bit of alert from bad reputation addresses wind up with deny action added.
Aggressive mode which lowers the range even more and adds deny actions to a larger range of alert risk ratings.
In order to determine if reputation itself denied a packet look in IME for the following:
Filter for alerts that have negative Reputation.
Look for alert with either a drop or deny action listed.
And look at their Risk Rating.
If the Risk Rating is 90 or higher then the default event action override (assuming you still have it enabled) is likely the cause.
If the Risk Rating is less than 90 (assuming you haven't change the default event action override) then it is likely denied by reputation itself.
In either case you can select the alert itself, and click Show All Details.
At the bottom of the alert it will tell you the reputation information, and if reputation added a deny action.
Hello Srinavas. We are using 4 IPS 4270 in our network for over a year now. We have realized that the operating system is very buggy and several times we have found out the IPS went into bypass mode because the analysis engine stopped working and other times the IPS freezed and don't let us even use the console.
Because of that we've opened several cases in Cisco TAC, they suggest an upgrade, we perform the upgrade, but a couple of weeks later comes another issue, Cisco TAC suggest another upgrade... and so on, we have performed three or four upgrades throughout this year and now we're using 6.1(3)E3.
However currently we have a new issue. I opened Cisco case 612459443 because the AnalysisEngine stopped (yet another time) and this is the error message:
getAnalysisEngineStatistics : ct-sensorApp.460 not responding, please check system
processes - The connect to the specified Io::ClientPipe failed
The engineer assigned to the case told me this is likely to be a new issue and he's awaiting the response of the BU to give me a formal answer.
What can we do? There are no further upgrades in train 6.1 and we don't think an upgrade to 6.2 o 7.0 is a solution because those are really new operating systems and it's likely they're even more buggy.
We have really lots of Cisco equipment, and very different technologies and all of them are stable. All but the IPS.
Also, I would like to know what the number "460" means in the error "getAnalysisEngineStatistics : ct-sensorApp.460 not responding" cause we have seen several similar errors but with different numbers, for example earlier this year we got the message "getAnalysisEngineStatistics : ct-sensorApp.397 not responding". So I guess the number "460" and "397" means something relevant and we currently don't know what those numbers are up to.
I see your frustration, but please be patient. The fact that you have a SR open means you'll get a resolution as a workaround or a bug fix. I just peaked into the SR, and it seems like the engineer got an update from the development team. Just ask him for an update, as you are working with him.
The development team only has access to tools which help them decipher these log codes. We normally send the error messages to them. It normally refers to a piece in the code which it is executing at that time.
Clearly communicate to the engineer who are working with, what your constraints are, and he can give you options which will work for you under your present circumstances.
Eduardoaliaga, we had the exact same problems. Since moving our IPS modules to version 7 they have been rock solid. You might want to consider upgrading.
I'll 2nd that - I have 15 AIP-SSM sensors and on 6.x was getting random lockups and application fails. Since going to 7.x things have been stable and haven't seen any hangs.
I'm pretty sure 7.0 is just 6.x with lots of bugfixes and the Global Corelation features.
I noticed there are three different types of logs: the file /usr/cids/idsRoot/log/main.log, the directory
/var/log/messages and the event-store. What are the meaning of those different logs? Are there any more logs?
If I were to use Cisco MARS, what logs will MARS receive from IPS? My guess is it will only receive logs from event store (via SDEE over SSL, since IPS doesn't support syslog) is that right?
My interest in this is that I found the following interesting logs in the file "main.log"
25Aug2009 14:33:29.523 1.841 interface Cid/W errWarning Inline data bypass
25Aug2009 14:35:39.532 18000.001 interface Cid/W errWarning Inline data
bypass has stopped.
I was wondering if MARS could receive those logs so I could create a rule to alert us whenever bypass starts. (IPS analysis engine has failed so many times without us noticing it for several days, so we DO need those alerts).
The IPS applications use LogApp to log messages. LogApp sends log messages at any of five levels of severity: debug, timing, warning, error, and fatal. LogApp writes the log messages to /usr/cids/idsRoot/log/main.log, which is a circular text file.
The main.log is included in the show tech-support command output. If the message is logged at warning level or above (error or fatal), LogApp converts the message to an evError event (with the corresponding error severity) and inserts it in Event Store.
Event Store keeps track all the policy violations. SensorApp performs packet capture and analysis. Policy violations are detected through signatures in SensorApp and the information about the violations is forwarded to the Event Store in the form of an alert.
When you are using MARS, it should report the log file, which is /usr/cids/idsRoot/log/main.log. MARS can be configured to act like a syslog server. So, you should be able to receive all these logs.
If you need further help with the configuration, I would recommend opening a TAC case and get live help with the MARS configuration.
Correction to my previous reply..
MARS actually only pulls alerts from the event store. MARS is Not able to receive syslogs from the sensor (the sensor will not send syslogs off the box).
And even if it did, the main.log is not syslogs and is just an internal debug log.
You have to go into the box periodically and manually pull these debugs.
The inline data bypass messages are actually written in at least 2 places.
They are written in main.log and they are also written as Error or Status events in the eventstore.You should be able to see these events using the CLI âshow eventsâ, or in an IDM event screen. Unfortunately none of our event viewers (neither MARS or IME) are built to monitor for Error or Status events.
The inline data bypass state is also included in Health Dashboard information in IME. I would recommend running IME and periodically check the sensor health in IME to see how the sensor is doing.
Thanks for your answer. The "interesting" logs are "warning" level, as you could see
25Aug2009 14:33:29.523 1.841 interface Cid/W errWarning Inline data bypass has started.
25Aug2009 14:35:39.532 18000.001 interface Cid/W errWarning Inline data bypass has stopped.
You told us that logs from "warning" level and above should be sent to the "event store", so then MARS could use "SDDE over SSL" protocol to pull the logs from IPS, and MARS could alert us if data bypass starts.
Please confirm if that's right.
Our IPS are protecting public DNS servers. Every now and then we notice an extremely high number of DNS queries to invalid or non-existent domains. So to avoid Denial of Service to our DNS servers we created atomic signatures using regex to identify the particular URL of the invalid or non-existen domain, so those DNS queries won't arrive to the DNS server. I attached the signature configuration. I was wondering if there was a better signature engine, or another preferable method in order to block those invalid domains.
For some reason, I am not able to open that signature.txt file. Can you paste the contents of the text file, if its not a big file.
signatures 60013 0
sig-name Attack darkbaron.be