How to test anomaly detection in IPS6 ?

ovt · ‎12-27-2007

Hi!

Does anybody have experience with AD in IPS6? I tried to test it today with 3 nmap sessions each scanning 100 different IPs. I saw the Sig 13003-0 (single scanner) fired:

signature: description=AD - External TCP Scanner id=13003 version=S262

alertDetails: . adExtraData: numDestIps=150; currentThreshold=150; destPort=80

The scanner threshold was indeed set to 150:

S1# sh ad-knowledge-base vs0 thresholds current

External Zone

TCP Services

Default

Scanner Threshold

User Configuration = 150

Threshold Histogram - User Configuration

Low = 10

Medium = 3

High = 1

UDP Services

This is ok. The problem is that the Sig 13003-1 (warm) didn't fire, however the number of scanned IPs was very high:

S1# sh statistics anomaly-detection vs0

Statistics for Virtual Sensor vs0

Attack in progress

Detection - ON

Learning - OFF

Next KB rotation at 10:00:00 MSK Fri Dec 28 2007

Internal Zone

TCP Protocol

UDP Protocol

Other Protocol

External Zone

TCP Protocol

Service 80

Source IP: 10.0.1.1 Num Dest IP: 280

Questions:

- what does Low/Medium/High exactly mean in threshold histogram?

- how does the sensor detect worms? When the Sig 13003-1 fires? What sequence of events should happen?

- how can I test it?

marcabal · ‎01-02-2008

The sensor constantly watches for scanners on each port.

There are 3 categories of scanners:

Low scanners - scanners that are only scanning a low number of hosts.

Medium scanners - scanners that are scanning a medium number of hosts

High scanners - scanners that are scanning a high number of hosts

NOTE: I can't remember for sure how many hosts must be scanned for it to be a "Low" number of hosts, or "Medium" or "High". But it may be something like 5 hosts scanned is a "Low" scanner, 20 for Medium and 100 for High. Once again I am not sure of those numbers.

Also be aware that the number of hosts scanned is not the Total numner of hosts scanned, but is instead the number of hosts scanned THAT did not respond.

If you connect to 100 web servers and all web servers respond then it does not count that as a scan. If you try to connect to 100 web servers and 92 respond, then for the 8 that don't respond you would be categorized as a Low scanner.

But just because a scanner is counted in a category does not mean an alert will be generated.

There are 2 types of alerts (subsig 0 alerts, and subsig 1 alerts)

Subsig 0 alerts are for a scanner that is scanning enough hosts that you want an alert for it even when no worm has been declared.

This is the "scanner Threshold / User Configuration = 150" that you see in the "show ad-knowledge-base vs0 thresholds current" output.

If a scanner scans more than 150 hosts then a specific alert is generated even though no worm has been declared.

Any scanners scanning less than 150 hosts are still categorized but do not have alerts generated for them when no worm has been declared.

The subsig 1 alerts are for when a Worm has been declared.

Here is how a worm gets declared:

The Thesholds for Low, Medium, and High that you see in "show ad-knowledge-base vs0 thresholds current" is the number of active scanners in each category that are allowed to normally be seen on your network (this is the number of scanners that will be seen on your network even when there are no worms).

A worm gets declared when the number of scanners in any one of the 3 catgeories goes above the threshold for that category.

Let's take for example Medium=3 as the threshold for port 21. And let's assume it takes a scan of 20 hosts to be categorized as a Medium scanner.

This means normally you could have up to 3 scanners on your network where each scanner is scanning 20 or more non-responding hosts on port 21.

(Maybe these are 3 network administrators periodically checking to see which machines have port 21 open)

Suddenly you have 5 scanners that start scanning on port 21 and each of the 5 winds up with 20 or more non-responding hosts.

That 5 has broken the threshold of 3, and a worm is declared. Now any Medium Category scanner on port 21 will begin being declared a scanner under a worm condition (subsig 1).

So for your testing.

Instead of running a scan of 100 hosts from just one machine, I would recommend you scan the same 100 hosts from 2 or 3 machines (NOTE: Only need to scan a single port across those 100 hosts).

Scanning 100 hosts should get them categorized as High scanners. And having 3 High Scanners should push it over the threshold of 1.

BUT keep in mind that it needs to be 100 hosts not responding on the scanned port.

Then you will also want to try it with fewer hosts being scanned (like say 25), but with say 5 machines running nmap doing the scanning.

ovt · ‎01-08-2008

Thanks a lot, Marcoa, I'll try it.

Another question: how does learning process calculate thresholds (scanner threshold and histogram thresholds) in learning mode? Does it use some multiplier? Say, the max scanning rate was 250 for 24 hours. Will the scanner threshold be set to 250 or, say, 500? What about histogram thresholds?

Also, if the max scanning rate was 150 for 24 hours will it remain the default (200) or set to 150?

And how does the rate is calculated? I mean what time interval is used? Something like running average for 1 minute interval, 30 sec. interval or what?

Also, what is happening in detect mode? I read somewhere that the thresholds _can_ go higher in the detect mode, but _only_ if no attack is detected. How can I understand this?

marcabal · ‎01-08-2008

When in learning mode there are 4 items learned for each port/protocol/zone.

These are

1) the single scanner threshold,

2) the threshold for number of low rate scanners (scanners scanning 5 or more addresses)

3) the threshold for number of medium rate scanners (scanners scanning 20 or more addresses)

4) and the threshold for number of high rate scanners (scanners scanning 100 or more addresses)

All thresholds are based on a 1 minute interval.

When in learning mode it looks for the highest rate scanner and then multiplies this number by a multiplier. I remember reading that for single scanner the multiplier is 2. So if you highest rate scanner is scanning 150 machines per minute then the threshold would be set to 300.

However, that threshold may not get used. For each protocol/zone there is a default scanner threshold. Users can also overwrite that threshold for the protocol/zone. The user can also specify a threshold for the port/protocol/zone.

The new threshold must be higher than the default (or user modified) theshold for the protocol/zone AND the user must not have configured a specific threshold for the port/protocol/zone in order for that new "learned" theshold to actually be put into the Knowledge Base.

So though learning is going on for all ports, only ports with a "learned" threshold higher than the default protocol/zone thresholds will actually be written to the Knowledge Base.

In the case of the numbers of Low, Medium, and High rate scanners the same thing is done except that the multiplier is "1.2".

So if 30 Low Rate scanners are detected then the "learned" Low Rate scanner threshold would be 36.

Now when the sensor is in Detect Mode then the sensor is still also able to learn new thresholds.

It keeps doing the same counting that it was doing in Learning mode.

If NO attack has been detected (all counts for all ports/protocols/zones are below the thresholds in the current Knowledge Base), then the sensor still goes ahead and uses the multipliers. If the new "learned" thresholds are highere than the currently used thresholds then the new learned thesholds will be written the next time a Knowledge Base is saved.

SO the for a new "learned" threshold to happen during Detection the counts must be lower than the current thresholds but when multiplied by the multiplier the learned thresholds are higher than the current.

Say for example we have a 80 threshold for Low Rate scanners on port 80.

During detect we count 70 scanners.

70 is multipled by 1.2 to get 84.

The next time the Knowledge Base is written it will set the Low Rate scanner threshold to the newly learned 84.

NOTE: If we had seen 81 scanners then a "worm" would be detected, and the sensor would not fo the multiplier and would not learn a new threshold.

You can configure a sensor to preiodically save off (and even use) a new Knowledge Base.

By default it is schedule to save off and use a new Knowledge Base every 24 hours.

When that new Knowledge Base is saved off it writes any new higher thresholds learned from any 1 minute interval in the past 24 hours.

ovt · ‎01-08-2008

Ok, this makes sense. However, it is very strange that "low", "medium" and "high" cannot be configured and predefined as 5, 20 and 100 destination IPs being scanned per minute.

One more question. When the histogram threshold is exceeded the Sig 1300x-1 fires (for example, if more than 3 hosts are scanning more than 20 IPs each) and the sensor goes into "worm detected" mode. Then, this signature (1300x-1) start firing for each host that is scanning more than 20 IPs, even if a single scanner remains active. The "worm-timeout" under "service anomaly-detection ad0" is used to stop this. If there are no hosts remain that scan more than 20 IPs for worm-timeout seconds, then the "worm detected" mode is turned off. Is my understanding correct?

ovt · ‎01-10-2008

It seems that learning is not working :( I've verified that detection is working well (both 1300x-0 and 1300x-1 for 2 "high" scanners fired), then turned detection off, then repeated my tests, but KB haven't changed :(

S1# show ad-knowledge-base vs0 thresholds current zone external protocol tcp

AD Thresholds

Creation Date = 2008-Jan-10-10_00_08

KB = 2008-Jan-10-10_00_08

External Zone

TCP Services

Default

Scanner Threshold

User Configuration = 150

Threshold Histogram - User Configuration

Low = 10

Medium = 3

High = 1

So, I definetely crossed the thresholds, but nothing has changed. Am I missing something here?

marcabal · ‎01-11-2008

The confusion here is what "current" means.

In the case of "show ad-knowledge-base vs0 thresholds current" it is not to look at what is the most up to date leanred information in the knowledge-base, but is instead showing you want the currently "applied" knowledge-base has. The "current" knowledge-base is one that was likley leanred Previously and is currently active. In the case of detection mode the "current" is what is being used for detecting worms and scanners. In the case of learning mode the "current" is what the sensor uses to compare recent scanner counts in order to determine if New learned threshols should be written then Next Time the knowledge-base gets Saved.

So to see what the New learned thresholds are you need to do a "Save". It will write the new thresholds to the file name you provide. You can then view that saved knowledge-base to see what the sensor had learned. The confusion often comes from the fact that in IDM it is called "Save Current". But it is not saving the "current" active knowledge-base, instead it is saving the most recent learned information.

If you prefer the new thresholds in the recently saved knowledge-base then you can "Load" that recently saved knowledge-base in order to make It the "current".

NOTE: By default the sensor is configured so that at 10:00:00 every morining it will automatically do the "Save" of the most recently "learned", and then also automatically "Load" it so it becomes the new "current".

So you can either "Save" it yourself and then "Load" it yourself. Or you can wait till 10:00:00 am and have the sensor do it automatically (assuming the config is still defaulted)

ovt · ‎01-11-2008

Well, I don't see this behaviour. Something seems to be wrong:

sh ad-knowledge-base files

Virtual Sensor vs0

Filename Size Created

initial 88 16:30:36 MSK Mon Dec 17 2007

* 2008-Jan-10-10_00_08 88 10:00:08 MSK Thu Jan 10 2008

2008-Jan-11-10_00_08 124 10:00:08 MSK Fri Jan 11 2008

The learning mode was turned ON (detection turned OFF) on Jan 10. Several attacks were performed on Jan 10. Today (Jan 11) new KB has been saved wich indeed contains new threholds (Jan-11 file). The problem is that this file isn't current (no *). The current file is Jan-10 file, which contains default thresholds. "Save current" saves this (older) file. I had to "Load" the Jan-11 KB file in IDM to make it current.

So, it seems that there is no way to see or use new thresholds until the system automatically saves the KB file. Then this file should be "Loaded" via IDM to make it current. Looks like a bug? I definetely need to retest it from scratch.

Anyway, I got the idea. Thank you for replays!

marcabal · ‎01-11-2008

Can you check your configuration of ad0.

Within in IDM on the Learning Accept Mode tab of the ad0 configuration cna you verify that the Action is set to "Rotate" and not set to "Save Only".

With Rotate it should Save And Load the new file at 10:00:00 am everyday.

If it is configured to the default Rotate action, then I wonder if the problem may be specific to Learned mode. I've seen it working when it was Learning while in Detect mode, but I have not checked for it when configured for just Learning mode.

ovt · ‎01-11-2008

S1(config-ano)# sh sett

worm-timeout: 600 seconds

learning-accept-mode

-----------------------------------------------

auto

-----------------------------------------------

action: rotate

schedule

-----------------------------------------------

periodic-schedule

-----------------------------------------------

start-time: 10:00:00

interval: 24 hours

S1# sh statistics anomaly-detection

Statistics for Virtual Sensor vs0

No attack

Detection - OFF

Learning - ON

Next KB rotation at 10:00:00 MSK Sun Jan 13 2008

ovt · ‎01-12-2008

I've retested it and the problem persist:

S1# sh ad-knowledge-base files

Virtual Sensor vs0

Filename Size Created

* 2008-Jan-11-10_00_08 124 10:00:08 MSK Fri Jan 11 2008

2008-Jan-12-10_00_05 124 10:00:05 MSK Sat Jan 12 2008

S1# sh ad-knowledge-base vs0 thresholds file 2008-Jan-11-10_00_08 zone ext p tcp

AD Thresholds

Creation Date = 2008-Jan-11-10_00_08

KB = 2008-Jan-11-10_00_08

External Zone

TCP Services

Default

Scanner Threshold

User Configuration = 150

Threshold Histogram - User Configuration

Low = 10

Medium = 3

High = 1

22

Scanner Threshold

Knowledge Base = 400

Threshold Histogram - Knowledge Base

Low = 10

Medium = 3

High = 2

S1# sh ad-knowledge-base vs0 thresholds file 2008-Jan-12-10_00_05 z ext p tcp

AD Thresholds

Creation Date = 2008-Jan-12-10_00_05

KB = 2008-Jan-12-10_00_05

External Zone

TCP Services

Default

Scanner Threshold

User Configuration = 150

Threshold Histogram - User Configuration

Low = 10

Medium = 3

High = 1

80

Scanner Threshold

Knowledge Base = 244

Threshold Histogram - Knowledge Base

Low = 10

Medium = 3

High = 3

The Jan-12 file was saved, but wasn't "loaded" (marked as "current"). Also, Jan-11 file was "loaded" (made current) by hands but it didn't put its thresholds into the runtime, so Jan-12 file didn't inherit them. This looks like a BUG too.