At a IDS Version 2.5(0)S0 we configured the sensor to records iplogs for the most interesting events. MinutesOfAutoLog is set to 1 minute. The resulting iplogs have all the size of 51200 bytes. The problem is now, when a web scan for example is causing traffic for more than 51200 bytes, the sensor will not log more than this limit, even if the scan is still triggering events that are configured to record an iplog.
nrvers is recording "v18.104.22.168 (Release) 00/09/14-05:06" for the sensor.
Starting in version 2.5 the method for ip logging has changed somewhat.
In 2.2.1 and prior versions the ip log would be created when the first packet was captured for that source. Then any additional packets captured were added to that file, and the user could execute the command "ls -l /usr/nr/var/iplog" and watch the ip log file grow as more packets were captured.
In 2.5 and later versions the ip log gets created when the first packet from that source is captured, but instead of creating a file only containing that first packet it will automatically create a 51,200 byte memory mapped file with that packet only taking up the first few bytes. What that means is that it automatically claims all 51,200 bytes of disk space for the file as soon as it is opened, and then begins filling it with packets as they are captured. When enough packets have been captured to fill the 51,200 byte file then the file is closed and moved to the /usr/nr/var/iplog/new directory, and a new 51,200 byte file is created in /usr/nr/var/iplog directory for additional packets being captured. So when viewing ip logs you first want to start with the older files already moved into the /usr/nr/var/iplog/new directory.
Well, the question is, what happens with the files, if the iplog produces more can 2 51200 bytes files in one minute. The file naming convention I've seen is using following format:
Does this mean that the first file is overwritten ? Or is the newer discarded?
The reason for my question was, that I found a bigger web scan, but only the first (or last) few parts of the conversation was logged. Every event in the events logs was triggering an iplog, but I got only a few. This would be a dangerous situation, if the attacker was able to find a vulnerability in the non logged part.
The file name format contains the work iplog, followed by the ip address, followed by a date and time stamp, and occasionally an integer value.
The timestamp is minute based, so if more than 1 iplog gets created in the same minute for the same address then the iplog is moved to the /usr/nr/var/iplog/new and a another iplog is opened with the same name but with an additional ".1" appended after it. If more are created then the ".1" is incremented to ".2", ".3" and so forth until the next minute is reached.
This sounds good, but isn't working on my sensor. For example, we logged an web scan. There were 24 different event types logged from 10:47:xx up to 10:49:xx. All are configured to create an iplog. The result is one iplog, named iplog.xxx.xxx.x.x.200105031047. Nothing more on this maschine (checked all subdirectories below /usr/nr/var).
It appears that everything is working as designed.
Here is the sequence of steps that the sensor sould be executing.
1) The first signature with IP Log set as an action is triggered.
2) Packetd creates an alarm and sends that alarm to the director (and any other destinations)
3) Packetd then signals the sniffing portion of packetd to start capturing all packets to and from the source of the alarm for the MinutesOfAutoLog amount of time.
(If you'll notice the ip logging is started AFTER the signature is triggered, so the resulting ip log will not contain the packets which actually triggered the alarm. Also the ip log is based on address and not on which signature is fired. The sniffing portion of packetd simply logs all packets to and from the source address.)
4) The captured packets are sent to loggerd, and loggerd creates an iplog file based on the source address and a timestamp. (loggerd is actually the program that writes the packets to disk)
5) Now the second signature triggers
6) Packetd creates an alarm and send it to the appropriate destinations.
7) Packetd then send another signal to the sniffing code to capture packets from the source address.
8) The sniffing code is already capturing those packets and sending them to loggerd. So the sniffing coce will determine that the same address in the request is already being logged and will then look at the new log time. If the new log time is smaller than what is left from the previous log time then it will keep following the older (longer) log time. If the new log time is larger than what is left than the older log time then it will reset it's timer for the newer time. It does not start a new ip log for each signature fired.
9) Since the packets are still from the same address loggerd continues placing them in the same ip log file.
If in your example, all 24 events were from the same source address, then the IP Log created is for all of the events, and you should only expect 1 ip log file. Remember that the IP Log is created after the alarm fires so the packets which caused the alarm are not in the IP Log file.
I also recommend reading the following discussion from the IDS forum:
Sorry, I thought it is clear that not all of the ip packets that are part of the 24 events I logged are captured. Otherwise I wouldn't post this questoins.
I got only 12 requests in the iplog file (including the answer of the first one). The 24 different events are exactly 32 single attacks (24 was a summary over event type). The attack started at 10:47:23 and ended at 10:49:30.
I got on iplog which ends at around 10:47:30. The 12 requests of this iplog do correspond to 5 events in the sensor log. The rest are other requests (all invalid but not catched by a signature).
There is no other iplog file on the sensor. Every of the missing 27 events would trigger a new iplog. Even the rest of the traffic of the first minute is missing. MinutesOfAutolog is set to 1.
Reading the appends I got the idea that this option is working for others. So there must be a configuration problem or something else.
1) Load the latest Service Pack 2.5(1)S0 and Signature Update 2.5(1)S2. I don;t think this will fix the problem you are seeing, but at least this way we can all be wokring with the latest version.
2) Consider the type of sensor, the rate of traffic, the rate of alarms, and the amount of packets being IP Logged to see if the sensor is being overloaded. Both IP Logs and alarms use the same communication infrastructure, so if there is a high rate of alarms and/or IP Logged packets then it will first try to send the alarms to their destinations before sending the IP Log packets.
3) You may have found a bug we haven't seen during testing. To help us in reproducing it in the lab, we would need a copy of your packetd.conf file, destinations file, and a copy of the scan tool you are using. You can email them directly to me instead of posting them to the forum.
Also a copy of the log output showing the alarms that fired would help as well.
We have configured the outside and inside Interface with official ipv6 adresses, set a default route on outside Interface to our router, we also have definied a rule , which also gets hits, to permit tcp from inside Interface to any6.
In Syslog I also se...