Pulling packet capture from IPS device

[haxworthy]

I work for a MSP (Managed Services Provider), we currently are evaluating CSM for mgt of 50 IPS/IDSM devices. To make analysis more effective, want to be able to pull the packet capture from the device. We have our own correlation engine, so we do not need MARS. We want to grab the packet and then put a copy into our ticketing system so the analyst has the data right in front of them.

Is the IP Log directory where the packet capture data is kept? Has anyone ever tried this before? What are the performance/health concerns with enabling packet captures for just high signatures? Does the IP log directory really "clean" itself out after a certain period of time?

[attmidsteam]

We are a MSP also (with our own SIM) and currently have log-attacker & log-victim enabled for all medium & high severity signatures. We only grab the iplog upon demand of the analyst.. note that the iplogID # is included in the event log so it can be parsed and then pulled quite easily. The overhead for logging is minimal and the sensor does clean its iplog directory automatically.

[haxworthy]

Thanks for responding! Another question for you:

You only grab the long upon demand of the analyst, does that mean you run the iplog manually with the analysts criteria, and then scp the log (log id) off the box?

[attmidsteam]

Third try at posting a reply

=============================

As long as you are tuning properly (aka not running the default signature set and using filters) you shouldn't be triggering hundreds a day. The overhead is really not that much (a couple of percent) but we've found that the sensors don't run at the rated speed anyway so odds are you are buying more sensor than is really needed. If you have a high sev signature trigging hundreds of times, the summarization should kick in and you won't get an iplog for the summary message thereby reducing your total iplog count for the sensor.

I would also recommend to change the default iplog size since the defaults result in huge captures when typically only the first couple hundred packets are useful (change the # of packets, max size, and time). You can pull the iplogs via the https iplog-server application with the iplogID # as the argument. If you already have the event pulled up, the iplog # will be in it so you know exactly which capture to pull.

[marcabal]

There are 4 event actions that can be used to capture packets.

The produce-verbose-alert event action will encode the trigger packet as part of the alert itself. So with this event action the packet is already included in the alerts you are already pulling off the sensor. You just need to modify your tool to strip off this packet, decode it, and then add it to your ticketing system at the same time as you add the alert.

This is where I would start.

Using the produceVerboseAlert uses very little additional sensor resources. It has only a very small affect on sensor performance. Because each alert will be larger than normal it will reduce the total number of alerts that can be stored in the sensor's eventstore. But if your application is actively subscribing for these events, then the reduction in total number of alerts stored on the sensor should not cause you any issues. So adding this for all High alerts woulc be a good practice.

The other 3 event actions are log-attacker-packets, log-pair-packets, and log-victim-packets. These event actions will trigger an IP Log (packet log) to be created (or increase the time for capture on an existing IP Log.

The IP Log system is a collection of numbered files on the sensor. As event actions trigger new IP Logs to be created the sensor will pick one of those numbered files and begin writing packets to that file. The sensor retains an internal mapping of what packets are being written to each file. If no empty files exist, then the sensor will automatically overwrite the oldest IP Log file with the new IP Log file. Larger platforms have up to 512 of these numbered files, and smaller platforms may have as few as 128 or even 64 of these numbered files. Each file is 1 Megabyte in size and usually stored in RAM memory. With the limited number of files, the storage of these logs on the sensor is very short term. And so should be pulled off the sensor as soon as possible (just like what you are planning to implement). The sensor also has a usual limit of only writing 20 IP Log files at any one time.

With these limitations on the IP Log files they shoudl be used sparingly. Configuring too many signatures or signatures that trigger often with these event actions can lead to problems. The IP Logs could easily be overwritten by newer IP Logs being triggered, and/or more than 20 could be requested at any one time which means some alerts won't be able to have an IP Log created.

So IP Logging event actions should be limited to only those alerts where the additional data is manditory.

Also understand that IP Logging can have a negative impact on sensor performance. If you plan on using IP Logging often, then consider using a sensor rated for higher speeds than what you will be monitoring.

[marcabal]

So how do you pull the IP Logs off the sensor?

The webserver on the sensor has an "iplog-server" cgi-bin servelet. An example IP Log request to the webserver:

POST /cgi-bin/iplog-server?ipLogId=1734556221&startTime=0077590467654321&stopTime=0077590476543210 HTTP/1.1

Authorization: Basic bmM0cvFuZ3I6XY0RYWrN

User-Agent: CIDS Client/4.0

Host: 10.10.10.59

From: ipLogClient@0010A4BB5319

Content-length: 0

(I believe the startTime and stopTime are optional)

So what is the basic flow you should plan for?

In you event subscription you will see alerts that contain a field for the IP Log ID. That same IP Log ID may be the same for several different alerts. Instead of attaching the IP Log to each ticket I would recommend storing them in one location and having a link in each ticket to the corresponding IP Log.

However, understand that the IP Log is not ready for download when the alert is received. The IP Log gets created after the alert is triggered and depending on your IP Log settings could take some time before it is completed.

The sensor will generate a Status message whenever an IP Log is started, and another evStatus message when the IP Log is completed and ready for download.

You will want your subscription to the sensor to receive evStatus messages (as well the evIdsAlerts it is already watching for). When it sees the evStatus message for the completed IP Log then it can send the request to the web server to download the IP Log and then it can store it away on your server for later analysis by your monitoring team.

The web server request for the IP Logs hasn't changed recently to my knowledge.

For more information on the web request URI for retreiving the IP Logs you shoudl be able to reference the older "Cisco IDS RDEP API v1.0" file available from this web site:

http://www.cisco.com/cgi-bin/dev_support/access_level/product_support?pcgi=1&product=IDS_INT_API

Just look for section 2.5 on IP Logs, and then also at the bottom of the document for the uri-iplog-request.

Hope this helps.

[mhellman]

Getting the packet captures is pretty easy to do. Get and parse the index and directory files. The ipaddress and the eventid are in the directory file and are hex encoded.

"Is the IP Log directory where the packet capture data is kept? Has anyone ever tried this before"

yes and yes.

"What are the performance/health concerns with enabling packet captures for just high signatures"

I'm sure there are implications, but we've not had any problems. We don't turn it on for everything though either.

"Does the IP log directory really "clean" itself out after a certain period of time?"

yes, it appears to use some sort of FIFO method.