Re: IDSM stops logging events in CSPM

jerryd · ‎01-07-2002

Has anyone experienced the same problem. I have CSPM 2.3.3i S13 installed and version 3.0(3)S10 on the IDSM.

The sensor works fine after it is started collecting events. The problem occurs whenever the cspm updates the sensor after a policy change or update. It seems to corrupt or break the packetd.conf file and the sensor stops sending events and there is no new events if you do a "show eventfile current" on the IDSM. A reboot of the sensor corrects the problem and the sensor starts sending events again. Is this a known problem and if so is there a work around to fix this.

marcabal · ‎01-07-2002

We haven't seen this before, but one of our test engineers is trying to replicate your problem.

Could you provide the output from the following commands prior to pushing the CSPM configuration and shortly after pushing the configuration:

show conf (you can remove the entries for the ids and ips when you post the output, we interested in the top of the output where it says what daemons are running)

show ip traffic (you can remove the command and control information, we are interested just in the monitoring interface information)

You may also want to run the report systemstatus command under the diagnostics command mode. It will require that you enter information for an ftp server. It will generate an html file with a complete diagnostic for the module and ftp it to your ftp server. You can then download it off your ftp server and open it in a web browser.

You can look at the output from multiple different diagnostic commands and the contents of configuration files.

If you find something interesting in the report or would like me to look at it then you can email it directly to me.

ibanezm · ‎01-07-2002

I could not replicate the problem.

On CSPM, the IDSM can be configured as IDSM 3.0(2)S10, IDSM 3.0(3)S12 or IDSM 3.0(3)S13 since there is no IDSM 3.0(3)S10 choice. The actual sensor is an IDSM 3.0(3)S10. I received S10 events as well as pre-S10 events initially. I then modified some S10 signatures via CSPM, then I pushed the new config files to the sensor and I continued to receive S10 events as well as pre-S10 events.

Is my test representative to what you're doing? if not, can you send more specifics as to how this is happening? Is this problem happening "everytime" you update the sensor? or is it intermittent?

jerryd · ‎01-07-2002

On my CSPM Ive set the sensor version to 3.0(2)S10. It seems that this problem is intermittent as I cant get it to break again. The testing is indicative of what Im doing. The reboot I did yesterday seems to have sorted out the problem. Ive pushed about 3 policy changes and each time the sensor is doing what its supposed to. The processes all show that they are running after the policy update. The only thing I can really see in the system status report is.

Contents of file errors.packetd.58

01/07/2002 14:16:09UTC E NrConfigData::readConfigFile : unrecognized token 'MinutesOfAutoLog' in config file 'C:/Program Files/Cisco Systems/NetRanger/etc/pack

etd.conf'

01/07/2002 14:16:09UTC E NrConfigData::readConfigFile : unrecognized token 'LevelOfTrafficLogging' in config file 'C:/Program Files/Cisco Systems/NetRanger/etc

/packetd.conf'

01/07/2002 14:16:09UTC E NrConfigData::readConfigFile : unrecognized token 'NumberOfRcptTo' in config file 'C:/Program Files/Cisco Systems/NetRanger/etc/packet

d.conf'

and

Event Log - application

Error 01/08/2002 06:47:30 CISCO_IDS 00012 Scp Service

The output of the show conf and sh ip traffic is:

Sensor version is : 3.0(3)S10

!

Sensor application status:

nr.postofficed running

nr.fileXferd running

nr.loggerd running

nr.sapd running

nr.packetd running

Configuration last modified Mon Nov 26 14:19:00 2001

Monitor Interface Statistics:

Statistics from: 01/07/2002 14:15:55

Number of seconds: 66187

IP Packets: 202167908

Filtered Packets: 0

ICMP Packets: 889257

TCP Packets: 196621193

UDP Packets: 4629307

Other Packets: 42820114

Bad IP Packets: 0

Bad ICMP Packets: 0

Bad TCP Packets: 0

Bad UDP Packets: 0

Objects: na

Number Of TCP Streams: 9537660

Packet Totals:

packetCount.total: 202181913

packetCount.arp 13875

packetCount.ip: 202167908

packetCount.icmp: 889257

packetCount.tcp: 196621193

packetCount.udp: 4629307

packetCount.bad: 42806239

packetCount.proto: 0

Memory in Use: 66929284

Previous HiWater: 69322552

Heap free: 1387912

Heap used: 79334008

Unalloced Memory: 16795648

Free Buffers 834

Buffers in FragQ: 0

Buffers in TcpQ: 0

Buffers in SSE: 0

Backlog info..

Names:

total - rxa - rxb - txa - txb - cai - cao - cbi - cbo -

216 - 0 - 0 - 0 - 0 - 216 - 0 - 0 - 0 -

Tcp Stream Info:

tcpFlow: 7832 tcpFlowObjs: 12731

tcpFlowTotal: 9537660 stringSearchCtx 25462

Partial Datagrams:

fragQueue: 0

Signature Elements:

sigData: 11724 ipFragOver: 0

icmpLoki: 0 icmpModLoki: 71

tcpQueso: 0 tcp-BO-2K: 917 ( 116)

tcpSynFlood: 158 tcpHijack: 517

floods: 18 sweeps: 2694

satan: 661 TFN2K: 71

udpBO: 488 nb-oobs: 0

smbauthfail: 4 redbuttons: 0

nb alarms received: 113063

nb alarms purged: 0

nb upcalls succeeded: 49907

nb upcalls Failed: 0

pktd alarms recv: 113063

pktd alarms lost: 0

After policy upload

Sensor version is : 3.0(3)S10

!

Sensor application status:

nr.postofficed running

nr.fileXferd running

nr.loggerd running

nr.sapd running

nr.packetd running

Configuration last modified Mon Nov 26 14:19:00 2001

Monitor Interface Statistics:

Statistics from: 01/07/2002 14:15:55

Number of seconds: 66610

IP Packets: 204177701

Filtered Packets: 0

ICMP Packets: 896364

TCP Packets: 198577314

UDP Packets: 4675572

Other Packets: 43431043

Bad IP Packets: 0

Bad ICMP Packets: 0

Bad TCP Packets: 0

Bad UDP Packets: 0

Objects: na

Number Of TCP Streams: 9631206

Packet Totals:

packetCount.total: 204191795

packetCount.arp 13964

packetCount.ip: 204177701

packetCount.icmp: 896364

packetCount.tcp: 198577314

packetCount.udp: 4675572

packetCount.bad: 43417079

packetCount.proto: 0

Memory in Use: 67920864

Previous HiWater: 69322552

Heap free: 1355688

Heap used: 79366232

Unalloced Memory: 16795648

Free Buffers 882

Buffers in FragQ: 0

Buffers in TcpQ: 0

Buffers in SSE: 2

Backlog info..

Names:

total - rxa - rxb - txa - txb - cai - cao - cbi - cbo -

182 - 0 - 0 - 0 - 0 - 182 - 0 - 0 - 0 -

Tcp Stream Info:

tcpFlow: 10222 tcpFlowObjs: 12731

tcpFlowTotal: 9631206 stringSearchCtx 25462

Partial Datagrams:

fragQueue: 0

Signature Elements:

sigData: 14365 ipFragOver: 0

icmpLoki: 0 icmpModLoki: 65

tcpQueso: 0 tcp-BO-2K: 751 ( 93)

tcpSynFlood: 127 tcpHijack: 467

floods: 11 sweeps: 3123

satan: 517 TFN2K: 106

udpBO: 352 nb-oobs: 0

smbauthfail: 4 redbuttons: 0

nb alarms received: 113917

nb alarms purged: 0

nb upcalls succeeded: 50259

nb upcalls Failed: 0

pktd alarms recv: 113916

pktd alarms lost: 0

Im going to monitor this and see if it happens again.

jerryd · ‎01-08-2002

Its happened again, this time all the services are running but its unable to talk to the monitor interface.

sensor-A# sho ip traffic

Monitor Interface Statistics:

Error timeout waiting for response

Command and Control Interface Statistics:

I will be sending a system status report to marcabal

jeff.gift · ‎01-09-2002

Jerry,

I am experiencing a similar issue with the same versions you are using. My IDSM logs events to the database for several hours and then just stops reporting to the CSPM. I checked the IDSM and all the services are still running. I then have to reset the IDSM and immediately push the most current policy again. This seems to have started after installing the 3.0(3)S10 IDSM.

Jeff

marcabal · ‎01-09-2002

If users are seeing this with version 3.0(2)Sx or earlier then they need to upgrade to 3.0(3)S10.

If you are seeing this in 3.0(3)S10, as you state, then please contact the TAC and request that a DDTS Issue be created for your problem. We need to be able to track this internally and have it documented that customers are reporting this.

You will need to supply the TAC with the html file from the "report systemstatus" command after the problem is seen on the IDSM.

The TAC will then need to contact development to determine the next stage in daignosing your issue.

zhucg · ‎02-15-2002

I have the exactly the same issue here (same version of IDSM/CSPM code, same error message in packetd.conf and same symptoms - stop capturing packet intermittently, 2 or 3 time a day). Is there any TAC case opened and what is the resolution/workaround? please advise. thanks

zhucg · ‎02-15-2002

FYI, there are a number of bugs are related to this issue: CSCdw49651 / CSCdv77620 /..

marcabal · ‎02-15-2002

Several of these issues are being addressed in the next version for IDSM.

Watch your email or the Forum for the announcement of the next verison.

However, there are issues with those symptoms that are not fixed by the new version,

and engineering is still working on them.

So when the new version comes out, you can load it and hopefully it will resolve the issues

you are seeing. If it does not then refer to the readme with that new version to determine

what issue you are seeing.

Marco

jballay · ‎03-26-2002

Has this issue been fixed? If yes, what's the release # to go with? If not, what's the most stable release to update the modules?

Thanks.