Four IDSM-2 Hang at once

Farrukh Haroon · ‎09-03-2008

Dear All

At one of our customers, four IDSM-2 blades stopped reponding 'ALL' at the same time (7 AM this morning). I can login to the CLI and see the following message:

Error: Cannot communicate with mainApp (getVersion). Please contact your system administrator.

Would you like to run cidDump?[no]: yes

As per Cisco, the solution is to reboot:

http://www.cisco.com/en/US/products/hw/vpndevc/ps4077/products_qanda_item09186a008025c533.shtml#ips

Does anyone ever faced this before, or have a better solution to the problem? :)

I have already captured the Core Dumps.

Regards

Farrukh

paulhignutt · ‎09-03-2008

Do they all monitor traffic from a common VLAN? Possibly some sort of traffic that they can't parse properly? Get the sniffers going again at 6:50 AM tomorrow... ;-)

Just a thought.

Farrukh Haroon · ‎09-03-2008

If it was caused by some traffic, then it would have been a broadcast/multicast packet, as under normal operation two of the IDSM do not pass any traffic (as they are in the chassis in which FWSM in standby/secondary). This happened once is more than two years I think, so the chances of it happening again would be quite less. All came up after reboot, but the real worry is WHAT caused it? :)

To answer your question, yes all IDSM(s) share the same VLANs. Two are present on one chassis bridging the VLANS on the switches with the FWSM SVIs (Primary FWSM). The other two are on the second Core switch with the Secondary/Standby FWSM. There is ECLB (load balancing) for both pairs.

Regards

Farrukh

paulhignutt · ‎09-03-2008

I have had this happen once before, with a single IDSM in each of two 6513's. It was a redundant switch fabric, and to be honest I just rebooted the IDSMs and didn't investigate it further. It never happened again, and that was on 5.x about a year ago. So it sounds like it might be the same thing. But who knows. In my situation they were both monitoring the same VLANs so that's why I was thinking some sort of anomalous broadcast traffic.

Farrukh Haroon · ‎09-03-2008

Thank you very much for your response(s).

It would be really nice if someone from the Cisco IPS Team could commend on this.

Regards

Farrukh