ASA-SSM-20 Unresponsive, cannot reimage

Unanswered Question
Sep 2nd, 2008
User Badges:

After following procedure on:


http://www.cisco.com/en/US/docs/security/ips/6.0/installation/guide/hwImage.html#wp1183593


Module becomes UNRESPONSIVE, after hw-module module 1 recover boot command.


I'm using IPS-SSM_20-K9-sys-1.1-a-6.1-1-E1.img


¿Any ideas, how to restore this module back to normal ops?

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
marcabal Tue, 09/02/2008 - 11:54
User Badges:
  • Cisco Employee,

This is documented in the guide at the bottom of step 9:

Note To debug any errors that may happen in the recovery process, use the debug module-boot command to enable debugging of the system reimaging process.


A usual poblem with the System Re-imaging process is that the card winds up in a boot loop because of an error. When ROMMON detects an error it reboots and tries the same steps again which usually winds up with the same error which causes a reboot, etc.....


So determining if the card is in a reboot loop, and what the error is would be the next step in your debugging process.


Execute "debug module-boot". Enter "hw-module module 1 recover stop". Wait for a few minutes, and then enter "hw-module module 1 recover boot".

The output from ROMMON on the SSM will be seen on your ASA connection.


Look at the configuration being passed to the SSM's ROMMON and look for any bad entries.

Watch to see if it able to download the System Image file, or if it continuously reboots.

If it continuously reboots, then look to see what error message is seen just prior to the reboot.



Some common problems:

1) Typos in IP address, gateway, tftp server IP, or system image filename.


2) If the tftp server is on the same subnet as the SSM's IP Address, then try leaving the Gateway address blank since it is not needed.


3) Remember that the IP Address is for the external interface of the SSM. So be sure you are using an address that is applicable for the network where you are pluggin in the SSM's external interface.


4) If the TFTP Server is on another subnet, then be sure there is a route to the other network. If having to route back through the ASA, then ensure that the ASA will allow TFTP packets to pass through the ASA. (The ASA could wind up blocking the TFTP packets depending on the ASA configuration)


5) Be sure the file can be downloaded from the TFTP server. Check the file permissions, and the directory where the file is located. From your desktop try to downlaod the file from the tftp server. This will ensure you are using the correct directory and that the file has correct permissions. Once common problem is that the file may be /tftpboot/sensorfiles/IPS-SSM_20-K9-sys-1.1-a-6.1-1-E1.img. But because the tftp server automatically starts in /tftpboot, you may need to NOT specify it for the file and instead just use: sensorfiles/IPS-SSM_20-K9-sys-1.1-a-6.1-1-E1.img


6) Check to make sure the file is not corrupted by running an md5sum and checking it against the value listed on cisco's web site.


=================


With all of that said, however, you need to question whether or not you should be using a System Image file in the first place.


The System Image files are meant to be used only when a complete erasing of the sensor's image is needed. This is generally because the installed files were corrupted, or so old that it would be easier to start over and make it look like it came from the factory; than to use the standard "upgrade" files.


In more than 90% of the cases, most customers will want to "upgrade" rather than do a System Image. The "upgrade" is done from within the sensor itself, and will both load the higher version as well as convert your current configuration to work with the newer version.


Instructions for "upgrading" to 6.1(1)E1:

http://www.cisco.com/en/US/docs/security/ips/6.1/configuration/guide/cli/cli_system_images.html#wp1142504


The IPS-K9-6.1-1-E1.pkg works across ALL sensors (including the SSM-20) except the AIM.

http://www.cisco.com/cgi-bin/tablebuild.pl/ips6


In fact I would use IPS-K9-6.1-1-E2.pkg rather than IPS-K9-6.1-1-E1.pkg. The E2 is the most recent set of releases.


Try using "hw-module module 1 recover stop" to stop the current recovery. If it goes back to an older running image, then use the "upgrade" method rather than the System Image method to get to the higher version.


J_Vansen_S Thu, 01/01/2009 - 19:21
User Badges:

hi Marcabal,


I am also having issue with reimaging my AIP-SSM module.


I made a mistake. I did the "hw-module module 1 configure" command without knowing the IPS-SSM interface ip address. I proceeded with it anyways. And now my SSM module status is "Recover, Data Plane Status = Not Applicable"


Is there any ways i can resolve this?

marcabal Mon, 01/05/2009 - 08:28
User Badges:
  • Cisco Employee,

Execute "debug module-boot" just to help you see what is going on.


Execute "hw-module module 1 recover stop" to stop the current recovery.


Execute "hw-module module 1 recover config" to modify the settings to be used for the recovery (uncluding changing the IP)


Wait a few minutes for the previous recovery to completely stop and allow the module to either go to Up or timeout and go to Unreponsive.


Execute "hw-module module 1 recover boot" to start the recovery again with the new parameter settings.

gusortiz Mon, 09/08/2008 - 08:13
User Badges:

Hi,


After making the procedure module is now on RECOVER mode, it never goes to UP.


--- -------------------------------------------- ------------------ -----------

1 ASA 5500 Series Security Services Module-20 ASA-SSM-20 JAF11120177


Mod MAC Address Range Hw Version Fw Version Sw Version

--- --------------------------------- ------------ ------------ ---------------

1 001b.d5c7.8505 to 001b.d5c7.8505 1.0 1.0(11)2


Mod SSM Application Name Status SSM Application Version

--- ------------------------------ ---------------- --------------------------


Mod Status Data Plane Status Compatibility

--- ------------------ --------------------- -------------

1 Recover Not Applicable


ciscoasa(config)#

ciscoasa(config)#


And still no SW shown in the output.


Any thoughts?

marcabal Mon, 09/08/2008 - 08:22
User Badges:
  • Cisco Employee,

Did you execute the "debug module-boot" command?

Did you see any output on the ASA console that looks like ROMMON lines from the SSM module?

If so can you paste that output into your response?

gusortiz Mon, 09/08/2008 - 08:24
User Badges:

ciscoasa(config)# hw-module module 1 recover boot


The module in slot 1 will be recovered. This may

erase all configuration and all data on that device and

attempt to download a new image for it.

Recover module in slot 1? [confirm]

Recover issued for module in slot 1

ciscoasa(config)# Slot-1 452> Cisco Systems ROMMON Version (1.0(11)2) #0: Thu Jan 26 10:43:08 PST 2006

Slot-1 453> Platform ASA-SSM-20

Slot-1 454> GigabitEthernet0/0

Slot-1 455> Link is UP

Slot-1 456> MAC Address: 001b.d5c7.8506

Slot-1 457> ROMMON Variable Settings:

Slot-1 458> ADDRESS=192.168.23.97

Slot-1 459> SERVER=192.168.23.13

Slot-1 460> GATEWAY=0.0.0.0

Slot-1 461> PORT=GigabitEthernet0/0

Slot-1 462> VLAN=untagged

Slot-1 463> IMAGE=IPS-SSM_20-K9-sys-1.1-a-6.1-1-E2.img

Slot-1 464> CONFIG=

Slot-1 465> LINKTIMEOUT=20

Slot-1 466> PKTTIMEOUT=4

Slot-1 467> RETRY=20

Slot-1 468> tftp [email protected]

Slot-1 469> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Slot-1 470> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


Slot-1 556> Received 28195232 bytes

Slot-1 557> Launching TFTP Image...


This the output, after this, module goes to recover....

marcabal Mon, 09/08/2008 - 08:40
User Badges:
  • Cisco Employee,

How long after the "Launching TFTP Image" has it been?


Was the above seen just once, or do you keep seeing it again and again?

(The above should be seen just once after starting the recovery)


When "Launching TFTP Image" is seen, this means that the System Image was properly download and the installation has begun.


It will take some time for the installation to complete. I've never timed the installation, but I think that it might take as much as 20 to 30 minutes. But I would go ahead and give it an hour just to be sure.


If it is still not UP after an hour, then I would assume something has gone wrong.


It is possible that the ASA itself may be confused and unable to re-establish communication with the SSM. A reboot of the ASA could help to re-establish that communication. Of course, rebooting the ASA could have implications in your network and may need to wait until a scheduled downtime.


It could be that the SSM could be having trouble communicating with the ASA. Wait until the SSM goes to an Unresponsive state, and then then try a "reset" of the SSM.


The other possibility is that your SSM may have a hardware problem. Was this SSM working fine before you started trying to load a new version?


The other possibility is that 6.1(1)E2 file is not installing properly, but I am not aware of any known issues with this file.

There is always the small chance that you could be hitting a bug that no one else has seen before.

You could wait until the SSM goes Unresponsive, and then try loading the System Image file for the previous version you had running on this SSM. If you get the old version up and running, then see if you can do an "upgrade" instead of a System Image to get the SSM up to 6.1(1)E2.

Actions

This Discussion