Suddenly unable to ssh into ASA firewalls

Unanswered Question
Mar 31st, 2009
User Badges:

Hi,


We were able to ssh into the ASA management interfaces and all of sudden it has stopped working. The ssh client is putty. I ran a debug on the ASA while initiating ssh from putty and following is the output.


ANTIX-ASA/stby# Device ssh opened successfully.

SSH1: SSH client: IP = '192.168.1.50' interface # = 3

SSH: host key initialised

SSH1: starting SSH control process

SSH1: Exchanging versions - SSH-1.99-Cisco-1.25


SSH1: send SSH message: outdata is NULL


server version string:SSH-1.99-Cisco-1.25SSH1: Session disconnected by SSH serve

r - error 0x3c "Time-out activated"

SSH1: receive SSH message: [no message ID: variable *data is NULL]

SSH1: receive unsuccessful - status 0x3c


Any clue ?

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
cisco_lite Wed, 04/01/2009 - 01:34
User Badges:

I tried wireshark from the putty client machine and noticed malformed ssh packets being sent out.


I tried doing ssh into ASA from other cisco devices and it works.



cisco_lite Wed, 04/01/2009 - 17:09
User Badges:

I tried recreating the keys but no difference. When I move the host into the same segment as management interface of ASA it works fine.


Howevever, when it is behind FWSM and goes thru Cat65K SVI to ASA the problem occurs. The source IP is natted via overload on the Cat65 SVI. NAT is working fine as before.


I can also see many 'TCP Retransmission Encrypted Response Packet' and 'TCP Dup ACK' packets in ethereal.


I checked on the ASA 8.0 Buglist and closest that comes to it is


CSCsw85251 Bug Details Bug #76 of 79 | < Previous | Next >


Firewall interface stops responding to management traffic

Symptom:

A PIX or ASA firewall may no longer process management traffic destined to an interface correctly resulting in management services like SSH/Telnet/ASDM/SNMP failing to function.


Conditions:

This has been seen on multiple versions of PIX/ASA code including, but not limited to, 8.0.3.


Workaround:

When this happens a simple fix is to just simply re-apply the ip address command to the affected interfaces configuration


----


Yet, it doesn't seem to apply as I can ssh into the device from the management segment.


Please assist.

Fernando_Meza Wed, 04/01/2009 - 21:15
User Badges:
  • Gold, 750 points or more

Hi,


According to your troubleshooting steps it seems like your have already ruled out an issue with Putty. As you are trying to connect from behind the FWSM I would check to make sure access-list, NAT etc is configured correctly. Also I would make sure that packets are not trying to flow on an asymetric fashion through the FWSM. I would create a capture on the FWSM and try initiaitng an ssh connection to the ASA from behind the FWSM and see what the FWSM is doing ..


just some ideas !!!


cisco_lite Tue, 04/07/2009 - 02:07
User Badges:

Hi,


I did a capture on FWSM and below are the few lines. I can notice that the two ends are not agreeing on the window size. My laptop (192.168.1.10) is sending Window size of 65516 and the ASA replying with Window size of 8192 and it continues as such.


385: 12:17:56.1154400674 802.1Q vlan#90 P0 192.168.1.10.28839 > 10.0.0.50.22: . ack 3176026488 win 65516

395: 12:17:59.1154403624 802.1Q vlan#90 P0 192.168.1.10.28839 > 10.0.0.50.22: P 2610432943:2610433479(536) ack 3176026488 win 65516

424: 12:18:11.1154415664 802.1Q vlan#90 P0 192.168.1.10.28839 > 10.0.0.50.22: P 2610432943:2610433587(644) ack 3176026488 win 65516

462: 12:18:26.1154430584 802.1Q vlan#90 P0 10.0.0.50.22 > 192.168.1.10.28839: FP 3176026488:3176026488(0) ack 2610432943 win 8192



cisco_lite Tue, 04/07/2009 - 09:54
User Badges:

Ok.


I removed the PAT, configured static NAT instead and it worked. Still don't know why.


Topology:


VLAN1 -> FWSM -> Cat65K -> ASA


The source IP of the host in VLAN1 was PAT'ed on the SVI of Cat65K. It was working till recently and then stopped. But would could be the probable reasons for the NAT to work and not PAT via overload on the SVI of Cat65K.


With PAT the initial translation does happen successfully, the NAT table is updated, server/client ssh hello happens, debug ip nat shows that PAT is working correctly without any errors but then somewhere in between the Cat65K does not NAT anymore and then resumes after certain retransmission packets, and then DUP/ACK takes place and ssh session doesn't get established.

Actions

This Discussion