ASA 5520 + FTP over Explicit TLS/SSL Client

Answered Question
Apr 7th, 2010

Hi guys

Been working on this issue for a while now, and I don't know ASA very well to know where to troubleshoot.

Here's our issue: we have issues connecting to FTPES sites (FTP over Explicit TLS/SSL (port 21), not SFTP (port 22) ) from our network.

Normal passive FTP connection works fine from our network.

I have nailed it down to the ASA, because when I bypass the box, my test machine can connect to the FTPES site just fine.

Using Filezilla, from our network, we can connect to the site and log in, but it failed when trying to do directory listing.

Here's the log from Filezilla:

Command:   MLSD
Error:   GnuTLS error -53: Error in the push function.
Response:   425 Can't open data connection.
Error:   Failed to retrieve directory listing
Response:   421 Connection timed out.

I have searched Google and it all points to NAT issue.

The thing is, I don't think the connection from that machine is NAT-ed (when I go to sites like whatsmyip, it shows the IP address of the server I am on and not the router).

I have also opened up IP connection from the external interface (the one connected to our main router) to my test machine, still no go.

Stuck on the same place. Any ideas?

I have this problem too.
0 votes
Correct Answer by Poonguzhali Sankar about 4 years 5 days ago

    Class-map: class_ftp

      Match: port tcp range 1 1023

It appears that you are doing ftp inspection for tcp ports 1 through 1023. Pls. change it to just match port 21.

There is another class-map as well which is doing ftp inspection. I am not sure what port that is matching on.

Pls. post the output of "sh run class-map".

The fact that you are masking all the IP address makes hard to review the logs.

Capture on the client and server is the best option. Pls. download wireshark and get captues on the client and server.

Are you sure that there is no web-filter kind of thing that may be blocking FTP TLS negotiation? Clearly the conn gets torn down due to FINs.

Also FTP SSL is supposed to be on port 990. It is not the case in your setup? Also, since this is encrypted inspection cannot look within the packet and automatically allow the data connection, you need to allow that in the acl applied on the External interface which you have done.  Right?

http://en.wikipedia.org/wiki/FTPS

-KS

  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 4.8 (6 ratings)
Poonguzhali Sankar Wed, 04/07/2010 - 17:25

could you post the output of "sh run policy-map" ?

Do you have ftp inspection configured? This is needed for regular (clear) ftp. If the client is on the inside and server is no the outside, passive ftp will work as the server sends the port command the client on the inside (higher security) opens a new connection to the server's high port to receive the data. If the server were to start a brand new connection from the lower security interface to send data and if the ACL applied on the outside does not allow this, data connection will break and you will not be able to list directory. Pls. make sure the outside ACL allows the high ports that the server will try to connect back to the client on.

what do you see in the logs?

conf t

logging buffered 7

exit

sh logg | i x.x.x.x

where x.x.x.x is the ftp server IP address

-KS

jafaruddinlie Wed, 04/07/2010 - 18:09

Hi kusankar

Here's the output from show run policy-map:

!

policy-map global_policy

class inspection_default

  inspect netbios

class class_ftp

  inspect ftp

class class_ftp1

  inspect ftp

policy-map type inspect h323 H323_Map_100

description H323 Policy Map

parameters

  call-party-numbers

  h245-tunnel-block action log

  state-checking h225

  state-checking ras

  rtp-conformance enforce-payloadtype

match media-type audio

  drop log

match media-type video

  drop log

match media-type data

  drop log

policy-map type inspect netbios NetBIOS_Map_100

description NetBIOS Intrusion Detection Map

parameters

  protocol-violation action log

policy-map type inspect http HTTP-Map-100

parameters

  body-match-maximum 256

  protocol-violation action log

match req-resp content-type mismatch

  log

match request header content-type violation

  log

match response header content-type violation

  log

class _default_gator

  drop-connection log

class _default_kazaa

  drop-connection log

class _default_http-tunnel

  drop-connection log

class _default_gnu-http-tunnel

  drop-connection log

class _default_httport-tunnel

  drop-connection log

class _default_firethru-tunnel

  drop-connection log

class _default_GoToMyPC-tunnel

  drop-connection log

class _default_windows-media-player-tunnel

  drop-connection log

class _default_shoutcast-tunneling-protocol

  drop-connection log

class _default_msn-messenger

  drop-connection log

class _default_aim-messenger

  drop-connection log

class _default_yahoo-messenger

  drop-connection log

match request method post

  log

match request method put

  log

match request method trace

  log

match request method delete

  log

match request method options

  log

match request method connect

  log

match request method get

  log

match request method head

  log

match request method unlock

  log

match request method edit

  log

match request method save

  log

match request method mkdir

  log

match request method copy

  log

match request method lock

  log

match request method index

  log

match request method move

  log

match request method unedit

  log

match request header transfer-encoding chunked

  log

match response header transfer-encoding chunked

  log        

match request header transfer-encoding compress

  log

match response header transfer-encoding compress

  log

match request header transfer-encoding deflate

  log

match response header transfer-encoding deflate

  log

match request header transfer-encoding gzip

  log

match response header transfer-encoding gzip

  log

match request header transfer-encoding identity

  log

match response header transfer-encoding identity

  log

policy-map type inspect ftp FTP-Map-100

parameters

  mask-banner

  mask-syst-reply

policy-map type inspect dns DNS_Map_100

description DNS Policy_Map

parameters

  message-length maximum 512

  id-randomization

  id-mismatch action log

policy-map type inspect sip SIP_Map_100

description SIP Intrusion prevention Policy Map

parameters

  ip-address-privacy

  max-forwards-validation action log

  state-checking action drop-connection log

  software-version action mask log

  strict-header-validation action drop log

  no traffic-non-sip

  uri-non-sip action mask log

  rtp-conformance enforce-payloadtype

policy-map type inspect skinny SCCP_Map_100

description Skinny Protocol Inspection Map

parameters

  enforce-registration

  message-id max 0x141

  sccp-prefix-len max 65536

  timeout media 0:01:00

  timeout signaling 0:05:00

  rtp-conformance enforce-payloadtype

policy-map global-policy

description Policy for IPS Sensor

class global-class

  inspect sqlnet

  inspect xdmcp

  inspect tftp

  inspect icmp error

  inspect rtsp

  inspect sunrpc

  inspect mgcp

  inspect pptp

  inspect ctiqbe

  inspect rsh

  inspect icmp

  inspect ils

  inspect snmp SNMP-Map-100

  inspect http HTTP-Map-100

  inspect ftp strict FTP-Map-100

  inspect dns DNS_Map_100

  inspect netbios NetBIOS_Map_100

  inspect sip SIP_Map_100

  inspect skinny SCCP_Map_100

  inspect h323 h225 H323_Map_100

  inspect h323 ras H323_Map_100

class IPS_Class_10

  ips inline fail-open

  set connection advanced-options TCP-Norm-Map-100

policy-map type inspect im Instant_Messaging_Inspect_Map_100

description Policy Map to detect and log Instant Messaging and Tunnelling protocols within IM and HTTP tunnels

parameters

match protocol msn-im yahoo-im

  log

match version regex _default_GoToMyPC-tunnel

  log

match version regex _default_GoToMyPC-tunnel_2

  log

match version regex _default_aim-messenger

  log

match version regex _default_firethru-tunnel_1

  log

match version regex _default_firethru-tunnel_2

  log

match version regex _default_gator

  log

match version regex _default_gnu-http-tunnel_arg

  log

match version regex _default_gnu-http-tunnel_uri

  log

match version regex _default_http-tunnel

  log        

match version regex _default_httport-tunnel

  log

match version regex _default_icy-metadata

  log

match version regex _default_msn-messenger

  log

match version regex _default_shoutcast-tunneling-protocol

  log

match version regex _default_windows-media-player-tunnel

  log

match version regex _default_x-kazaa-network

  log

match service chat conference games voice-chat webcam

  log

!

So yeah, inspect ftp is running. As I mentioned before, regular FTP works fine.
I'll actually qualify that statement. Regular FTP works to public sites like ftp.microsoft.com, but since the site I am trying to connect to don't actually have a normal FTP site, I can't test.
Nothing from sh logg | i ftp.server.address, looks like I need help in setting up proper logging. I have tried this while trying to connect to that site, nothing on the log.
Poonguzhali Sankar Wed, 04/07/2010 - 18:17

Oh boy ! didn't expect this output.

I need "sh run service-policy" output.

To enable logging following this.

logging on

logging buffered 7

logging buffer-size 1048576

exit

clear logg buff

sh logg | i x.x.x.x

Do this (sh logg command) quickly when you try the ftp connection.  If this is a busy firewall, which I think it is, the buffer may wrap too quickly.

You can also collect the output of

sh service-policy flow tcp host a.a.a.a host x.x.x.x eq 21

where a.a.a.a is the client IP and x.x.x.x is the ftp server IP.

-KS

jafaruddinlie Wed, 04/07/2010 - 18:37

Firstly, thank you for your help so far.

Secondly, oops, my apologies for that. Ran the wrong command.

Here's probably what you want:

acerasa# sh run service-policy       

service-policy global_policy global

acerasa#

A bit of background: I am currently testing it from our Internal network.

x.x.x.x is the address of the FTP site we're testing.

y.y.y.y is the address of the interface labeled "External". That's the IP address we use to get out from our internal interface.

As you can see in the last line, something is denied. I think we're getting somewhere, if you can help me decipher this.

Here's an output from the log:

acerasa# sh logg | i x.x.x.x

<190>:Apr 08 11:23:43 EST: %ASA-session-6-302014: Teardown TCP connection 197913265 for External:x.x.x.x/21 to Internal:10.10.0.162/1144 duration 0:00:22 bytes 1712 TCP FINs

<190>:Apr 08 11:23:43 EST: %ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(2839) hit-cnt 1 first hit [0xfd329ad8, 0x0]

<190>:Apr 08 11:23:48 EST: %ASA-session-6-106100: access-list Internal_access_in permitted tcp Internal/10.10.0.162(1145) -> External/x.x.x.x(21) hit-cnt 1 first hit [0xd6d51b48, 0x4b81b8a0]

<190>:Apr 08 11:23:48 EST: %ASA-session-6-302013: Built outbound TCP connection 197913995 for External:x.x.x.x/21 (x.x.x.x/21) to Internal:10.10.0.162/1145 (y.y.y.y/20049)

<190>:Apr 08 11:24:09 EST: %ASA-session-6-302014: Teardown TCP connection 197913995 for External:x.x.x.x/21 to Internal:10.10.0.162/1145 duration 0:00:21 bytes 1712 TCP FINs

<190>:Apr 08 11:24:09 EST: %ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(20049) hit-cnt 1 first hit [0xfd329ad8, 0x0]

Poonguzhali Sankar Wed, 04/07/2010 - 18:59

ASA-session-6-302013: Built outbound TCP connection 197913995 for External:x.x.x.x/21 (x.x.x.x/21) to Internal:10.10.0.162/1145 (y.y.y.y/20049)

ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(20049)

So, the inside client 10.10.0.162 looks like the external interface y.y.y.y upon NAT and tries to talk to x.x.x.x on the outside of external?

Where is this External_access_in acl applied? "IN" bound on the External interface? Seems like the response is getting dropped when it should be automatically allowed. Internal is higher level security than external correct?

Did you get the sh service-policy flow command output?

-KS

jafaruddinlie Wed, 04/07/2010 - 19:10

"So, the inside client 10.10.0.162 looks like the external interface y.y.y.y upon NAT and tries to talk to x.x.x.x on the outside of external?"

Yes. That is correct.

"Where is this External_access_in acl applied? "IN" bound on the External interface? Seems like the response is getting dropped when it should be automatically allowed. Internal is higher level security than external correct?"

Yes, it is applied inbound on the External interface.

I tried putting a rule that would permit x.x.x.x to talk to y.y.y.y on high ports on the External interface, and the log changes to this instead:

<190>:Apr 08 11:54:00 EST: %ASA-session-6-106015: Deny TCP (no connection) from x.x.x.x/21 to y.y.y.y/51519 flags ACK  on interface External

<190>:Apr 08 11:54:00 EST: %ASA-session-6-106015: Deny TCP (no connection) from x.x.x.x/21 to y.y.y.y/51519 flags ACK  on interface External

Looks like NAT gone screwy?

How do I check the higher security level of each interface?

"Did you get the sh service-policy flow command output?"

Ah yes, here it is.

Global policy:

  Service-policy: global_policy

    Class-map: class_ftp

      Match: port tcp range 1 1023

      Action:

        Input flow:  inspect ftp

    Class-map: class-default

      Match: any

      Action:

Poonguzhali Sankar Wed, 04/07/2010 - 19:24

<190>:Apr 08 11:23:48 EST: %ASA-session-6-302013: Built outbound TCP connection 197913995 for External:x.x.x.x/21 (x.x.x.x/21) to Internal:10.10.0.162/1145 (y.y.y.y/20049)

<190>:Apr 08 11:24:09 EST: %ASA-session-6-302014: Teardown TCP connection 197913995 for External:x.x.x.x/21 to Internal:10.10.0.162/1145 duration 0:00:21 bytes 1712 TCP FINs

<190>:Apr 08 11:24:09 EST: %ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(20049) hit-cnt 1 first hit [0xfd329ad8, 0x0]

Here is the explanation.

1. We build a connection from internal to external - and hope the respose will automatically be allowed.

2. the connection gets torn down for TCP FINs reason

3. After which the server responds - too late see the time. So, by the time the response comes the connection is not there so, we decide to check the acl for permission and deny the flow.

4. Even after you added the permission - Now you got deny tcp 106015 meaning, the response is now allowed but the connection or request that is matching this flow does not exist on the firewall and was torn down.

packet captures (wireshark) on the client on the server both taken simultaneously will explain things further.

sh nameif

this above command will show you the security levels on the interfaces.

-KS

jafaruddinlie Wed, 04/07/2010 - 20:47

Thank you for that.

I have understood a heck a lot more now.

Half-way there, thanks to your help.

It is definitely a NAT issue now.

We have 4 interfaces on the ASA, external, internal, dmz1 and dmz2.

I have managed to connect the machine to the FTP site from dmz1 (not NAT-ed), all I have to do is allow the machine to connect to port 30000 - 60000 to the FTP server.

In the internal interface, the connection is NAT-ed.

I figured because this is FTP over TLS/SSL, somehow the routing detail must have gotten encrypted as well, hence why I am getting the deny tcp 106015.

Is there a way around this?

Correct Answer
Poonguzhali Sankar Wed, 04/07/2010 - 21:20

    Class-map: class_ftp

      Match: port tcp range 1 1023

It appears that you are doing ftp inspection for tcp ports 1 through 1023. Pls. change it to just match port 21.

There is another class-map as well which is doing ftp inspection. I am not sure what port that is matching on.

Pls. post the output of "sh run class-map".

The fact that you are masking all the IP address makes hard to review the logs.

Capture on the client and server is the best option. Pls. download wireshark and get captues on the client and server.

Are you sure that there is no web-filter kind of thing that may be blocking FTP TLS negotiation? Clearly the conn gets torn down due to FINs.

Also FTP SSL is supposed to be on port 990. It is not the case in your setup? Also, since this is encrypted inspection cannot look within the packet and automatically allow the data connection, you need to allow that in the acl applied on the External interface which you have done.  Right?

http://en.wikipedia.org/wiki/FTPS

-KS

jafaruddinlie Wed, 04/07/2010 - 21:41

acerasa# sh run class-map

!

class-map IPS_Class_10

match access-list global_mpc

class-map global-class

match default-inspection-traffic

class-map class_ftp

match port tcp range 1 1023

class-map class_ftp1

match port tcp range 1024 65535

class-map type inspect im match-all Instant_Messaging_Map_100

description Class Map for intstant Messaging

match service chat conference file-transfer games voice-chat webcam

match protocol msn-im yahoo-im

class-map inspection_default

match default-inspection-traffic

!

Unfortunately I don't have access to the FTP server (we're just connecting to them, it's not ours)
And I am not running FTPS, but FTPES (FTP over Explicit TLS/SSL). On the wikipage you linked, it is the first one under Method of Invoking:

Methods of invoking

Two separate methods were developed to invoke client security for use with FTP clients: Explicit or Implicit. The former method is a legacy compatible implementation where FTPS aware clients can invoke security with an FTPS aware server without breaking overall FTP functionality with non-FTPS aware clients. The latter method requires clients to be FTPS aware, and thus is incompatible with non-FTPS-aware clients.

[edit]Explicit

In explicit mode (also known as FTPES), an FTPS client must "explicitly request" security from an FTPS server and then step-up to a mutually agreed encryption method. If a client does not request security, the FTPS server can either allow the client to continue insecure or refuse/limit the connection.

The mechanism for negotiating authentication and security with FTP was added under RFC 2228, which included the new FTP command AUTH. While this RFC does not explicitly define any required security mechanisms (ie, SSL or TLS), it does require that the FTPS client will challenge the FTPS server with a mutually known mechanism. If the FTPS client challenges the FTPS server with an unknown security mechanism, the FTPS server will respond to the AUTH command with error code 504 (not supported). Clients could determine which mechanisms were supported by querying the FTPS server with the FEAT command, although it should be noted that servers are not necessarily required to be honest in disclosing what levels of security they support. Common methods of invoking FTPS security included: AUTH TLS and AUTH SSL.

In the later RFC 4217, FTPS compliance required that clients always negotiate using the AUTH TLS method. The RFC also recommends FTPS servers to accept the draft mechanism AUTH TLS-C.

We do have another layer of firewall between ASA and our Internal server, but I have allowed FTP and also high ports to the external FTP server. I'll try to get as many packet capture as possible.


"Also, since this is encrypted inspection cannot look within the packet and automatically allow the data connection, you need to allow that in the acl applied on the External interface which you have done.  Right?"

I have allowed incoming to our External interface ports 30000 - 60000, still getting this:

acerasa# sho logg | i 78.46.61.19

<190>:Apr 08 14:35:56 EST: %ASA-session-6-106100: access-list Internal_access_in permitted tcp Internal/10.10.0.162(1331) -> External/78.46.61.19(21) hit-cnt 1 first hit [0xd6d51b48, 0x4b81b8a0]

<190>:Apr 08 14:35:56 EST: %ASA-session-6-302013: Built outbound TCP connection 198370172 for External:78.46.61.19/21 (78.46.61.19/21) to Internal:10.10.0.162/1331 (203.xxx.xxx.xxx/44726)

<190>:Apr 08 14:36:19 EST: %ASA-session-6-302014: Teardown TCP connection 198370172 for External:78.46.61.19/21 to Internal:10.10.0.162/1331 duration 0:00:22 bytes 1712 TCP FINs

<190>:Apr 08 14:36:19 EST: %ASA-session-6-106015: Deny TCP (no connection) from 78.46.61.19/21 to 203.xxx.xxx.xxx/44726 flags ACK  on interface External

<190>:Apr 08 14:36:19 EST: %ASA-session-6-106015: Deny TCP (no connection) from 78.46.61.19/21 to 203.xxx.xxx.xxx/44726 flags ACK  on interface External

203.xxx.xxx.xxx is our ASA external interface.
78.46.61.19 is the FTP server in Germany we're trying to connect to.

jafaruddinlie Wed, 04/07/2010 - 22:32

SOLVED!!!!

Your last message about a filter got me thinking.

I had a closer look at the internal firewall that we have (Watchguard).

It turns out that it is doing some funky inspection stuff with FTP.

I created a new FTP rule and deleted the old one, voila, everything works now.

Thanks for your help, I do learn heaps from this

Poonguzhali Sankar Thu, 04/08/2010 - 05:27

Glad to hear. I was certain it had to do with someone tearing the connection down.

If it was NAT the connection would not have been built.

Anyway, pls. rate the posts that helped you.

-KS

Actions

Login or Register to take actions

This Discussion

Posted April 7, 2010 at 4:51 PM
Stats:
Replies:12 Avg. Rating:4.8
Views:6095 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard

Rank Username Points
1 7,861
2 6,140
3 3,165
4 1,473
5 1,446