04-07-2010 04:51 PM - edited 03-11-2019 10:30 AM
Hi guys
Been working on this issue for a while now, and I don't know ASA very well to know where to troubleshoot.
Here's our issue: we have issues connecting to FTPES sites (FTP over Explicit TLS/SSL (port 21), not SFTP (port 22) ) from our network.
Normal passive FTP connection works fine from our network.
I have nailed it down to the ASA, because when I bypass the box, my test machine can connect to the FTPES site just fine.
Using Filezilla, from our network, we can connect to the site and log in, but it failed when trying to do directory listing.
Here's the log from Filezilla:
Command: MLSD
Error: GnuTLS error -53: Error in the push function.
Response: 425 Can't open data connection.
Error: Failed to retrieve directory listing
Response: 421 Connection timed out.
I have searched Google and it all points to NAT issue.
The thing is, I don't think the connection from that machine is NAT-ed (when I go to sites like whatsmyip, it shows the IP address of the server I am on and not the router).
I have also opened up IP connection from the external interface (the one connected to our main router) to my test machine, still no go.
Stuck on the same place. Any ideas?
Solved! Go to Solution.
04-07-2010 09:20 PM
Class-map: class_ftp
Match: port tcp range 1 1023
It appears that you are doing ftp inspection for tcp ports 1 through 1023. Pls. change it to just match port 21.
There is another class-map as well which is doing ftp inspection. I am not sure what port that is matching on.
Pls. post the output of "sh run class-map".
The fact that you are masking all the IP address makes hard to review the logs.
Capture on the client and server is the best option. Pls. download wireshark and get captues on the client and server.
Are you sure that there is no web-filter kind of thing that may be blocking FTP TLS negotiation? Clearly the conn gets torn down due to FINs.
Also FTP SSL is supposed to be on port 990. It is not the case in your setup? Also, since this is encrypted inspection cannot look within the packet and automatically allow the data connection, you need to allow that in the acl applied on the External interface which you have done. Right?
http://en.wikipedia.org/wiki/FTPS
-KS
04-07-2010 05:25 PM
could you post the output of "sh run policy-map" ?
Do you have ftp inspection configured? This is needed for regular (clear) ftp. If the client is on the inside and server is no the outside, passive ftp will work as the server sends the port command the client on the inside (higher security) opens a new connection to the server's high port to receive the data. If the server were to start a brand new connection from the lower security interface to send data and if the ACL applied on the outside does not allow this, data connection will break and you will not be able to list directory. Pls. make sure the outside ACL allows the high ports that the server will try to connect back to the client on.
what do you see in the logs?
conf t
logging buffered 7
exit
sh logg | i x.x.x.x
where x.x.x.x is the ftp server IP address
-KS
04-07-2010 06:09 PM
Hi kusankar
Here's the output from show run policy-map:
!
policy-map global_policy
class inspection_default
inspect netbios
class class_ftp
inspect ftp
class class_ftp1
inspect ftp
policy-map type inspect h323 H323_Map_100
description H323 Policy Map
parameters
call-party-numbers
h245-tunnel-block action log
state-checking h225
state-checking ras
rtp-conformance enforce-payloadtype
match media-type audio
drop log
match media-type video
drop log
match media-type data
drop log
policy-map type inspect netbios NetBIOS_Map_100
description NetBIOS Intrusion Detection Map
parameters
protocol-violation action log
policy-map type inspect http HTTP-Map-100
parameters
body-match-maximum 256
protocol-violation action log
match req-resp content-type mismatch
log
match request header content-type violation
log
match response header content-type violation
log
class _default_gator
drop-connection log
class _default_kazaa
drop-connection log
class _default_http-tunnel
drop-connection log
class _default_gnu-http-tunnel
drop-connection log
class _default_httport-tunnel
drop-connection log
class _default_firethru-tunnel
drop-connection log
class _default_GoToMyPC-tunnel
drop-connection log
class _default_windows-media-player-tunnel
drop-connection log
class _default_shoutcast-tunneling-protocol
drop-connection log
class _default_msn-messenger
drop-connection log
class _default_aim-messenger
drop-connection log
class _default_yahoo-messenger
drop-connection log
match request method post
log
match request method put
log
match request method trace
log
match request method delete
log
match request method options
log
match request method connect
log
match request method get
log
match request method head
log
match request method unlock
log
match request method edit
log
match request method save
log
match request method mkdir
log
match request method copy
log
match request method lock
log
match request method index
log
match request method move
log
match request method unedit
log
match request header transfer-encoding chunked
log
match response header transfer-encoding chunked
log
match request header transfer-encoding compress
log
match response header transfer-encoding compress
log
match request header transfer-encoding deflate
log
match response header transfer-encoding deflate
log
match request header transfer-encoding gzip
log
match response header transfer-encoding gzip
log
match request header transfer-encoding identity
log
match response header transfer-encoding identity
log
policy-map type inspect ftp FTP-Map-100
parameters
mask-banner
mask-syst-reply
policy-map type inspect dns DNS_Map_100
description DNS Policy_Map
parameters
message-length maximum 512
id-randomization
id-mismatch action log
policy-map type inspect sip SIP_Map_100
description SIP Intrusion prevention Policy Map
parameters
ip-address-privacy
max-forwards-validation action log
state-checking action drop-connection log
software-version action mask log
strict-header-validation action drop log
no traffic-non-sip
uri-non-sip action mask log
rtp-conformance enforce-payloadtype
policy-map type inspect skinny SCCP_Map_100
description Skinny Protocol Inspection Map
parameters
enforce-registration
message-id max 0x141
sccp-prefix-len max 65536
timeout media 0:01:00
timeout signaling 0:05:00
rtp-conformance enforce-payloadtype
policy-map global-policy
description Policy for IPS Sensor
class global-class
inspect sqlnet
inspect xdmcp
inspect tftp
inspect icmp error
inspect rtsp
inspect sunrpc
inspect mgcp
inspect pptp
inspect ctiqbe
inspect rsh
inspect icmp
inspect ils
inspect snmp SNMP-Map-100
inspect http HTTP-Map-100
inspect ftp strict FTP-Map-100
inspect dns DNS_Map_100
inspect netbios NetBIOS_Map_100
inspect sip SIP_Map_100
inspect skinny SCCP_Map_100
inspect h323 h225 H323_Map_100
inspect h323 ras H323_Map_100
class IPS_Class_10
ips inline fail-open
set connection advanced-options TCP-Norm-Map-100
policy-map type inspect im Instant_Messaging_Inspect_Map_100
description Policy Map to detect and log Instant Messaging and Tunnelling protocols within IM and HTTP tunnels
parameters
match protocol msn-im yahoo-im
log
match version regex _default_GoToMyPC-tunnel
log
match version regex _default_GoToMyPC-tunnel_2
log
match version regex _default_aim-messenger
log
match version regex _default_firethru-tunnel_1
log
match version regex _default_firethru-tunnel_2
log
match version regex _default_gator
log
match version regex _default_gnu-http-tunnel_arg
log
match version regex _default_gnu-http-tunnel_uri
log
match version regex _default_http-tunnel
log
match version regex _default_httport-tunnel
log
match version regex _default_icy-metadata
log
match version regex _default_msn-messenger
log
match version regex _default_shoutcast-tunneling-protocol
log
match version regex _default_windows-media-player-tunnel
log
match version regex _default_x-kazaa-network
log
match service chat conference games voice-chat webcam
log
!
04-07-2010 06:17 PM
Oh boy ! didn't expect this output.
I need "sh run service-policy" output.
To enable logging following this.
logging on
logging buffered 7
logging buffer-size 1048576
exit
clear logg buff
sh logg | i x.x.x.x
Do this (sh logg command) quickly when you try the ftp connection. If this is a busy firewall, which I think it is, the buffer may wrap too quickly.
You can also collect the output of
sh service-policy flow tcp host a.a.a.a host x.x.x.x eq 21
where a.a.a.a is the client IP and x.x.x.x is the ftp server IP.
-KS
04-07-2010 06:37 PM
Firstly, thank you for your help so far.
Secondly, oops, my apologies for that. Ran the wrong command.
Here's probably what you want:
acerasa# sh run service-policy
service-policy global_policy global
acerasa#
A bit of background: I am currently testing it from our Internal network.
x.x.x.x is the address of the FTP site we're testing.
y.y.y.y is the address of the interface labeled "External". That's the IP address we use to get out from our internal interface.
As you can see in the last line, something is denied. I think we're getting somewhere, if you can help me decipher this.
Here's an output from the log:
acerasa# sh logg | i x.x.x.x
<190>:Apr 08 11:23:43 EST: %ASA-session-6-302014: Teardown TCP connection 197913265 for External:x.x.x.x/21 to Internal:10.10.0.162/1144 duration 0:00:22 bytes 1712 TCP FINs
<190>:Apr 08 11:23:43 EST: %ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(2839) hit-cnt 1 first hit [0xfd329ad8, 0x0]
<190>:Apr 08 11:23:48 EST: %ASA-session-6-106100: access-list Internal_access_in permitted tcp Internal/10.10.0.162(1145) -> External/x.x.x.x(21) hit-cnt 1 first hit [0xd6d51b48, 0x4b81b8a0]
<190>:Apr 08 11:23:48 EST: %ASA-session-6-302013: Built outbound TCP connection 197913995 for External:x.x.x.x/21 (x.x.x.x/21) to Internal:10.10.0.162/1145 (y.y.y.y/20049)
<190>:Apr 08 11:24:09 EST: %ASA-session-6-302014: Teardown TCP connection 197913995 for External:x.x.x.x/21 to Internal:10.10.0.162/1145 duration 0:00:21 bytes 1712 TCP FINs
<190>:Apr 08 11:24:09 EST: %ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(20049) hit-cnt 1 first hit [0xfd329ad8, 0x0]
04-07-2010 06:59 PM
ASA-session-6-302013: Built outbound TCP connection 197913995 for External:x.x.x.x/21 (x.x.x.x/21) to Internal:10.10.0.162/1145 (y.y.y.y/20049)
ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(20049)
So, the inside client 10.10.0.162 looks like the external interface y.y.y.y upon NAT and tries to talk to x.x.x.x on the outside of external?
Where is this External_access_in acl applied? "IN" bound on the External interface? Seems like the response is getting dropped when it should be automatically allowed. Internal is higher level security than external correct?
Did you get the sh service-policy flow command output?
-KS
04-07-2010 07:10 PM
"So, the inside client 10.10.0.162 looks like the external interface y.y.y.y upon NAT and tries to talk to x.x.x.x on the outside of external?"
Yes. That is correct.
"Where is this External_access_in acl applied? "IN" bound on the External interface? Seems like the response is getting dropped when it should be automatically allowed. Internal is higher level security than external correct?"
Yes, it is applied inbound on the External interface.
I tried putting a rule that would permit x.x.x.x to talk to y.y.y.y on high ports on the External interface, and the log changes to this instead:
<190>:Apr 08 11:54:00 EST: %ASA-session-6-106015: Deny TCP (no connection) from x.x.x.x/21 to y.y.y.y/51519 flags ACK on interface External
<190>:Apr 08 11:54:00 EST: %ASA-session-6-106015: Deny TCP (no connection) from x.x.x.x/21 to y.y.y.y/51519 flags ACK on interface External
Looks like NAT gone screwy?
How do I check the higher security level of each interface?
"Did you get the sh service-policy flow command output?"
Ah yes, here it is.
Global policy:
Service-policy: global_policy
Class-map: class_ftp
Match: port tcp range 1 1023
Action:
Input flow: inspect ftp
Class-map: class-default
Match: any
Action:
04-07-2010 07:24 PM
<190>:Apr 08 11:23:48 EST: %ASA-session-6-302013: Built outbound TCP connection 197913995 for External:x.x.x.x/21 (x.x.x.x/21) to Internal:10.10.0.162/1145 (y.y.y.y/20049)
<190>:Apr 08 11:24:09 EST: %ASA-session-6-302014: Teardown TCP connection 197913995 for External:x.x.x.x/21 to Internal:10.10.0.162/1145 duration 0:00:21 bytes 1712 TCP FINs
<190>:Apr 08 11:24:09 EST: %ASA-session-6-106100: access-list External_access_in denied tcp External/x.x.x.x(21) -> Internal/y.y.y.y(20049) hit-cnt 1 first hit [0xfd329ad8, 0x0]
Here is the explanation.
1. We build a connection from internal to external - and hope the respose will automatically be allowed.
2. the connection gets torn down for TCP FINs reason
3. After which the server responds - too late see the time. So, by the time the response comes the connection is not there so, we decide to check the acl for permission and deny the flow.
4. Even after you added the permission - Now you got deny tcp 106015 meaning, the response is now allowed but the connection or request that is matching this flow does not exist on the firewall and was torn down.
packet captures (wireshark) on the client on the server both taken simultaneously will explain things further.
sh nameif
this above command will show you the security levels on the interfaces.
-KS
04-07-2010 08:47 PM
Thank you for that.
I have understood a heck a lot more now.
Half-way there, thanks to your help.
It is definitely a NAT issue now.
We have 4 interfaces on the ASA, external, internal, dmz1 and dmz2.
I have managed to connect the machine to the FTP site from dmz1 (not NAT-ed), all I have to do is allow the machine to connect to port 30000 - 60000 to the FTP server.
In the internal interface, the connection is NAT-ed.
I figured because this is FTP over TLS/SSL, somehow the routing detail must have gotten encrypted as well, hence why I am getting the deny tcp 106015.
Is there a way around this?
04-07-2010 09:20 PM
Class-map: class_ftp
Match: port tcp range 1 1023
It appears that you are doing ftp inspection for tcp ports 1 through 1023. Pls. change it to just match port 21.
There is another class-map as well which is doing ftp inspection. I am not sure what port that is matching on.
Pls. post the output of "sh run class-map".
The fact that you are masking all the IP address makes hard to review the logs.
Capture on the client and server is the best option. Pls. download wireshark and get captues on the client and server.
Are you sure that there is no web-filter kind of thing that may be blocking FTP TLS negotiation? Clearly the conn gets torn down due to FINs.
Also FTP SSL is supposed to be on port 990. It is not the case in your setup? Also, since this is encrypted inspection cannot look within the packet and automatically allow the data connection, you need to allow that in the acl applied on the External interface which you have done. Right?
http://en.wikipedia.org/wiki/FTPS
-KS
04-07-2010 09:41 PM
acerasa# sh run class-map
!
class-map IPS_Class_10
match access-list global_mpc
class-map global-class
match default-inspection-traffic
class-map class_ftp
match port tcp range 1 1023
class-map class_ftp1
match port tcp range 1024 65535
class-map type inspect im match-all Instant_Messaging_Map_100
description Class Map for intstant Messaging
match service chat conference file-transfer games voice-chat webcam
match protocol msn-im yahoo-im
class-map inspection_default
match default-inspection-traffic
!
Two separate methods were developed to invoke client security for use with FTP clients: Explicit or Implicit. The former method is a legacy compatible implementation where FTPS aware clients can invoke security with an FTPS aware server without breaking overall FTP functionality with non-FTPS aware clients. The latter method requires clients to be FTPS aware, and thus is incompatible with non-FTPS-aware clients.
In explicit mode (also known as FTPES), an FTPS client must "explicitly request" security from an FTPS server and then step-up to a mutually agreed encryption method. If a client does not request security, the FTPS server can either allow the client to continue insecure or refuse/limit the connection.
The mechanism for negotiating authentication and security with FTP was added under RFC 2228, which included the new FTP command AUTH. While this RFC does not explicitly define any required security mechanisms (ie, SSL or TLS), it does require that the FTPS client will challenge the FTPS server with a mutually known mechanism. If the FTPS client challenges the FTPS server with an unknown security mechanism, the FTPS server will respond to the AUTH command with error code 504 (not supported). Clients could determine which mechanisms were supported by querying the FTPS server with the FEAT command, although it should be noted that servers are not necessarily required to be honest in disclosing what levels of security they support. Common methods of invoking FTPS security included: AUTH TLS and AUTH SSL.
In the later RFC 4217, FTPS compliance required that clients always negotiate using the AUTH TLS method. The RFC also recommends FTPS servers to accept the draft mechanism AUTH TLS-C.
We do have another layer of firewall between ASA and our Internal server, but I have allowed FTP and also high ports to the external FTP server. I'll try to get as many packet capture as possible.
"Also, since this is encrypted inspection cannot look within the packet and automatically allow the data connection, you need to allow that in the acl applied on the External interface which you have done. Right?"
I have allowed incoming to our External interface ports 30000 - 60000, still getting this:
acerasa# sho logg | i 78.46.61.19
<190>:Apr 08 14:35:56 EST: %ASA-session-6-106100: access-list Internal_access_in permitted tcp Internal/10.10.0.162(1331) -> External/78.46.61.19(21) hit-cnt 1 first hit [0xd6d51b48, 0x4b81b8a0]
<190>:Apr 08 14:35:56 EST: %ASA-session-6-302013: Built outbound TCP connection 198370172 for External:78.46.61.19/21 (78.46.61.19/21) to Internal:10.10.0.162/1331 (203.xxx.xxx.xxx/44726)
<190>:Apr 08 14:36:19 EST: %ASA-session-6-302014: Teardown TCP connection 198370172 for External:78.46.61.19/21 to Internal:10.10.0.162/1331 duration 0:00:22 bytes 1712 TCP FINs
<190>:Apr 08 14:36:19 EST: %ASA-session-6-106015: Deny TCP (no connection) from 78.46.61.19/21 to 203.xxx.xxx.xxx/44726 flags ACK on interface External
<190>:Apr 08 14:36:19 EST: %ASA-session-6-106015: Deny TCP (no connection) from 78.46.61.19/21 to 203.xxx.xxx.xxx/44726 flags ACK on interface External
04-07-2010 10:32 PM
SOLVED!!!!
Your last message about a filter got me thinking.
I had a closer look at the internal firewall that we have (Watchguard).
It turns out that it is doing some funky inspection stuff with FTP.
I created a new FTP rule and deleted the old one, voila, everything works now.
Thanks for your help, I do learn heaps from this
04-08-2010 05:27 AM
Glad to hear. I was certain it had to do with someone tearing the connection down.
If it was NAT the connection would not have been built.
Anyway, pls. rate the posts that helped you.
-KS
05-11-2018 09:23 AM
Is there any compromise in bypassing the inspection of FTP over Explicit TLS/SSL Client on firewall? as i have ASA firewall waiting your reply.
if inspection required to be kept then is there any way that Firewall doesn't restrict the connection
awaiting your reply.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: