ASA 5505: Outbound SMTP Route Problem (rDNS)

[newmindeye]

Hi All.

Have seen a similar issue but am not sure how to approach the problem given my setup.

The specific issue is that when any of the following occurs:

1) remote users connect/authenticate to the mail server over smtp

2) an internal app server relays outbound mail via mail server (both on DMZ)

3) a VPN user (me) port forwards localhost 25 of my laptop over SSH to send mail (i.e. workaround ISP blocking port 25)

then outbound smtp traffic is matched to the external IP of the ASA itself, and not the mail server. The end result is that there is a reverse DNS mismatch (rDNS exists on mail server domain, not ASA external IP) which is causing some remote mail servers to reject the mail.

What I would like to have happen is for outbound smtp traffic to route via the external IP of the mail server by default. I am using object-groups, but can define manual access-list, static(dmz,outside) entries for each port if that is what is required to get this working.

Applicable config:

*************
object-group network web-services
network-object host xx.xxx.xx.101
network-object host xx.xxx.xx.102
...
object-group service open-tcp tcp
port-object eq smtp
...

access-list out_in extended permit tcp any object-group web-services object-group open-tcp
...
global (outside) 1 interface
global (dmz) 1 interface
nat (Inside) 0 access-list nonat
nat (Inside) 1 10.1.0.0 255.255.0.0
nat (dmz) 0 access-list nonat
nat (dmz) 1 0.0.0.0 0.0.0.0
...

static (dmz,outside) xx.xxx.xx.101 172.16.20.2 netmask 255.255.255.255 
static (dmz,outside) xx.xxx.xx.102 172.16.20.3 netmask 255.255.255.255
...

access-group out_in in interface outside
route outside 0.0.0.0 0.0.0.0 xx.xxx.xx.97 1
*************

Thanks for suggestions

[Jouni Forss]

Hi,

Are you saying that the problem is that your DMZ servers outbound connections wouldnt be visible with the public IP address configured in the "static" NAT configuration?

Just wondering as there arent really many things that should override Static NAT. Basically Static Policy NAT should do that BUT only if its before the Static NAT in order.

You can use "packet-tracer" command to confirm which translation is applied to certain traffic/packets

packet-tracer input dmz tcp 12345

- Jouni

[newmindeye]

Jouni, hi, thanks for the reply.

Mail flows in and out no problem. The issue is that in all 3 scenarios listed in my original post, outbound smtp traffic is bound to the external IP of the ASA itself.

I can see why this happens when connected over VPN, I port forward localhost 25 from my laptop over SSH. In this scenario I am sending mail directly from internal to external; there is no static routing rule for this, so the default routing policy (external IP of ASA) is used.

Same as VPN user above, when an internal app server relays via mail server, the traffic originates internally and therefore default routing policy takes precedence again.

I need to prevent internally based smtp traffic from going out on external IP of ASA.

Also, I need to figure out why remote users authenticating on the mail server at mail.clientsite.com, are also having outbound SMTP traffic sent out on external IP of ASA -- this traffic "should" be matched by static NAT rule and and go out on external IP of mail.clientsite.com.

I would think there would be a way to force route all outbound smtp traffic via external IP of the mail server, no?

[newmindeye]

Interesting, packet tracer results in Allow action for phases 1-6, but a Drop action result in phase 7 for all DMZ addresses except for default DMZ address 172.16.20.1 of the mail server.

Drop-reason: (sp-security-failed) Slowpath security checks failed

Very strange that 172.16.20.1 would allow smtp traffic in/out since I have no static NAT rule in place on this address and port, only DNS server on port 53.

[Jouni Forss]

Hi,

Sorry, I am not sure if I get the whole picture even yet

But in general, if traffic is originating from the hosts which have Static NAT then the public IP address for outbound and inbound connections should always be the one in the Static NAT configurations.

If you have any other hosts that have outbound SMTP traffic for example and want to have that traffic also use the public IP addresses defined in the Static NAT configurations then you will have to configure some sort of Dynamic Policy PAT configurations just for the SMTP traffic.

The above mentioned Dynamic Policy PAT configurations is very easy to configure. My problem at the moment is understanding exactly what hosts need to have this Dynamic Policy PAT performed.

Or maybe I have understood something completely wrong

- Jouni

[newmindeye]

I can't really go into much more detail, I've explained the situation as I understand it.

Maybe this will help though. Here I am sending mail from a client site email to my personal gmail account. This is what gmail receives:

Return-Path: 
Received: from mymailserver.com (xx.xxx.xx.98.my.isp.com. [xx.xxx.xx.98])
        by mx.google.com

So, .98 external IP above is default IP of the ASA; my ISP supplies a (useless) reverse DNS entry for this IP which has nothing to do with mymailserver.com with external IP on .101

Gmail accepts the mail but some external mail servers are rejecting mail sent from our mail server due to a rDNS mismatch.

Painful figuring this stuff out. I'd just like ALL outbound smtp traffic to be routed via external IP of our mail server (on .101) and NOT external IP of the ASA (on .98).

If you have an idea how to pull this off, let me know ;-)

Thanks

[Jouni Forss · Accepted solution]

Hi,

This would require that I know a bit more about your setup.

I would need to know

• What are the interfaces behind which hosts are located that need to show up as the Mail Server public IP when initiating outbound SMTP connections?
• What are the IP addresses/networks of these hosts

Lets take an example where I presume that we have a network 10.10.10.0/24 behind the "inside" interface and want to do Dynamic Policy PAT for the whole network when initiating outbound SMTP connections.

access-list INSIDE-SMTP-POLICYPAT remark Outbound SMTP Policy PAT

access-list INSIDE-SMTP-POLICYPAT permit tcp 10.10.10.0 255.255.255.0 any eq smtp

global (outside) 25 x.x.x.101

nat (inside) 25 access-list INSIDE-SMTP-POLICYPAT

The above setup would apply to most of other cases. There would be some problems if there are host with Static NAT configured towards the "outside" interface. In that case I am not sure if we could do this. Static NAT overrides Dynamic Policy PAT in the NAT ordering of ASA.

- Jouni

[newmindeye]

Hmmm, thanks for the idea: how will dynamic PAT fly with object-groups and static NAT as listed in my original post?

My network consists of:

1) private vlan on 10.1.1.1 that VPN comes in on.

2) dmz vlan on 172.16.1.1

3) outside vlan on xx.xxx.xx.98

All static NAT entries point from outside to private or dmz vlan; as far as I can tell there are no NAT entries pointing to the outside.

In my original post, I have:

global (outside) 1 interface
global (dmz) 1 interface
nat (Inside) 0 access-list nonat
nat (Inside) 1 10.1.0.0 255.255.0.0
nat (dmz) 0 access-list nonat
nat (dmz) 1 0.0.0.0 0.0.0.0

This entry looks like a catchall:

nat (dmz) 1 0.0.0.0 0.0.0.0

Anyway, looks like we may be getting closer

[Jouni Forss]

Hi,

I would need to clarification related to the VPN you mention.

Do you mean that the ASA a VPN connection (either Client VPN or L2L VPN) from which you should be able to initiate outbound SMTP connections using the correct public NAT IP address?

Or do you perhaps mean that there is some VPN device in addition to the ASA firewall which provides VPN connectivity (Client VPN or L2L VPN) from which you should be able to initiate outbound SMTP connections using the correct public NAT IP address?

The most important thing to know with regards to building the NAT configurations

• The interface "nameif" of ALL the interfaces behind which hosts needing this NAT are located
• All the networks/subnets behind those specific interface which need this NAT

With regards to the "object-group" question,

You have the "object-group" configurations to both group the public NAT IP addresses of the Mail servers and a separate "object-group" to hold SMTP service. These are then used in the ACL that allows traffic to those servers. This ACL only applies to connection inbound from the public network.

The interface ACL will have nothing to do with the connections the LAN hosts form outbound (as return traffic IS NOTmatched agains the WAN interface ACL)

On the other hand, when building the Dynamic Policy PAT rules mentioned in my earlier post, you can use the "object-group" configurations to group source addresses/networks and services if needed for the actual Dynamic Policy PAT rule.

- Jouni

[newmindeye]

Thanks for hanging in there Jouni ;-)

No separate VPN device, VPN provided by ASA itself, I VPN in on Inside 10.1.1.1 network below.

Network graph is simple (255.255.0.0 mask):

nameIf Inside    = 10.1.1.1

nameIf dmz      = 172.16.1.1

nameIf outside  = xx.xxx.xx.98

Outbound mail on dmz for internal app servers and remote clients authenticating over smtp.

Outbound mail on Inside for VPN users (currently only me) that port forward localhost 25 over SSH

DMZ servers live on 172.16.xx.xx 255.255.0.0

So, for example I have a Java app server on 172.16.40.1 which relays mail through mail server on 172.16.20.2.

I would be inclined to try the following:

access-list INSIDE-SMTP-POLICYPAT remark Outbound SMTP Policy PAT

access-list INSIDE-SMTP-POLICYPAT permit tcp 172.16.1.1 255.255.0.0 any eq smtp

global (outside) 25 x.x.x.101

nat (dmz) 25 access-list INSIDE-SMTP-POLICYPAT

With the hope that all outbound smtp traffic bound to DMZ address range would be routed on .101 address of the mail server.

In the end I think part of the problem is that the mail server software sends out all relay clients on localhost, while ASA is expecting the traffic to come out on 172.16.20.2, 3, 4, etc. of target client domain, and therefore chooses default DMZ route as a fallback.

[Jouni Forss · Accepted solution]

Hi,

If you have VPN Client configured on the ASA then those VPN Clients have their own VPN Pool network which has not yet been mentioned. With regards to ASAs routing, the routing table will view those VPN Client Pool IP addresses located behind the "outside" interface rather than the "inside" interface.

If your aim with the above configuration was to configure so that the whole DMZ networks outbound SMTP connections would be Dynamic PATed to the IP address x.x.x.101 then you have to make slight modification.

• Chance the ACL name of refer to the DMZ (only cosmetic meaning)
• Change the source in the ACL to refer to the network address and mask

access-list DMZ-SMTP-POLICYPAT remark Outbound SMTP Policy PAT

access-list DMZ-SMTP-POLICYPAT permit tcp 172.16.0.0 255.255.0.0 any eq smtp

global (outside) 25 x.x.x.101

nat (dmz) 25 access-list DMZ-SMTP-POLICYPAT

This Dynamic Policy PAT configuration would only apply to hosts that DO NOT have Static NAT / Static Policy NAT configured (as it would override this NAT)

Now, if your aim would also be to apply to the same Dynamic Policy PAT for the whole INSIDE network of 10.1.0.0/16 then you could add the following

access-list INSIDE-SMTP-POLICYPAT remark Outbound SMTP Policy PAT

access-list INSIDE-SMTP-POLICYPAT permit tcp 10.1.0.0 255.255.0.0 any eq smtp

nat (inside) 25 access-list INSIDE-SMTP-POLICYPAT

Also if the VPN Client users (localted behind "outside" interface) also formed outbound SMTP connections through the ASA then this would mean the following things

• You would have to configure Full Tunnel VPN where all traffic is forwarded through the VPN while its active
• For traffic to come from the VPN Client to the ASA and head back out from the "outside" interface through the ASAs Internet connection you would need a similiar NAT configuration for the "outside" interface also and also an additional configuration to allow the traffic to enter and leave the same interface. This command is mentioned first below.

same-security-traffic permit intra-interface

access-list OUTSIDE-SMTP-POLICYPAT remark Outbound SMTP Policy PAT

access-list OUTSIDE-SMTP-POLICYPAT permit tcp any eq smtp

nat (outside) 25 access-list OUTSIDE-SMTP-POLICYPAT

What still throws me of is the fact that you mention something about using the SSH port for SMTP traffic? Yet I dont see any configuration referring to that. No other local network behind the ASA seems to be doing any sort of Static PAT to manipulate the port.

But again, I must just not have understood something.

- Jouni

[newmindeye]

Jouni, let me start with your last point first.

I'm on the road a lot; most ISPs block outgoing smtp traffic and force you to have an email account in their system (e.g. me@isp.com). To get around this, on my laptop I connect to the mail server over SSH and port forward laptop localhost 25 to localhost port 25 of the mail server. This allows me to send outbound email as me@mycompany.com.

access-list DMZ-SMTP-POLICYPAT remark Outbound SMTP Policy PAT
access-list DMZ-SMTP-POLICYPAT permit tcp 172.16.0.0 255.255.0.0 any eq smtp
global (outside) 25 x.x.x.101
nat (dmz) 25 access-list DMZ-SMTP-POLICYPAT
This  Dynamic Policy PAT configuration would only apply to hosts that DO NOT  have Static NAT / Static Policy NAT configured (as it would override  this NAT)

Hmmmm, unless this only applies to outbound traffic, looks like it will interfere with my object-group services and static NAT mapping to dmz addresses as per this config that I have for several different external IPs:

object-group network web-services
network-object host xx.xxx.xx.101
 ...
object-group service open-tcp tcp
port-object eq smtp
port-object eq www
...
access-list out_in extended permit tcp any object-group web-services object-group open-tcp
static (dmz,outside) xx.xxx.xx.101 172.16.20.2 netmask 255.255.255.255

What affect will adding your dynamic PAT below

global (outside) 25 x.x.x.101
nat (dmz) 25 access-list DMZ-SMTP-POLICYPAT

on my existing global, nat entries here?

global (outside) 1 interface
global (dmz) 1 interface
nat (Inside) 0 access-list nonat
nat (Inside) 1 10.1.0.0 255.255.0.0
nat (dmz) 0 access-list nonat
nat (dmz) 1 0.0.0.0 0.0.0.0

I suppose I can give it a shot, just don't want to destroy a mostly functional network in the process ;-)

VPN user can access both dmz and Inside networks -- VPN Config:

**************

access-list MyCompanyVPN_splitTunnelAcl standard permit 10.1.0.0 255.255.0.0

access-list MyCompanyVPN_splitTunnelAcl standard permit 172.16.0.0 255.255.0.0

...

group-policy MyCompanyVPN internal

group-policy MyCompanyVPN attributes

vpn-tunnel-protocol IPSec

split-tunnel-policy tunnelspecified

split-tunnel-network-list value MyCompanyVPN_splitTunnelAcl

username someuser1 password ***** encrypted

username someuser2 password ***** encrypted

tunnel-group MyCompanyVPN type remote-access

tunnel-group  MyCompanyVPN general-attributes

address-pool RemoteClientsPool

default-group-policy MyCompanyVPN

tunnel-group MyCompanyVPN ipsec-attributes

pre-shared-key *****

******************

[Jouni Forss]

Hi,

The Dynamic Policy PAT will NOT apply to any host which has Static NAT configuration. The Static NAT always overrides Dynamic NAT/PAT configuration and will always be used. So even if you configure Dynamic Policy PAT for outbound connectivity and a host has a Static NAT, it will always use the Static NAT IP.

As you can see, you already have normal Dynamic PAT which applies to the server which also have Static NAT configured. Yet, since Static NAT overrides Dynamic PAT/NAT it means that the servers will always use their Static NAT IP. (between the interfaces specified in the NAT configurations)

If we look at the NAT existing NAT configurations there we can determine the following

• NAT0 configuration on DMZ and INSIDE will always be matched before ANY other NAT configuration on the ASA
• Dynamic Policy PAT (if added) will apply for all outbound SMTP traffic towards the "outside" interface.
• All other traffic from both DMZ and INSIDE outbound to "outside" will use the interface IP address of "outside" as their Dynamic PAT address

Looking at your VPN configuration it seems to be using Split Tunnel. This means that you will only forward traffic towards your INSIDE and DMZ networks to the ASA through the VPN. Any connections towards any other destination IP address will use the local Internet connection of your client/host computer.

So that in mind, the VPN Client (when connected) wont send any SMTP traffic that would come directly to ASA and which ASA would forward outbound through "outside" interface. This because only traffic destined to INSIDE and DMZ are coming from the VPN Client to the ASA, nothing headed to any public IP addresses.

- Jouni

[newmindeye]

Jouni, thanks for all the detail here.

I am seeing the reverse DNS mismatch in all 3 scenarios listed in original email (connected over vpn; dmz app server relay through mail server; remote user authenticate to mail server).

I can see that the mail server itself sends all outbound traffic on localhost (127.0.0.1) which is on the primary server NIC, eth0.

What must be happening is that inbound smtp traffic comes in correctly based on static NAT rule (e.g. 172.16.20.3), but then goes out from the mail server on ASA default DMZ address, 172.16.1.1, which of course routes to external IP of the ASA itself.

Since static NAT overrides dynamic PAT I'm not sure what I can do. Perhaps I can remove static NAT for smtp traffic entirely and then PAT inbound smtp traffic to mail server DMZ address, and PAT outbound all smtp traffic to external IP of the mail server.

Not sure if that will break things ;-)

[Jouni Forss · Accepted solution]

Hi,

If the ASAs interface IP address is 172.16.1.1 then NO SMTP traffic will be originated from its interface IP address (as this is traffic generated by hosts/server). The IP address 172.16.1.1 in that case would only be the next hop/default gateway for the DMZ servers.

Having the Static NAT for the actual DMZ Mail Server and Dynamic Policy PAT to the same public IP address for outbound SMTP connections should mean that absolutely no internal host should be using the ASA "outside" interface IP address as the PAT IP address for outbound SMTP connections

If outbound SMTP connections are visible to the public network with the "outside" interface IP address then it simply means that the connections are hitting the normal Dynamic PAT rule mentioned below

global (outside) 1 interface
nat (Inside) 1 10.1.0.0 255.255.0.0
nat (dmz) 1 0.0.0.0 0.0.0.0

If you have the above Dynamic PAT and the new Dynamic Policy PAT configured then the Dynamic Policy PAT will override the Dynamic PAT. And as mentioned before, this applies only for outbound SMTP traffic. All other outbound traffic uses the above configuration for Dynamic PAT.

You can also confirm the translations done to certain connections with the "packet-tracer" command

The general format is

packet-tracer input tcp 12345 25

Where the is the interface behind which the host is located.

- Jouni