ASA 5540 - VLAN Subinterfaces and Routing

sdhill · ‎05-30-2010

We are testing the use of layer 3 switching, trunking, and VLAN subinterfaces with the (4) INSIDE networks. Layer 2 and Layer 3 pings are working at the switch, however from the subinterfaces on the firewall they can only see their subnet.

Pinging from systems on the subnets can see their subnets but not the others.

IP routes show connected on both the switch and firewall. Only INSIDE subnet is getting out to the OUTSIDE interface, the other 3 are isolated for some reason.

FIREWALL ROUTES:

Gateway of last resort is xx.xx.61.1 to network 0.0.0.0

C xx.xx.61.0 255.255.255.0 is directly connected, OUTSIDE

C 172.16.1.0 255.255.255.0 is directly connected, DMZ

C 192.168.102.0 255.255.255.0 is directly connected, vmKERNEL

C 192.168.1.0 255.255.255.0 is directly connected, INSIDE

C 192.168.2.0 255.255.255.0 is directly connected, prodPS

C 192.168.101.0 255.255.255.0 is directly connected, vmCONSOLE

S* 0.0.0.0 0.0.0.0 [1/0] via xx.xx.61.1, OUTSIDE

SWITCH ROUTES:

Gateway of last resort is not set

C 192.168.102.0/24 is directly connected, Vlan202

C 192.168.254.0/24 is directly connected, Vlan911

C 192.168.1.0/24 is directly connected, Vlan101

C 192.168.2.0/24 is directly connected, Vlan102

C 192.168.101.0/24 is directly connected, Vlan201

We are getting portmap translation errors in reference to the other 3 INSIDE networks which all have the same security level of 100.

We have been looking at this too long, can't see the forest thru the trees.

Firewall Config:

interface GigabitEthernet0/0

description OUTSIDE - VLAN 666

speed 100

duplex full

nameif OUTSIDE

security-level 0

ip address xx.xx.61.226 255.255.255.0

!

interface GigabitEthernet0/1

speed 100

duplex full

no nameif

security-level 100

no ip address

!

interface GigabitEthernet0/1.101

description PROD-RS - VLAN 101

vlan 101

nameif PROD-RS

security-level 100

ip address 192.168.1.1 255.255.255.0

!

interface GigabitEthernet0/1.102

description PROD-PS - VLAN 102

vlan 102

nameif PROD-PS

security-level 100

ip address 192.168.2.1 255.255.255.0

!

interface GigabitEthernet0/1.201

description VM-CONSOLE - VLAN 201

vlan 201

nameif VM-CONSOLE

security-level 100

ip address 192.168.101.1 255.255.255.0

!

interface GigabitEthernet0/1.202

description VM-KERNEL - VLAN 202

vlan 202

nameif VM-KERNEL

security-level 100

ip address 192.168.102.1 255.255.255.0

!

interface GigabitEthernet0/2

description DMZ - VLAN 411

speed 100

duplex full

nameif DMZ

security-level 25

ip address 172.16.1.1 255.255.255.0

!

interface GigabitEthernet0/3

speed 100

duplex full

no nameif

no security-level

no ip address

!

interface Management0/0

description MANAGE - VLAN 911

speed 100

duplex full

nameif MANAGE

security-level 100

ip address 192.168.254.1 255.255.255.0

!

boot system disk0:/asa821-k8.bin

ftp mode passive

clock timezone CST -6

clock summer-time CDT recurring

dns server-group DefaultDNS

name-server 192.168.1.30

domain-name PetiteSirens.com

same-security-traffic permit inter-interface

same-security-traffic permit intra-interface

pager lines 24

logging enable

logging buffer-size 30000

logging buffered debugging

logging asdm notifications

mtu OUTSIDE 1500

mtu PROD-RS 1500

mtu DMZ 1500

mtu MANAGE 1500

mtu PROD-PS 1500

mtu VM-CONSOLE 1500

mtu VM-KERNEL 1500

ip verify reverse-path interface OUTSIDE

ip verify reverse-path interface DMZ

no failover

icmp unreachable rate-limit 1 burst-size 1

asdm image disk0:/asdm-631.bin

asdm history enable

arp timeout 14400

global (OUTSIDE) 101 interface

nat (PROD-RS) 101 0.0.0.0 0.0.0.0

nat (DMZ) 101 0.0.0.0 0.0.0.0

nat (MANAGE) 101 0.0.0.0 0.0.0.0

nat (PROD-PS) 101 0.0.0.0 0.0.0.0

route OUTSIDE 0.0.0.0 0.0.0.0 xx.xx.61.1 1

SWITCH config:

hostname aswitch01

!

no aaa new-model

clock timezone UTC -6

clock summer-time UTC recurring

switch 1 provision ws-c3750-24p

system mtu routing 1500

vtp domain INSIDE

vtp mode transparent

authentication mac-move permit

ip subnet-zero

ip routing

!

spanning-tree mode pvst

spanning-tree portfast bpduguard default

spanning-tree etherchannel guard misconfig

spanning-tree extend system-id

!

vlan internal allocation policy ascending

!

vlan 101

name PROD-RS

!

vlan 102

name PROD-PS

!

vlan 201

name VM-CONSOLE

!

vlan 202

name VM-KERNEL

!

vlan 911

name MANAGE

!

interface Loopback0

no ip address

!

interface FastEthernet1/0/1

switchport access vlan 101

switchport mode access

power inline never

speed 100

duplex full

no cdp enable

spanning-tree portfast

spanning-tree guard root

!

interface FastEthernet1/0/24

description TRUNK to FIREWALL INT G0/1

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 101,102,201,202,911

switchport mode trunk

power inline never

speed 100

duplex full

!

interface Vlan1

no ip address

shutdown

!

interface Vlan101

ip address 192.168.1.11 255.255.255.0

!

interface Vlan102

ip address 192.168.2.11 255.255.255.0

!

interface Vlan201

ip address 192.168.101.11 255.255.255.0

!

interface Vlan202

ip address 192.168.102.11 255.255.255.0

!

interface Vlan911

ip address 192.168.254.11 255.255.255.0

!

ip default-gateway 192.168.1.1

ip classless

ip http server

ip http secure-server

Maykol Rojas · ‎05-30-2010

Hello Sdhill

Topology Internet

ASA

||

|| Trunk

||

--------------

| L3 switch |

--------------

/ | \

Vlan101 Vlan102 Vlan103

I am assuming that the only interface is that is able to go out to the internet is PROD-RS - VLAN 101 is that correct? Well this actually makes sense and let me explain you why.

Lets say that you are sitting on the vlan PROD-PS - VLAN 102, this are the steps (based on your routing) on how a packet would flow when going to the outside:

-It is going to go from the computer to the Layer 3 switch

-From the layer 3 switch its going to pick up the default route which poing to 192.168.1.1 and head the the PROD-RS - VLAN 101 interface of the Firewall

-Then the return packet from the outside comes to the firewall with a destination that is directly connected to it and it is going to try to send it to PROD-PS - VLAN 102

-The problem is that the firewall already has an state entry that says that the packet first went out throuh the interface PROD-RS - VLAN 101, and then since it is not the same interface as it was when it went out, the packet will be discarded. (Because of asymmetric routing)

Nature rule of every Stateful Firewall, if packet goes out on one interface, the return packet should be send on the same one.

But, why does it work with the Vlan 101?

The packet enters and leaves on the same interface, opposite on what happens when you start a connection on Vlan 102, or any other vlan.

I am pretty sure that if you change the default route on the switch to be 192.168.2.1, everyone on that vlan will be able to access the outside interface but Vlan 101 and the rest would be blocked.

How to solve this?

If you want to protect your Network on an effective way, I would recommend you to have the routing being done only on the firewall, thus disabling routing capabilities on the switch and leaving only the l2 Vlan segmentation.

If you have any doubts, please let me know, I would be more than glad to assist.

Mike

sdhill · ‎05-30-2010

Howdy Maykol,

I disagree slightly with your trace since all 4 subnets are trunked to the firewall using VLAN subinterfaces. The gateway of last resort is the OUTSIDE interfaces' peer at the ISP. Each VLAN is designated on the firewall have their own gateways.

The ip default-gateway actually doesn't apply in the switch configuration (even though it is defined) since Layer 3 routing is enabled and the gateway of last resort is defined on the firewall.

We tried disabling ip routing on the switch but the results were not much better.

If we went to layer 2, then we would have to remove the subinterfaces and VLANs from the firewall, remove the trunk, and implement a Layer 3 switchport on the switch then define static routes on the firewall -seems to be the best practice recommendation.

Not sure which is more effective or best practice.

Things to consider, in the future we have to implement VPN access that has issues with hairpins. Trying to keep things more flexible without excessive manipulation of the NATs (static) especially before upgrading to v8.3.

I think my issue is more about NAT since they are dynamic and not static since we only get portmap translation issues on the firewall for VLANs 102, 201, and 202.

Scott