Solved: WAN routing for a subnet that appears in two places in the network

branfarm1 · ‎12-21-2009

Hi everybody,

I was brought in last minute on a migration project where I discovered the first phase of the project is to move a subset of servers to a new location. The new location will eventually receive all of the servers currently in this subnet, 10.80.3.X/24, but for a short time there will be servers in two locations with the same subnet address. Fortunately, the servers that are moving are only accessed by a handful of other machines. I was wondering if PBR will solve my problem here, and if so, if my hardware can support it. I have two 6509's w/Sup720 with PFC3A.

My first idea is to create an access-list for servers that will be accessing the devices that are moving, then use a route-map to set the next hop for these devices to be the firewall (WAN connectivity is still being built out, so I'm using an IPsec tunnel between two ASA's for now).

Will this work?

Thanks in advance,

Brandon

Jon Marshall · ‎12-21-2009

branfarm1 wrote:

Hi everybody,

I was brought in last minute on a migration project where I discovered the first phase of the project is to move a subset of servers to a new location. The new location will eventually receive all of the servers currently in this subnet, 10.80.3.X/24, but for a short time there will be servers in two locations with the same subnet address. Fortunately, the servers that are moving are only accessed by a handful of other machines. I was wondering if PBR will solve my problem here, and if so, if my hardware can support it. I have two 6509's w/Sup720 with PFC3A.

My first idea is to create an access-list for servers that will be accessing the devices that are moving, then use a route-map to set the next hop for these devices to be the firewall (WAN connectivity is still being built out, so I'm using an IPsec tunnel between two ASA's for now).

Will this work?

Thanks in advance,

Brandon

Your 6500 will support PBR so yes you could use this. If you can narrow down which machines access the moved servers then yes you could use PBR but bear in mind you will may get asymmetric routing ie. traffic for the servers goes to the old location, the traffic is then PBR'd to the new location but if the new location is also connected to the WAN the traffic can go straight back to the source rather than the server old location.

This may or may not be a problem for you depending on how the new location is connected to the rest of your network.

You could also NAT the moved servers and have the other machines that need access to send traffic to the Natted address. This generally works okay if you resolve by name but if anything has a hardcoded IP address then probably not a good idea.

Another option depending on the kit is to extend the vlan across the routed WAN with something like L2TPv3.

Finally ,on one the migration projects i worked on we had a temporary L2 link between the 2 sites, but they were only abit 10 miles apart and if it last minute it sounds like you may not have this option.

Jon

View solution in original post

Reza Sharifi · ‎12-21-2009

Hi,

If you are not running MPLS, then you can use GRE to archive this but you need SIP 400 for your 6500s.

Here is more info:

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/white_paper_c11_493718.html#wp9000226

HTH

Reza

Jon Marshall · ‎12-21-2009

branfarm1 wrote:

Hi everybody,

I was brought in last minute on a migration project where I discovered the first phase of the project is to move a subset of servers to a new location. The new location will eventually receive all of the servers currently in this subnet, 10.80.3.X/24, but for a short time there will be servers in two locations with the same subnet address. Fortunately, the servers that are moving are only accessed by a handful of other machines. I was wondering if PBR will solve my problem here, and if so, if my hardware can support it. I have two 6509's w/Sup720 with PFC3A.

My first idea is to create an access-list for servers that will be accessing the devices that are moving, then use a route-map to set the next hop for these devices to be the firewall (WAN connectivity is still being built out, so I'm using an IPsec tunnel between two ASA's for now).

Will this work?

Thanks in advance,

Brandon

Your 6500 will support PBR so yes you could use this. If you can narrow down which machines access the moved servers then yes you could use PBR but bear in mind you will may get asymmetric routing ie. traffic for the servers goes to the old location, the traffic is then PBR'd to the new location but if the new location is also connected to the WAN the traffic can go straight back to the source rather than the server old location.

This may or may not be a problem for you depending on how the new location is connected to the rest of your network.

You could also NAT the moved servers and have the other machines that need access to send traffic to the Natted address. This generally works okay if you resolve by name but if anything has a hardcoded IP address then probably not a good idea.

Another option depending on the kit is to extend the vlan across the routed WAN with something like L2TPv3.

Finally ,on one the migration projects i worked on we had a temporary L2 link between the 2 sites, but they were only abit 10 miles apart and if it last minute it sounds like you may not have this option.

Jon

branfarm1 · ‎12-21-2009

Thanks Jon -- I was worried about PBR on this 6500 because it has PFC3A + CFC -- wasn't sure if that would be a problem. Asymmetric routing shouldn't be problem at this point -- the only access to the new location will be via IPsec tunnel until the WAN circuit comes in. Once the circuit is in, I'll either be changing the route map to point to the WAN or, hopefully removing the route-map since everything will be migrated.

I like that NATing idea, but I'm unsure how I would do this. The existing location is off a 6509, which can NAT, and the new location will be behind a L2L Ipsec tunnel, ASA, and 3750. It would make sense to NAT on the remote end, but I don't think the 3750-E supports NAT -- is that right?

--Brandon

Jon Marshall · ‎12-21-2009

branfarm1 wrote:

Thanks Jon -- I was worried about PBR on this 6500 because it has PFC3A + CFC -- wasn't sure if that would be a problem. Asymmetric routing shouldn't be problem at this point -- the only access to the new location will be via IPsec tunnel until the WAN circuit comes in. Once the circuit is in, I'll either be changing the route map to point to the WAN or, hopefully removing the route-map since everything will be migrated.

I like that NATing idea, but I'm unsure how I would do this. The existing location is off a 6509, which can NAT, and the new location will be behind a L2L Ipsec tunnel, ASA, and 3750. It would make sense to NAT on the remote end, but I don't think the 3750-E supports NAT -- is that right?

--Brandon

Brandon

Your'e right, the 3750-E does not support NAT but the ASA does so if you wanted to you could do the NAT on that. You could pick an unused subnet to present the moved servers with new addressing. You can then advertise this subnet out of the old location if there are clients that need access that are not actually in the old location but across the WAN.

As i say there are 2 drawbacks with NAT -

1) some apps don't work but to be fair most do

2) If there are hardcoded IP addresses then your's stuffed to be honest. You need name lookups for all communication and you need to factor in DNS caching ie. if you change the IP address to name with the new NAT IP address you want to make sure that clients don't cache the name to old IP so you would need to shorten your DNS timeouts before implementing.

Jon

branfarm1 · ‎12-22-2009

So I tried configuring this today and it didn't quite work right, and I'm hoping you can point me to my mistake.

The migrating servers are 10.80.3.101-110, Vlan530, 10.80.7.0 is off Vlan501, 10.80.1.0 is off Vlan502.

I started by creating an access-list for servers on other Vlan's that will be accessing these servers:

10.80.7.20-24, and 10.80.1.20-24.

access-list 123 permit ip host 10.80.7.20 10.80.3.96 255.255.255.240

access-list 123 permit ip host 10.80.7.21 10.80.3.96 255.255.255.240

access-list 123 permit ip host 10.80.7.22 10.80.3.96 255.255.255.240

access-list 123 permit ip host 10.80.7.23 10.80.3.96 255.255.255.240

access-list 123 permit ip host 10.80.7.24 10.80.3.96 255.255.255.240

same thing for 10.80.1.20-24, ACL 124.

Then I created a route map:

route-map migration123 permit 10

match ip address 123

set ip next-hop 192.168.2.1

route-map migration124 permit 10

match ip address 124

set ip next-hop 192.168.2.1

Next I added the route-map to the Vlan501 and Vlan502 interface:

int vlan501

ip policy route-map migration123

int vlan502

ip policy route-map migration124

Once I enabled PBR on the interfaces, servers/users in other vlans couldn't reach the servers I had listed in the access-list. For example, 10.80.1.25 couldn't reach 10.80.7.20-24.

I'm guessing it's something wrong with my access-list, but I don't know enough about route-maps to know where I"m going.

Thanks,

Brandon

Jon Marshall · ‎12-22-2009

Brandon

Could you post the exact acls you were using ?

Jon

branfarm1 · ‎12-22-2009

Here they are:

Access-list 123 remark ASH_MIGRATION_ACCESS_10.80.7.X
Access-list 123 permit ip host 10.80.7.20 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.21 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.22 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.24 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.26 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.30 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.31 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.32 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.40 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.41 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.42 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.43 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.62 10.80.3.96 255.255.255.240

Access-list 124 remark ASH_MIGRATION_ACCESS_10.80.1.X
Access-list 124 permit ip host 10.80.1.17 10.80.3.96 255.255.255.240

Jon Marshall · ‎12-22-2009

branfarm1 wrote:

Here they are:

Access-list 123 remark ASH_MIGRATION_ACCESS_10.80.7.X
Access-list 123 permit ip host 10.80.7.20 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.21 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.22 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.24 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.26 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.30 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.31 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.32 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.40 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.41 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.42 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.43 10.80.3.96 255.255.255.240
Access-list 123 permit ip host 10.80.7.62 10.80.3.96 255.255.255.240

Access-list 124 remark ASH_MIGRATION_ACCESS_10.80.1.X
Access-list 124 permit ip host 10.80.1.17 10.80.3.96 255.255.255.240

Brandon

Sorry, was just about to ask for route-map details as well.

Jon

branfarm1 · ‎12-22-2009

Route-map MIGRATION_10.80.7.X permit 10
Match ip address 123
Set ip next-hop 192.168.2.1

Route-map MIGRATION_10.80.1.X permit 10
Match ip address 124
Set ip next-hop 192.168.2.1

Int vlan 501 (10.80.7.0/24)
Ip policy route-map MIGRATION_10.80.7.X

Int vlan 502 (10.80.1.0/24)
Ip policy route-map MIGRATION_10.80.1.X

192.168.2.1 is the inside interface for the firewall.

branfarm1 · ‎12-22-2009

Just noticed something from the router -- I need to use wildcard masks. is that right?

Jon Marshall · ‎12-22-2009

Brandon

Nothing stands out as being wrong with the config. PBR basically works in that if there is no match in the acl you use in the route-map then the routing table is used so it should just default to the route table.

When you applied the route-map did all traffic stop working other than what you specified in your acls ?

Jon

branfarm1 · ‎12-22-2009

Thanks Jon -- That's what I was reading in one of the docs.

I just noticed though, that when I do a show access-list 124 from the command line, I see:

6509-1#sh access-list 124
Extended IP access list 124
10 permit ip host 10.80.1.17 0.0.0.0 255.255.255.240

Jon Marshall · ‎12-22-2009

branfarm1 wrote:

Thanks Jon -- That's what I was reading in one of the docs.

I just noticed though, that when I do a show access-list 124 from the command line, I see:

6509-1#sh access-list 124
Extended IP access list 124
10 permit ip host 10.80.1.17 0.0.0.0 255.255.255.240

Ahh, stupid me, i should have spotted that. Your acls are using the wrong masks ie. you should be using inverse masks so -

change 255.255.255.240 to 0.0.0.15 and that should do it ie

access-list 124 permit ip host 10.80.1.17 10.80.3.96 0.0.0.15

you'll need to change acl 123 as well.

That's the problem with answering questions in Network infrastructure and Security as pix/ASA firewalls using normal subnet masks whereas router acls use inverse masks

Jon

branfarm1 · ‎12-22-2009

I hear ya -- I spend 99% of my time changing firewall configs -- not writing acl's for switches.

I can't remember -- do switches/router always use wildcard masks, or only sometimes?