10-13-2010 01:45 AM - edited 03-04-2019 10:06 AM
Hello, I know this question is raised many times, but unfortunately I could not find answer for my case. I have a router (R1) and 2 connections to 2 different ISP. In another location I have 1 router (R2) with 1 connection to 1 ISP. The goal is to make.
1. 2 tunnels between R1 and R2.
R1 - tun1 - ISP1 - Internet - ISP3 - R2
R1 - tun2 - ISP2 - Internet - ISP3 - R3
Note: both tunnels must be Up simultaneously, no SLA, BACKUP, etc... solutions are acceptable.
2. Provide LAN (rfc1912 IP range) behind R1 Internet connectivity.
Note: ISP1 is primary, ISP2 is backup, here SLA solution is acceptable.
3. From Internet must be possible to connect to VPN server on R1 via both ISPs
What I have tried.
a. SLA works fine for point 2. But point 1 and 3 cannot be solved with SLA
b. ip local route-map and route-map. Works fine for point 2, but no point 1. Did not check point 3 but ping from Internet was OK via both ISP.
c. vrf-lite. works for point 1, but not for point 2. Did not check point 3 but again ping from Internet was OK via both ISP.
The important points are 1 and 2.
Anybody has deal with such setup? Any advise, documentation, comments are appreciated.
Thanks everybody for help and no help:)
P.S.
In ideal case PI IPv4 /24 network + ASN and BGP is the best solution, then I need only 1 tunnel. But small companies like mine cannot afford such things, unfortunately.
10-13-2010 03:07 AM
To be more specific... network diagram is attached. Configs of CE and R are below for VRF-lite version.
Everything works fine except NAT. I understand I cannot configure interface fa 1/0 on CE to be in 2 of VRFs at the same time. So NAT is not working. How this can be solved?
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname ce
!
boot-start-marker
boot-end-marker
!
no logging console
!
no aaa new-model
memory-size iomem 5
ip cef
!
ip vrf p1e
rd 100:1
!
ip vrf p2e
rd 200:1
!
no ip domain lookup
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
multilink bundle-name authenticated
!
archive
log config
hidekeys
!
interface Tunnel1
description VPN to R via P1E
ip address 10.214.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 1000
tunnel source FastEthernet0/0
tunnel destination 3.3.3.33
tunnel path-mtu-discovery
tunnel vrf p1e
!
interface Tunnel2
description VPN to R via P1E
ip address 10.215.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 2000
tunnel source FastEthernet0/1
tunnel destination 3.3.3.33
tunnel path-mtu-discovery
tunnel vrf p2e
!
interface FastEthernet0/0
ip vrf forwarding p1e
ip address 1.1.1.11 255.255.255.0
ip nat enable
speed 100
full-duplex
!
interface FastEthernet0/1
ip vrf forwarding p2e
ip address 2.2.2.22 255.255.255.0
ip nat enable
speed 100
full-duplex
!
interface FastEthernet1/0
ip address 10.10.10.1 255.255.255.0
ip nat enable
speed 100
full-duplex
!
router ospf 1
ispf
log-adjacency-changes
passive-interface default
no passive-interface Tunnel1
no passive-interface Tunnel2
network 10.10.10.1 0.0.0.0 area 0
network 10.214.0.6 0.0.0.0 area 0
network 10.215.0.6 0.0.0.0 area 0
!
ip forward-protocol nd
ip route vrf p1e 0.0.0.0 0.0.0.0 1.1.1.1
ip route vrf p2e 0.0.0.0 0.0.0.0 2.2.2.1
!
no ip http server
no ip http secure-server
ip nat source list nat interface FastEthernet0/0 vrf p1e overload
ip nat source list nat interface FastEthernet0/1 vrf p2e overload
!
ip access-list extended nat
permit ip host 10.10.10.10 any
!
control-plane
!
line con 0
line aux 0
line vty 0 4
login
!
end
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname r
!
boot-start-marker
boot-end-marker
!
no logging console
!
no aaa new-model
memory-size iomem 5
ip cef
!
no ip domain lookup
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
multilink bundle-name authenticated
!
archive
log config
hidekeys
!
interface Loopback1
ip address 10.30.0.1 255.255.255.0
!
interface Tunnel1
description VPN to CE via P1E
ip address 10.214.0.1 255.255.255.248
ip mtu 1416
ip ospf cost 1000
tunnel source FastEthernet0/0
tunnel destination 1.1.1.11
tunnel path-mtu-discovery
!
interface Tunnel2
description VPN to CE via P2E
ip address 10.215.0.1 255.255.255.248
ip mtu 1416
ip ospf cost 2000
tunnel source FastEthernet0/0
tunnel destination 2.2.2.22
tunnel path-mtu-discovery
!
interface FastEthernet0/0
ip address 3.3.3.33 255.255.255.0
speed 100
full-duplex
!
interface FastEthernet0/1
no ip address
shutdown
speed 100
full-duplex
!
router ospf 1
ispf
log-adjacency-changes
passive-interface default
no passive-interface Tunnel1
no passive-interface Tunnel2
network 10.30.0.1 0.0.0.0 area 0
network 10.214.0.1 0.0.0.0 area 0
network 10.215.0.1 0.0.0.0 area 0
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 3.3.3.1
!
no ip http server
no ip http secure-server
!
control-plane
!
line con 0
line aux 0
line vty 0 4
login
!
end
10-13-2010 11:56 PM
Talking myself?
Managed almost everything to work. Only some problem with dynamic NAT.
It is working when ISP1 (p1e) is Up, but not working when it goes down.
Here is configuration. If anybody knows better way for solution of mentioned problem I would be happy for comments. Thanks.
--
Bests,
Karen
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname ce
!
boot-start-marker
boot-end-marker
!
logging buffered 4096
no logging console
enable password test
!
no aaa new-model
memory-size iomem 5
ip cef
!
ip vrf local
rd 0:0
route-target export 0:0
route-target import 0:0
route-target import 100:1
route-target import 200:1
!
ip vrf p1e
rd 100:1
route-target export 100:1
route-target import 100:1
route-target import 200:1
route-target import 0:0
!
ip vrf p2e
rd 200:1
route-target export 200:1
route-target import 200:1
route-target import 100:1
route-target import 0:0
!
no ip domain lookup
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
multilink bundle-name authenticated
!
username test password 0 test
archive
log config
hidekeys
!
track 1 rtr 1
!
interface Tunnel1
description VPN to R via P1E
ip vrf forwarding local
ip address 10.214.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 1000
tunnel source FastEthernet0/0
tunnel destination 3.3.3.33
tunnel path-mtu-discovery
tunnel vrf p1e
!
interface Tunnel2
description VPN to R via P1E
ip vrf forwarding local
ip address 10.215.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 2000
tunnel source FastEthernet0/1
tunnel destination 3.3.3.33
tunnel path-mtu-discovery
tunnel vrf p2e
!
interface FastEthernet0/0
ip vrf forwarding p1e
ip address 1.1.1.11 255.255.255.0
ip nat enable
speed 100
full-duplex
!
interface FastEthernet0/1
ip vrf forwarding p2e
ip address 2.2.2.22 255.255.255.0
ip nat enable
speed 100
full-duplex
!
interface FastEthernet1/0
ip vrf forwarding local
ip address 10.10.10.1 255.255.255.0
ip nat enable
speed 100
full-duplex
!
router ospf 1 vrf local
ispf
log-adjacency-changes
passive-interface default
no passive-interface Tunnel1
no passive-interface Tunnel2
network 10.10.10.1 0.0.0.0 area 0
network 10.214.0.6 0.0.0.0 area 0
network 10.215.0.6 0.0.0.0 area 0
!
router bgp 65535
no synchronization
bgp log-neighbor-changes
no auto-summary
!
address-family ipv4 vrf p2e
redistribute connected
redistribute static
no synchronization
exit-address-family
!
address-family ipv4 vrf p1e
redistribute connected
redistribute static
no synchronization
exit-address-family
!
address-family ipv4 vrf local
redistribute connected
redistribute static
no synchronization
exit-address-family
!
ip forward-protocol nd
ip route vrf local 0.0.0.0 0.0.0.0 1.1.1.1 track 1
ip route vrf p1e 0.0.0.0 0.0.0.0 1.1.1.1
ip route vrf p2e 0.0.0.0 0.0.0.0 2.2.2.1
ip route vrf local 0.0.0.0 0.0.0.0 2.2.2.1 2
!
no ip http server
no ip http secure-server
ip nat source list nat-p1e interface FastEthernet0/0 vrf local overload
ip nat source list nat-p2e interface FastEthernet0/1 vrf local overload
ip nat source static tcp 10.10.10.10 23 1.1.1.11 23 vrf local extendable
ip nat source static tcp 10.10.10.10 23 2.2.2.22 23 vrf local extendable
!
ip access-list extended nat-p1e
permit ip host 10.10.10.10 any
ip access-list extended nat-p2e
permit ip host 10.10.10.10 any
!
ip sla 1
icmp-echo 3.3.3.1 source-ip 1.1.1.11
timeout 3000
vrf p1e
frequency 10
ip sla schedule 1 life forever start-time now
!
control-plane
!
line con 0
line aux 0
line vty 0 4
login local
line vty 5 15
login local
!
end
10-14-2010 03:11 AM
OK, seems dynamic nat problem is also solved. Don't know if this is acceptable to use route-map with nvi. According docs this is not supported.
Thanks everybody.
--
Bests,
Karen
!
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname ce
!
boot-start-marker
boot-end-marker
!
logging buffered 4096
no logging console
enable password test
!
no aaa new-model
memory-size iomem 5
ip cef
!
ip vrf local
rd 0:0
route-target export 0:0
route-target import 0:0
route-target import 100:1
route-target import 200:1
!
ip vrf p1e
rd 100:1
route-target export 100:1
route-target import 100:1
route-target import 200:1
route-target import 0:0
!
ip vrf p2e
rd 200:1
route-target export 200:1
route-target import 200:1
route-target import 100:1
route-target import 0:0
!
no ip domain lookup
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
multilink bundle-name authenticated
!
username cco password 0 test
archive
log config
hidekeys
!
track 1 rtr 1
!
interface Tunnel1
description VPN to R via P1E
ip vrf forwarding local
ip address 10.214.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 1000
tunnel source FastEthernet0/0
tunnel destination 3.3.3.33
tunnel path-mtu-discovery
tunnel vrf p1e
!
interface Tunnel2
description VPN to R via P1E
ip vrf forwarding local
ip address 10.215.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 2000
tunnel source FastEthernet0/1
tunnel destination 3.3.3.33
tunnel path-mtu-discovery
tunnel vrf p2e
!
interface FastEthernet0/0
ip vrf forwarding p1e
ip address 1.1.1.11 255.255.255.0
ip nat enable
speed 100
full-duplex
!
interface FastEthernet0/1
ip vrf forwarding p2e
ip address 2.2.2.22 255.255.255.0
ip nat enable
speed 100
full-duplex
!
interface FastEthernet1/0
ip vrf forwarding local
ip address 10.10.10.1 255.255.255.0
ip nat enable
speed 100
full-duplex
!
router ospf 1 vrf local
ispf
log-adjacency-changes
passive-interface default
no passive-interface Tunnel1
no passive-interface Tunnel2
network 10.10.10.1 0.0.0.0 area 0
network 10.214.0.6 0.0.0.0 area 0
network 10.215.0.6 0.0.0.0 area 0
!
router bgp 65535
no synchronization
bgp log-neighbor-changes
no auto-summary
!
address-family ipv4 vrf p2e
redistribute connected
redistribute static
no synchronization
exit-address-family
!
address-family ipv4 vrf p1e
redistribute connected
redistribute static
no synchronization
exit-address-family
!
address-family ipv4 vrf local
redistribute connected
redistribute static
no synchronization
exit-address-family
!
ip forward-protocol nd
ip route vrf local 0.0.0.0 0.0.0.0 1.1.1.1 track 1
ip route vrf p1e 0.0.0.0 0.0.0.0 1.1.1.1
ip route vrf p2e 0.0.0.0 0.0.0.0 2.2.2.1
ip route vrf local 0.0.0.0 0.0.0.0 2.2.2.1 2
!
no ip http server
no ip http secure-server
ip nat source route-map nat-p1e interface FastEthernet0/0 vrf local overload
ip nat source route-map nat-p2e interface FastEthernet0/1 vrf local overload
ip nat source static tcp 10.10.10.10 23 1.1.1.11 23 vrf local extendable
ip nat source static tcp 10.10.10.10 23 2.2.2.22 23 vrf local extendable
!
ip access-list extended nat-p1e
permit ip host 10.10.10.10 any
ip access-list extended nat-p2e
permit ip host 10.10.10.10 any
!
ip sla 1
icmp-echo 3.3.3.1 source-ip 1.1.1.11
timeout 3000
vrf p1e
frequency 10
ip sla schedule 1 life forever start-time now
!
route-map nat-p2e permit 10
match ip address nat-p2e
match interface FastEthernet0/1
!
route-map nat-p1e permit 10
match ip address nat-p1e
match interface FastEthernet0/0
!
control-plane
!
line con 0
line aux 0
line vty 0 4
login local
line vty 5 15
login local
!
!
end
10-28-2010 06:56 AM
Karen, congratulations.
P.S.: Sorry about my English.
REALLY creative solution!
I'm on a similar problem right now, and I was forgetting about VRF.
My problems began when our costumer changed his primary ISP.
The previous ISP accepted packets sent with different source IP Address, does not dropping as spoofing.
My solution until that was:
- Tracks SLA to both gateways
- Static Nats based on RouteMap
- Dynamic Nats based on RouteMap
- PBR to a RouteMap with a "set ip next-hop verify-reachability", to stations.
- Default-Routes with tracks and different administrative distances
The things where working well until that
Now with the new primary ISP, that solution isn't working anymore.
The new ISP blocks Spoofing.
What I'm doing at this moment is almost the same that previous, but with a route to the other router/32 point to ISP2 with a track and a lower administrative distance.
- Only the side of ISP1 is responding to incoming NAT (here we are using DNS SRV)
- Only the second tunnel is UP(over ISP2)
- When one of the links goes down the services come up on the other side.
But this situation is not really comfortable.
One friend had mentioned to use to default routes.
With that the router would use a "per-flow" based routing.
So if the client comes first from one side, the route cache for that source would be assigned to that ISP link.
I don't trust really much on that, but I'm disposed to try it.
Bottom Line
Is this environment working well now?
What is the IOS Version and router model?
10-28-2010 09:23 PM
Thanks for congratulations,
I have tested this in GNS3 with real IOS so this must not be problem. Only the issue is dynamic NAT with VRF. I read in documents that route-map is not supported with VRF, but it worked for me during test. So I will be happy if some expert would confirm that this is ok.
HW was Cisco 2691 (in real life I would use Cisco 2800 series, just GNS does not support it so I did test with 2691)
IOS version I have used is c2691-advipservicesk9-mz.124-15.T4.bin
Your solution with 2 default route is ok when we are talking about access to Internet. But how VPN Tunnels will work? Just imagine 1 time it will try to connect to the same destination tunnel with 1st ISP (depends on CEF condition) and other time it may use other ISP. In the end it is possible that router will try to establish 2 tunnels via the same ISP. This was the case for me, I needed to have two tunnel up simultaneously - one via 1st ISP and 2nd via the other ISP.
10-31-2010 11:42 PM
OK, to make sure I am not doing anything unsupported (route-map with VRF and NVI), I have removed ip nat enabled and back to ip nat inside/outside domains. I can say my all problems mentioned in very first post are solved. I post working configuration here, I hope this can help someone else to solve the similar problem. Thanks and good luck.
used HW: cisco 2691
used IOS: c2691-advipservicesk9-mz.124-15.T4.bin
configuration:
version 12.4
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname ce
!
boot-start-marker
boot-end-marker
!
logging buffered 4096
no logging console
enable password test
!
no aaa new-model
memory-size iomem 5
ip cef
!
ip vrf local
rd 0:0
route-target export 0:0
route-target import 0:0
route-target import 100:1
route-target import 200:1
!
ip vrf p1e
rd 100:1
route-target export 100:1
route-target import 100:1
route-target import 200:1
route-target import 0:0
!
ip vrf p2e
rd 200:1
route-target export 200:1
route-target import 200:1
route-target import 100:1
route-target import 0:0
!
no ip domain lookup
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
!
multilink bundle-name authenticated
!
username cco password 0 test
archive
log config
hidekeys
!
track 1 rtr 1
!
interface Tunnel1
description VPN to R via P1E
ip vrf forwarding local
ip address 10.214.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 1000
tunnel source FastEthernet0/0
tunnel destination 6.6.6.2
tunnel path-mtu-discovery
tunnel vrf p1e
!
interface Tunnel2
description VPN to R via P1E
ip vrf forwarding local
ip address 10.215.0.6 255.255.255.248
ip mtu 1416
ip ospf cost 2000
tunnel source FastEthernet0/1
tunnel destination 6.6.6.2
tunnel path-mtu-discovery
tunnel vrf p2e
!
interface FastEthernet0/0
ip vrf forwarding p1e
ip address 1.1.1.11 255.255.255.0
ip nat outside
speed 100
full-duplex
!
interface FastEthernet0/1
ip vrf forwarding p2e
ip address 2.2.2.22 255.255.255.0
ip nat outside
speed 100
full-duplex
!
interface FastEthernet1/0
ip vrf forwarding local
ip address 10.10.10.1 255.255.255.0
ip nat inside
speed 100
full-duplex
!
router ospf 1 vrf local
ispf
log-adjacency-changes
passive-interface default
no passive-interface Tunnel1
no passive-interface Tunnel2
network 10.10.10.1 0.0.0.0 area 0
network 10.214.0.6 0.0.0.0 area 0
network 10.215.0.6 0.0.0.0 area 0
!
router bgp 65535
bgp router-id 10.10.10.1
no synchronization
bgp log-neighbor-changes
no auto-summary
!
address-family ipv4 vrf p2e
redistribute connected
redistribute static
no synchronization
exit-address-family
!
address-family ipv4 vrf p1e
redistribute connected
redistribute static
no synchronization
exit-address-family
!
address-family ipv4 vrf local
redistribute connected
redistribute static
no synchronization
exit-address-family
!
no ip forward-protocol nd
ip route vrf local 0.0.0.0 0.0.0.0 1.1.1.1 track 1
ip route vrf local 0.0.0.0 0.0.0.0 2.2.2.1 2
ip route vrf p1e 0.0.0.0 0.0.0.0 1.1.1.1
ip route vrf p2e 0.0.0.0 0.0.0.0 2.2.2.1
!
no ip http server
no ip http secure-server
ip nat inside source route-map nat-p1e interface FastEthernet0/0 vrf local overload
ip nat inside source route-map nat-p2e interface FastEthernet0/1 vrf local overload
ip nat inside source static tcp 10.10.10.10 23 1.1.1.11 23 vrf local extendable
ip nat inside source static tcp 10.10.10.10 23 2.2.2.22 23 vrf local extendable
!
ip access-list extended nat-p1e
permit ip host 10.10.10.10 any
ip access-list extended nat-p2e
permit ip host 10.10.10.10 any
!
ip sla 1
icmp-echo 3.3.3.1 source-ip 1.1.1.11
timeout 3000
vrf p1e
frequency 10
ip sla schedule 1 life forever start-time now
!
route-map nat-p2e permit 10
match ip address nat-p2e
match interface FastEthernet0/1
!
route-map nat-p1e permit 10
match ip address nat-p1e
match interface FastEthernet0/0
!
control-plane
!
line con 0
line aux 0
line vty 0 4
login local
line vty 5 15
login local
!
end
11-05-2010 03:24 AM
Karen
By now we decided used the two default routes in our environment, so I couldn't use your solution.
But there is some routers available here, and when a get some time I will make a physical lab.
I will inform you of the results.
Again, congrats for the great solution, and thanks for the help.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide