Routing between IPSec tunnels

[mwwg]

I have a cisco1841, i have created 2 ipsec tunnel to branch sites, & its working fine, but i discoved the form hub to spoke its working fine but from spoke to spoke its not working, & it became worse when i added Cisco VPN clients to the network as they too could use data on the hub site but not on the spoke sites.

please any help is apperciated, i am attaching my config

[jackyoung]

Could you please provide the spoke site config. ?

Why there is static and rip for same subnet ?

[mwwg]

dear sir, as for having static & rip, i am sorry i was experminting all options as i am not an expert with cisco devices.

the spoke are not cisco router, but they were working OK before i replaced the old router with the Cisco 1841, so i assume they are not the problem especially when i found the VPN client users facing same problem.

even ping from CLI of the router to the subnets connected using VPN is not working

[jackyoung]

Thanks for your clarification. I asked for the config. that is due to I want to clarify how the routing works in this network.

Could you please advise how the routing works ? e.g. Use static, rip, can you provide "show ip route" at the Cisco router ?

[mwwg]

thanks for your reply, i see i have posted my config on the first time, so i hope you could have a look on it

i used RIP 1st but when i found i ahving problems i began setting statis routes hoping to avoid such problems.

Cisco1841#sh ip route

Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area

N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2

E1 - OSPF external type 1, E2 - OSPF external type 2

i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2

ia - IS-IS inter area, * - candidate default, U - per-user static route

o - ODR, P - periodic downloaded static route

Gateway of last resort is 62.135.105.161 to network 0.0.0.0

S 192.168.85.0/24 is directly connected, FastEthernet0/0

S 192.168.1.0/24 is directly connected, FastEthernet0/0

62.0.0.0/29 is subnetted, 1 subnets

C 62.135.105.160 is directly connected, FastEthernet0/0

C 192.168.100.0/24 is directly connected, FastEthernet0/1

S 192.168.101.0/24 [1/0] via 192.168.100.199

S* 0.0.0.0/0 [1/0] via 62.135.105.161

here is trace from burg router to mercia

Borg#trace 192.168.85.254

Type escape sequence to abort.

Tracing the route to 192.168.85.254

1 10.10.10.1 20 msec 24 msec 36 msec

2 * * *

3 * * *

also i got the cisco 1600 router connecting spoke to burg site config from the service provider, but it was working with old router & they said it wasn't changed since the installation 2 years ago.

thanks for your help, waiting for your reply

[jackyoung]

According to the config. What I think the diagram as below :

1841<-->Alex<-->Burg

Where the host in Burg try to trace a host with 192.168.85.254.

But I cannot find the router which is connecting to 192.168.85.0 subnet, could you please provide the router's config which is connecting to 192.168.85.0 ? It looks like the host of 192.168.85.254 do know how to return the packet to 1841 then forward back to Alex & Burg.

Moreover, you can remove below static route in Burg that due to the default route already included this subnet unless it is different next-hop.

ip route 192.168.100.0 255.255.255.0 10.10.10.

I suggest you can draw a network diagram w/ IP address assignment and trace the paths in the diagram, then you will have a clear idea how to configure the static route to make it work. If you can, you can simple enable dynamic routing protocol (RIP or EIGRP if it supports)to simplify the configuration and design.

Wait for your reply.

[mwwg]

it seems i didn't make a clear diagram for the network, i will try again

burg-(192.168.101.x)c1600(10.10.10.2)<-->(10.10.10.1) [ Alex-c1600 (192.168.100.199) <--> (192.168.100.2) C1841 ] (WAN - public IP) <--> { internet } <-->(MERCIA public IP) mercia (192.168.85.x) (3rd party router) & (CAIRO public IP) Cairo (192.168.1.x)(3rd part router)

so Main office contains [ Alex router cisco 1600 & the Cisco 1841 ], where burg have cisco 1600 & both cairo & mercia have a 3rd party router.

as for RIP its supported by routers that's why i enabled it, but i don't know why it's not working

thanks for pointing me to this ip route thing i did remove that line from the configuration & i am enabling rip on both Cisco 1600 routers

thanks for your help i hope you would bear with me until this is solved, thanks

[jackyoung]

You're welcome and thanks for the clarification. Please advise the result after you fine tune the config.

For the RIP, it was because there is static route in the network and it is preferred, so the routing table show static. If you remove the static, you will find the RIP.

However, there are many limitation in RIP, you may consider to use RIPv2, EIGRP (but not supported at non-Cisco router). Or you still can use static route but not to include the "permanent" parameter, it was because if you add this parameter, the static will still active even the next-hop of the static is down. Therefore, if the next-hop down, but static route still active, the traffic will flow into the black hole....

I am sorry that I don't understand the diagram and you said main office include Alex & 1841 but burg inclyde cario & merica. Could you please draw it out w/ ip address with Visio or similar program ?

What I expect the packet should arrive 10.0.0.1 then 192.168.100.2 at 1841...

Wait for your result.

[mwwg]

sorry i am very bad in explaining myself & diagraming

burg(c1600) <--> [ main (alex) office (c1600 <-> c1841) ] <--> [ internet ] <--> [ cairo ] & [ merica ]

its alex (main office) to burg using c1600 & a frame relay VPN , then also in alex using the c1841 to internet using 2 IPSec tunnels to Cairo & to Mercia.

too bad it still not working out as you said there is a black hole, preventing the communication. i have attached results from both side mercia & Burg sites, also with a black hole in c1841 IP.

using extend ping on burg router i discovered that in burg router CLI (c1600) pinging using WAN IP failed but using LAN IP succeed, i did include burg WAN IP in ACL on c1841 but still it fails, same goes from C1841 CLI to any of the IPSec tunneled sites, but not to burg site. do i need to add their WAN IP's too, also i am ping a LAN client / port?

thanks for bearing with me

[johnnylingo]

I agree a visual network diagram would really help. Also, I agree to make sure you are running RIPv2 on all interfaces. Easiest way to do this is:

router rip

version 2

network 0.0.0.0

no auto-summary

!

Also, you only want to use "passive-interface" for devices connected to networks outside yours, such as ones that connect to an external ISP.

[jackyoung]

According to the captures & config., I have below comments:

1) There is no need to have below config. at 1841, due to they are point to the ADSL WAN. Please clarify :

ip route 192.168.1.0 255.255.255.0 FastEthernet0/0 permanent

ip route 192.168.85.0 255.255.255.0 FastEthernet0/0 permanent

2) Please double confirm the IPSEC tunnels between 1841, cairo & merica are up.

3) The config. in burg looks fine but also better to remove below due to it is provided by the default route :

ip route 192.168.100.0 255.255.255.0 10.10.10.1

4) The config. in Alex is fine.

5) The reason not able to ping from WAN of burg that is due to the static route of berg WAN (10.10.10.x) is not defined at 1841. The 10.10.10.x packet will follow the default route and forward to Internet.

6) Please clarify below static routes in merica that why are they pointing to the1841's internal LAN for next-hop. I believe it should be the WAN of Internet at remote side or IF3. Please confirm it is correct or not. It is one of the key factor that these static routes tell the router how to route to burg's LAN, if we setup incorrect next-hop, if will fail.

S 192.168.101.0/ 255.255.255.0 via 192.168.100.2, IF0

S 192.168.1.0/ 255.255.255.0 via 192.168.100.2, IF0

7) Please try to ping from burg to cairo and obtain the result.

Please confirm the IPSEC is up and the static route at merica & cairo are correct.

Hope this helps. Good luck.

[mwwg]

thanks so much for your reply & the other reply too, it was so helpful, as pointing me to use RIP was very helpful, here is what i did:

1- i removed nearly all static routes

2- i doubled check IPSec tunnels, as they are established automatically on need,thus, they are always on.

3- i guess yes, there is a NAT problem as after RIP worked, traceing packets out burg went to internet, before, it was just lost?!!

4- the c1841 is the hub for all spokes / routers, so in mercia router its the next hop, but i removed all static routes & left it for RIP

5- cairo is same as mercia(same router, tunnel, ..etc)

thanks, i am now looking to troubleshoot the NAT on burg, besdie, reveiwing the RIP & routes established.

thanks a lot

[jackyoung]

You're welcome. It is great that we isolate the problem.

What I mean to remove the static that is because the default route already included those subnet and there is other subnet or path, so we can remove it.

Do you mean the NAT is done on Berg not on the 1841 ? I believe the NAT should be enabled at the exit point to the Internet.

If the tunnels are active, it means the connectivity is fine and only the routing issue.

If there is still the problem, please provide the routing table at all routers.

Hope this helps.

[ilya.varlashkin]

I think following happens:

when you ping from Burg router using its WAN IP, then c1841 doesn't have route back to that address, so any traceroute or ping sourced from Burg's WAN IP will be able to send packets out but will never be able to receive packets because nobody except Alex knows what to do with packets for 10.10.10.0/30 network. You need to add a route on c1841 pointing to this network via 192.168.100.199. Adding 10.10.10.0/30 to extended access list Mercia (for IP sec) is also required, but you said you've just done it. So you need both routing entry for that network and entry in the access list for this network.

Second problem I see in your network setup is that network 192.168.100.0/24 is configured on LAN iterfaces of c1841&Alex, but also on IF4 of Mercia. In your current configuration there is no way traffic will for this network would be exchanged between Mercia and c1841/Alex. That's why ping from Mercia to 192.168.100.2 (c1841) fails. The only way to make it working is if bridging would be configured all the way between c1841 and Mercia, though I wouldn't expect Mercia to be capable of bridging (just guessing the model of the router based on your provided CLI output). And without using GRE tunnels I believe Cisco doesn't support briding over IPsec too. So you need to remove network 192.168.100.0/24 from either Mercia or Alex/c1841 configuration(s) and use something else there.

These two issues above are apparently your main problem, and they're reason why you don't have communication (besides routes at Mercia pointing to private IP of c1841, but that might work if IPsec makes you way to reach 192.168.100.2). But there's one more obscure thing in your setup.

Based on parts of the configs you've provided for Alex and Burg I don't see any value in having 'bridge group' configured on any interface. It's not an error per see, but since it's doing nothing (unless there's something you didn't mention) I'd remove it for clarity sake.

Please let me know if this resolves the problem.