09-15-2010 09:07 AM - edited 02-21-2020 04:50 PM
Hi,
I have an issues with an access-list configured in INBOUND on Tunnel interface at the HUB routers (HUB1 and HUB2).
I attach a simple drawing of my setup:Drawing-Lab-Setup.Jpeg
Let me explain my configuration quickly:
Security requirements:
In order to meet the security requirements and simplify at the maximum the configuration on the spokes I thought that I could implement an inbound access-list on the tunnel interface at HUB1 and HUB2. So Like that everytime I add a new spoke I don't have to configure more lines in the spoke config. I attach the access-list which I have configured on HUB1 and HUB2 and also the configuration of the tunnel interface (only for HUB1, HUB 2 is the same).
DMVPN-TunnelIN-Acl-And-TunnelConf.txt
My isssue starts here. When I apply the access-list which is called DMVPN_INSIDE_IN on the tunne interface, spokes can ping the HUB location, no problem. The issue is when a spoke host try to reach the Internet (ping from 192.168.100.2) in this case 200.200.200.200 (see drawing) the access-list deny the packet by saying the following:
%SEC-6-IPACCESSLOGDP: list DMVPN_INSIDE_IN denied icmp 80.10.10.2 -> 200.200.200.200 (8/0), 1 packet
But the firewall sees actually the right source address before to be natted:
%FW-6-SESS_AUDIT_TRAIL_START: Start icmp session: initiator (192.168.100.2:8) -- responder (200.200.200.200:0)
If I remove the access-list everything is working fine! It looks like the access-list is inspecting the packet after the NAT process. Actually sometimes is working sometimes not. If I remove the access-list and put it back on again 192.168.100.2 can ping 200.200.200.200 without problem.
So what I don't understand is how I can apply this access-list on the tunnel interface? Should it be OUTBOUND instead of INBOUND? I don't really understand the Cisco IOS process here. How the Tunnel Interface viewed in this case?
Any ideas what is it happening here?
Best regards,
Laurent
Solved! Go to Solution.
09-16-2010 12:22 AM
Laurent,
Looks to be related to CEF then (at least to me not knowing too much about it). Probably now a valid adjacency is installed and it will work until it's removed from FIB for whatever reason.
A good test would be to check if it will keep working after you remove and add cef, or was is just some minor problem with access-lists.
Marcin
09-15-2010 10:17 AM
Laurent,
I'm not aware of any order of operation change when using tunnel interface and NAT should be done after access-list checks according to the best of my knowledge.
What software are you running on the hubs?
I would advise to add an explicit permit on the access-list
line 1 permit ip h local_ip h external_ip
line 2 permit ip h global_ip h external_ip
and re-run the test.
As I notice you don't have inspection enabled inbound on this tunnel interface - meaning that for one reason or another the inspection engine on egress interface (in out direction?).
Maybe problem with routing?
Marcin
09-15-2010 10:24 AM
Just to add clarity of what I'm aiming at:
http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080133ddd.shtml
In your case
check input access list - ? routing - done. NAT inside to outside (local to global translation) - done. inspect (Context-based Access Control (CBAC)) - done.
And we see input access-list check for NATed IP address of source ... it's like we're hitting the tunnel ACL again.
09-15-2010 12:19 PM
Hi Marcin,
Thank you for your post!
I have tested on 2 different Cisco router platforms:
On C2811 I am running IOS c2800nm-advipservicesk9-mz.124-24.T3.bin
On C2921 I am running IOSc2900-universalk9-mz.SPA.150-1.M1.bin with securityk9+ipbasek9
As you asked I have added this 2 lines to the access-list and the re-run was successful!
permit ip host 192.168.100.2 host 200.200.200.200
permit ip host 80.10.10.2 host 200.200.200.200
I re-run the test : ping 200.200.200.200 from 192.168.100.2
Pinging 200.200.200.200 with 32 bytes data:
Reply from 200.200.200.200: byte=32 tid=160ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125
Reply from 200.200.200.200: byte=32 tid=157ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125
Here is the output on HUB1:
Start icmp session: initiator (192.168.100.2:8) -- responder (200.200.200.200:0)
Stop icmp session: initiator (192.168.100.2:8) sent 128 bytes -- responder (200.200.200.200:0) sent 128 bytes
And there is match for the two lines added in the ACL:
Extended IP access list DMVPN_INSIDE_IN
10 permit ip host 192.168.100.2 host 200.200.200.200 (4 matches)
20 permit ip host 80.10.10.2 host 200.200.200.200 (1 match)
30 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
40 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (30 matches)
50 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (20 matches)
60 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
70 permit tcp 192.168.0.0 0.0.255.255 any eq www
80 permit tcp 192.168.0.0 0.0.255.255 any eq 443
90 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
100 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
110 permit icmp 192.168.0.0 0.0.255.255 any echo
120 permit icmp 192.168.0.0 0.0.255.255 any traceroute
130 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
140 permit icmp 192.168.0.0 0.0.255.255 any unreachable
150 deny ip any any log
That is strange because there is only one match for 80.10.10.2. It seems that the first packet with global source has to be authorized but not the others!
My CBAC inspect is outbounf on Outside physical interface on HUB1.
If I remove th access-list everything is working perfect! So I don't know what is happening with this access-list!
Any more ideas?
Regards,
Laurent
09-15-2010 02:32 PM
Laurent,
OK the releases seem to be fairly recent... that's always good.
My concern is that in my opinion we should not see the second entry at all, not on "inside".
I think what we're actually seeing is (probably) first packet being switched differently then others.
ie. typical router ping is 5 probes. In the output you attached 4 pass and 1 fails if I interpret this correctly.
Are you able to turn off CEF and test? Alternatively you can enable "log" keyword on access-list entries which should prohibit CEF from being used a process switching to be used. Again for testing purposes.
Marcin
09-15-2010 10:37 PM
Hi Marcin,
I didn't know that CEF was beeing disabled when the "log" keyword was added to an access-list entry! Interesting.
So as you adviced me I add the "log" keyword on the 2 first entries so the access--list is looking like that now:
Extended IP access list DMVPN_INSIDE_IN
10 permit ip host 192.168.100.2 host 200.200.200.200 log
20 permit ip host 80.10.10.2 host 200.200.200.200 log
30 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
40 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255
50 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (4 matches)
60 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
70 permit tcp 192.168.0.0 0.0.255.255 any eq www
80 permit tcp 192.168.0.0 0.0.255.255 any eq 443
90 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
100 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
110 permit icmp 192.168.0.0 0.0.255.255 any echo
120 permit icmp 192.168.0.0 0.0.255.255 any traceroute
130 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
140 permit icmp 192.168.0.0 0.0.255.255 any unreachable
150 deny ip any any log
I re-run the test : ping 200.200.200.200 from 192.168.100.2
Pinging 200.200.200.200 with 32 bytes data:
Reply from 200.200.200.200: byte=32 tid=160ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125
Reply from 200.200.200.200: byte=32 tid=157ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125
Here is the output on HUB1:
SEC-6-IPACCESSLOGDP: list DMVPN_INSIDE_IN permitted icmp 192.168.100.2 -> 200.200.200.200 (8/0), 1 packet
FW-6-SESS_AUDIT_TRAIL_START: Start icmp session: initiator (192.168.100.2:8) -- responder (200.200.200.200:0)
And this time there is no match for the second entry as you can see here:
Extended IP access list DMVPN_INSIDE_IN
10 permit ip host 192.168.100.2 host 200.200.200.200 log (4 matches)
20 permit ip host 80.10.10.2 host 200.200.200.200 log
30 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
40 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (8 matches)
50 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (28 matches)
60 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
70 permit tcp 192.168.0.0 0.0.255.255 any eq www
80 permit tcp 192.168.0.0 0.0.255.255 any eq 443
90 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
100 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
110 permit icmp 192.168.0.0 0.0.255.255 any echo
120 permit icmp 192.168.0.0 0.0.255.255 any traceroute
130 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
140 permit icmp 192.168.0.0 0.0.255.255 any unreachable
150 deny ip any any log
You said there are 5 probes so I don't know where number five went?;-) So as you said it seems the first packet is switched differently. Why is that? Do you know?
What is the best solution to solve this problem?
Best Regards,
Laurent
09-15-2010 11:08 PM
Hi again,
Actually it is really strange becasue if I remove the 2 first lines from the access-list:
Extended IP access list DMVPN_INSIDE_IN
10 permit ip host 192.168.100.2 host 200.200.200.200 log
20 permit ip host 80.10.10.2 host 200.200.200.200 log
It is actually working now! Here is the result:
Extended IP access list DMVPN_INSIDE_IN
10 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
20 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (90 matches)
30 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (156 matches)
40 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
50 permit tcp 192.168.0.0 0.0.255.255 any eq www
60 permit tcp 192.168.0.0 0.0.255.255 any eq 443
70 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
80 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
90 permit icmp 192.168.0.0 0.0.255.255 any echo (8 matches)
100 permit icmp 192.168.0.0 0.0.255.255 any traceroute
110 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
120 permit icmp 192.168.0.0 0.0.255.255 any unreachable
130 deny ip any any log
So sometimes it is working sometimes not.....
Regards,
Laurent
09-16-2010 12:22 AM
Laurent,
Looks to be related to CEF then (at least to me not knowing too much about it). Probably now a valid adjacency is installed and it will work until it's removed from FIB for whatever reason.
A good test would be to check if it will keep working after you remove and add cef, or was is just some minor problem with access-lists.
Marcin
09-16-2010 12:33 AM
Hi Marcin,
Actually if I reload the router and leave the access-list like this then is not working again:
Extended IP access list DMVPN_INSIDE_IN
10 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
20 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (1001 matches)
30 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (650 matches)
40 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1 (25 matches)
50 permit tcp 192.168.0.0 0.0.255.255 any eq www
60 permit tcp 192.168.0.0 0.0.255.255 any eq 443
70 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
80 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
90 permit icmp 192.168.0.0 0.0.255.255 any echo (1263 matches)
100 permit icmp 192.168.0.0 0.0.255.255 any traceroute
110 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
120 permit icmp 192.168.0.0 0.0.255.255 any unreachable
130 deny ip any any log (24 matches)
And the access-list is denying the traffic again:
SEC-6-IPACCESSLOGDP: list DMVPN_INSIDE_IN denied icmp 80.10.10.2 -> 200.200.200.200 (8/0), 17 packets
But if I disable CEF on tunnel interface than everything is working find!
interface Tunnel1
bandwidth 100000
ip address 10.1.0.1 255.255.255.0
ip access-group DMVPN_INSIDE_IN in
no ip redirects
ip mtu 1400
ip nat inside
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 100000
ip nhrp holdtime 600
ip nhrp redirect
ip virtual-reassembly
no ip route-cache cef
ip tcp adjust-mss 1380
no ip split-horizon eigrp 1
ip summary-address eigrp 1 0.0.0.0 0.0.0.0 5
delay 5
qos pre-classify
tunnel source FastEthernet0/0
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile vpn
But what will be the impact of disabling CEF on the router? the customer has more or less 50 remote sites.
Should I open a TAC case?
Regards,
Laurent
09-16-2010 12:40 AM
Laurent,
I would definetly advise to open a TAC case.
Wihout CEF switching we will rely on CPU cycles to move packets around, so depending on traffic volume - you will just see CPU a bit higher or even high CPU due to interrupts.
I think you can open a TAC case and point engineer here for they way you've narrowed things down.
Please make sure
technology is Routing Protocols (Includes NAT and HSRP)
sub-technology is CEF (Cisco Express Forwarding) Switching
Once you create it, please let me know the number.Ffor sake of curiosity I'll be spying on it ;-)
Marcin
09-16-2010 06:54 AM
Hi,
I attach the TAC case number: 615471811
Best Regards and thanks for your help!
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide