DMVPN AND INBOUND TUNNEL ACL:Technical Question for Expert

Answered Question

Hi,

I have an issues with an access-list configured in INBOUND on Tunnel interface at the HUB routers (HUB1 and HUB2).

I attach a simple drawing of my setup:Drawing-Lab-Setup.Jpeg


Let me explain my configuration quickly:

  • I have configureddual HUB and DUAL DMVPN layout

  • DMVPN phase 3 is configured and I use EIGRP
  • All the traffic is going (including Internet) through HUB location
  • All spokes are configured with FVRF and receive only a default route from HUB routers
  • Spoke to spoke traffic is possible and can be restricted if necessary by  configuring a route to Null on spokes router
  • HUB1 is the primary router and HUB2 is the backup router

Security requirements:

  • Spokes access Internet through HUB and are only permitted to access HTTP,HTTPS,FTP and ICMP
  • Spokes can access everything at HUB location

In order to meet the security requirements and simplify at the maximum the configuration on the spokes I thought that I could implement an inbound access-list on the tunnel interface at HUB1 and HUB2. So Like that everytime I add a new spoke I don't have to configure more lines in the spoke config. I attach the access-list which I have configured on HUB1 and HUB2 and also the configuration of the tunnel interface (only for HUB1, HUB 2 is the same).

DMVPN-TunnelIN-Acl-And-TunnelConf.txt

My isssue starts here. When I apply the access-list which is called DMVPN_INSIDE_IN on the tunne interface, spokes can ping the HUB location, no problem. The issue is when a spoke host try to reach the Internet (ping from 192.168.100.2) in this case 200.200.200.200 (see drawing) the access-list deny the packet by saying the following:


%SEC-6-IPACCESSLOGDP: list DMVPN_INSIDE_IN denied icmp 80.10.10.2 -> 200.200.200.200 (8/0), 1 packet

But the firewall sees actually the right source address before to be natted:


%FW-6-SESS_AUDIT_TRAIL_START: Start icmp session: initiator (192.168.100.2:8) -- responder (200.200.200.200:0)

If I remove the access-list everything is working fine! It looks like the access-list is inspecting the packet after the NAT process. Actually sometimes is working sometimes not. If I remove the access-list and put it back on again 192.168.100.2 can ping 200.200.200.200 without problem.

So what I don't understand is how I can apply this access-list on the tunnel interface? Should it be OUTBOUND instead of INBOUND? I don't really understand the Cisco IOS process here. How the Tunnel Interface viewed in this case?

Any ideas what is it happening here?

Best regards,

Laurent

I have this problem too.
0 votes
Correct Answer by Marcin Latosiewicz about 6 years 2 months ago

Laurent,

Looks to be related to CEF then (at least to me not knowing too much about it). Probably now a valid adjacency is installed and it will work until it's removed from FIB for whatever reason.

A good test would be to check if it will keep working after you remove and add cef, or was is just some minor problem with access-lists.

Marcin

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Marcin Latosiewicz Wed, 09/15/2010 - 10:17

Laurent,

I'm not aware of any order of operation change when using tunnel interface and NAT should be done after access-list checks according to the best of my knowledge.

What software are you running on the hubs?

I would advise to add an explicit permit on the access-list

line 1 permit ip h local_ip h external_ip

line 2 permit ip h global_ip h external_ip

and re-run the test.

As I notice you don't have inspection enabled inbound on this tunnel interface - meaning that for one reason or another the inspection engine on egress interface (in out direction?).

Maybe problem with routing?

Marcin

Marcin Latosiewicz Wed, 09/15/2010 - 10:24

Just to add clarity of what I'm aiming at:

http://www.cisco.com/en/US/tech/tk648/tk361/technologies_tech_note09186a0080133ddd.shtml

In your case

  • check input access list - ?

  • routing - done.

  • NAT inside to outside (local to global                       translation) - done.

  • inspect (Context-based Access Control                       (CBAC)) - done.

And we see input access-list check for NATed IP address of source ... it's like we're hitting the tunnel ACL again.

Hi Marcin,

Thank you for your post!

I have tested on 2 different Cisco router platforms:

On C2811 I am running IOS c2800nm-advipservicesk9-mz.124-24.T3.bin

On C2921 I am running IOSc2900-universalk9-mz.SPA.150-1.M1.bin with securityk9+ipbasek9

As you asked I have added this 2 lines to the access-list and the re-run was successful!

permit ip host 192.168.100.2 host 200.200.200.200

permit ip host 80.10.10.2 host 200.200.200.200

I re-run the test : ping 200.200.200.200 from 192.168.100.2

Pinging 200.200.200.200 with 32 bytes data:

Reply from 200.200.200.200: byte=32 tid=160ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125
Reply from 200.200.200.200: byte=32 tid=157ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125

Here is the output on HUB1:


Start icmp session: initiator (192.168.100.2:8) -- responder (200.200.200.200:0)

Stop icmp session: initiator (192.168.100.2:8) sent 128 bytes -- responder (200.200.200.200:0) sent 128 bytes

And there is match for the two lines added in the ACL:

Extended IP access list DMVPN_INSIDE_IN

    10 permit ip host 192.168.100.2 host 200.200.200.200 (4 matches)

    20 permit ip host 80.10.10.2 host 200.200.200.200 (1 match)

    30 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255

    40 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (30 matches)

    50 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (20 matches)

    60 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1

    70 permit tcp 192.168.0.0 0.0.255.255 any eq www

    80 permit tcp 192.168.0.0 0.0.255.255 any eq 443

    90 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big

    100 permit icmp 192.168.0.0 0.0.255.255 any echo-reply

    110 permit icmp 192.168.0.0 0.0.255.255 any echo

    120 permit icmp 192.168.0.0 0.0.255.255 any traceroute

    130 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded

    140 permit icmp 192.168.0.0 0.0.255.255 any unreachable

    150 deny ip any any log

That is strange because there is only one match for 80.10.10.2. It seems that the first packet with global source has to be authorized but not the others!

My CBAC inspect is outbounf on Outside physical interface on HUB1.

If I remove th access-list everything is working perfect! So I don't know what is happening with this access-list!

Any more ideas?

Regards,

Laurent

Marcin Latosiewicz Wed, 09/15/2010 - 14:32

Laurent,

OK the releases seem to be fairly recent... that's always good.

My concern is that in my opinion we should not see the second entry at all, not on "inside".

I think what we're actually seeing is (probably) first packet being switched differently then others.

ie. typical router ping is 5 probes. In the output you attached 4 pass and 1 fails if I interpret this correctly.

Are you able to turn off CEF and test? Alternatively you can enable "log" keyword on access-list entries which should prohibit CEF from being used a process switching to be used. Again for testing purposes.

Marcin

Hi Marcin,

I didn't know that CEF was beeing disabled when the "log"  keyword was added to an access-list entry! Interesting.

So as you adviced me I add the "log" keyword on the 2 first entries so the access--list is looking like that now:

Extended IP access list DMVPN_INSIDE_IN
    10 permit ip host 192.168.100.2 host 200.200.200.200 log
    20 permit ip host 80.10.10.2 host 200.200.200.200 log

    30 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
    40 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255
    50 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (4 matches)
    60 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
    70 permit tcp 192.168.0.0 0.0.255.255 any eq www
    80 permit tcp 192.168.0.0 0.0.255.255 any eq 443
    90 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
    100 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
    110 permit icmp 192.168.0.0 0.0.255.255 any echo
    120 permit icmp 192.168.0.0 0.0.255.255 any traceroute
    130 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
    140 permit icmp 192.168.0.0 0.0.255.255 any unreachable
    150 deny ip any any log

I re-run the test : ping 200.200.200.200 from 192.168.100.2

Pinging  200.200.200.200 with 32 bytes data:

Reply from 200.200.200.200:  byte=32 tid=160ms TTL=125
Reply from 200.200.200.200: byte=32 tid=156ms TTL=125
Reply  from 200.200.200.200: byte=32 tid=157ms TTL=125
Reply  from 200.200.200.200: byte=32 tid=156ms TTL=125

Here is the  output on HUB1:


SEC-6-IPACCESSLOGDP: list DMVPN_INSIDE_IN permitted icmp 192.168.100.2 -> 200.200.200.200 (8/0), 1 packet 


FW-6-SESS_AUDIT_TRAIL_START: Start icmp session: initiator (192.168.100.2:8) -- responder (200.200.200.200:0)

And this time there is no match for the second entry as you can see here:

Extended IP access list DMVPN_INSIDE_IN
   10 permit ip host 192.168.100.2 host 200.200.200.200 log (4 matches)
   20 permit ip host 80.10.10.2 host 200.200.200.200 log
    30 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
    40 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (8 matches)
    50 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (28 matches)
    60 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
    70 permit tcp 192.168.0.0 0.0.255.255 any eq www
    80 permit tcp 192.168.0.0 0.0.255.255 any eq 443
    90 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
    100 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
    110 permit icmp 192.168.0.0 0.0.255.255 any echo
    120 permit icmp 192.168.0.0 0.0.255.255 any traceroute
    130 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
    140 permit icmp 192.168.0.0 0.0.255.255 any unreachable
    150 deny ip any any log

You said there are 5 probes so I don't know where number five went?;-) So as you said it seems the first packet is switched differently. Why is that? Do you know?

What is the best solution to solve this problem?

Best Regards,

Laurent

Hi again,

Actually it is really strange becasue if I remove the 2 first lines from the access-list:

Extended IP access list DMVPN_INSIDE_IN
    10 permit ip host  192.168.100.2 host 200.200.200.200 log
     20 permit ip host 80.10.10.2 host 200.200.200.200 log

It is actually working now! Here is the result:

Extended IP access list DMVPN_INSIDE_IN
    10 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
    20 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (90 matches)
    30 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (156 matches)
    40 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1
    50 permit tcp 192.168.0.0 0.0.255.255 any eq www
    60 permit tcp 192.168.0.0 0.0.255.255 any eq 443
    70 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
    80 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
    90 permit icmp 192.168.0.0 0.0.255.255 any echo (8 matches)
    100 permit icmp 192.168.0.0 0.0.255.255 any traceroute
    110 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
    120 permit icmp 192.168.0.0 0.0.255.255 any unreachable
    130 deny ip any any log

So sometimes it is working sometimes not.....

Regards,

Laurent

Correct Answer
Marcin Latosiewicz Thu, 09/16/2010 - 00:22

Laurent,

Looks to be related to CEF then (at least to me not knowing too much about it). Probably now a valid adjacency is installed and it will work until it's removed from FIB for whatever reason.

A good test would be to check if it will keep working after you remove and add cef, or was is just some minor problem with access-lists.

Marcin

Hi Marcin,

Actually if I reload the router and leave the access-list like this then is not working again:

Extended IP access list DMVPN_INSIDE_IN
    10 permit ip 192.168.0.0 0.0.255.255 10.1.100.0 0.0.0.255
    20 permit ip 10.10.10.0 0.0.0.255 10.1.100.0 0.0.0.255 (1001 matches)
    30 permit eigrp 10.1.0.0 0.0.0.255 host 224.0.0.10 (650 matches)
    40 permit eigrp 10.1.0.0 0.0.0.255 host 10.1.0.1 (25 matches)
    50 permit tcp 192.168.0.0 0.0.255.255 any eq www
    60 permit tcp 192.168.0.0 0.0.255.255 any eq 443
    70 permit icmp 192.168.0.0 0.0.255.255 any packet-too-big
    80 permit icmp 192.168.0.0 0.0.255.255 any echo-reply
    90 permit icmp 192.168.0.0 0.0.255.255 any echo (1263 matches)
    100 permit icmp 192.168.0.0 0.0.255.255 any traceroute
    110 permit icmp 192.168.0.0 0.0.255.255 any time-exceeded
    120 permit icmp 192.168.0.0 0.0.255.255 any unreachable
    130 deny ip any any log (24 matches)

And the access-list is denying the traffic again:

SEC-6-IPACCESSLOGDP: list DMVPN_INSIDE_IN denied icmp 80.10.10.2 -> 200.200.200.200 (8/0), 17 packets

But if I disable CEF on tunnel interface than everything is working find!

interface Tunnel1
bandwidth 100000
ip address 10.1.0.1 255.255.255.0
ip access-group DMVPN_INSIDE_IN in
no ip redirects
ip mtu 1400
ip nat inside
ip nhrp authentication cisco
ip nhrp map multicast dynamic
ip nhrp network-id 100000
ip nhrp holdtime 600
ip nhrp redirect
ip virtual-reassembly
no ip route-cache cef
ip tcp adjust-mss 1380
no ip split-horizon eigrp 1
ip summary-address eigrp 1 0.0.0.0 0.0.0.0 5
delay 5
qos pre-classify
tunnel source FastEthernet0/0
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile vpn

But what will be the impact of disabling CEF on the router? the customer has more or less 50 remote sites.

Should I open a TAC case?

Regards,

Laurent

Marcin Latosiewicz Thu, 09/16/2010 - 00:40

Laurent,

I would definetly advise to open a TAC case.


Wihout CEF switching we will rely on CPU cycles to move packets around, so depending on traffic volume - you will just see CPU a bit higher or even high CPU due to interrupts.

I think you can open a TAC case and point engineer here for they way you've narrowed things down.

Please make sure

technology is Routing Protocols (Includes NAT and HSRP)

sub-technology is CEF (Cisco Express Forwarding) Switching

Once you create it, please let me know the number.Ffor sake of curiosity I'll be spying on it ;-)

Marcin

Actions

This Discussion