cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1891
Views
0
Helpful
13
Replies

Broken Access List applied to Layer 3 SVI (VLAN interface)

Paul Wedde
Level 1
Level 1

Hi everyone,

I've just applied an ACL to a VLAN interface across 3 switches and have observed some very strange behaviour.

The ACL I've applied is:

access-list 101 permit ip 192.168.1.0 0.0.0.7 192.168.1.0 0.0.0.7

Effectively I'm just trying to restrict communication between these 3 switches on 192.168.1.1, .2, .3 and 2 routers on .4 & .5 address.

As soon as I apply this ACL to the VLAN interface (inbound):

interface vlan 101

ip access-group 101 in

Traffic bound for these addresses on .1 and .3 stops working (as expected) but miraculously traffic bound for the .2 address continues to pass!

I've modified the list on the .2 address to this:

access-list 101 permit ip 192.168.1.0 0.0.0.7 192.168.1.0 0.0.0.7 log

access-list 101 deny ip any any log

..to attempt to understand better what is going on. I can see the hit count incrementing on the first entry when I ping from another 192.168.1.x address so this indicates to me that the ACL is infact being hit but the deny any any statment is not being hit and traffic from any other address is still being allowed through.

The only difference between the 3 switches that I have applied this list to is that 192.168.1.1 and .3 both have physical interfaces assigned to VLAN 101 where as 192.168.1.2 does not.

Doe's anyone know what might be causing this behaviour?

Cheers.

13 Replies 13

abmehta
Level 1
Level 1

Hi Paul,

This is very odd, Could you share how  it is connected the configurations you have applied on the .1 & .3 interfaces of the switches.

Also understanding from the description i see 192.168.1.1 , 1.2 & 1.3 are all IP addresses of different switches connected directly is that right?

Ganesh Hariharan
VIP Alumni
VIP Alumni

Hi everyone,

I've just applied an ACL to a VLAN interface across 3 switches and have observed some very strange behaviour.

The ACL I've applied is:

access-list 101 permit ip 192.168.1.0 0.0.0.7 192.168.1.0 0.0.0.7

Effectively I'm just trying to restrict communication between these 3 switches on 192.168.1.1, .2, .3 and 2 routers on .4 & .5 address.

As soon as I apply this ACL to the VLAN interface (inbound):

interface vlan 101

ip access-group 101 in

Traffic bound for these addresses on .1 and .3 stops working (as expected) but miraculously traffic bound for the .2 address continues to pass!

I've modified the list on the .2 address to this:

access-list 101 permit ip 192.168.1.0 0.0.0.7 192.168.1.0 0.0.0.7 log

access-list 101 deny ip any any log

..to attempt to understand better what is going on. I can see the hit count incrementing on the first entry when I ping from another 192.168.1.x address so this indicates to me that the ACL is infact being hit but the deny any any statment is not being hit and traffic from any other address is still being allowed through.

The only difference between the 3 switches that I have applied this list to is that 192.168.1.1 and .3 both have physical interfaces assigned to VLAN 101 where as 192.168.1.2 does not.

Doe's anyone know what might be causing this behaviour?

Cheers.

Hi,

Can you share the diagramtic vew of your architurecure,to understand little bit more on the problem.

Ganesh.H

Hi guys,

I've also done a "debug ip icmp" to confirm source:

Apr 30 08:30:19: ICMP: echo reply sent, src 192.168.1.2, dst [IP Address of another device]

I've run some extended pings from the 192.168.1.2 device to itself:

L3SWITCH2#ping
Protocol [ip]:
Target IP address: 192.168.1.2
Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface: 192.168.1.2

Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
Packet sent with a source address of 192.168.1.2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
L3SWITCH2#
L3SWITCH2#
Apr 30 08:32:30: %SEC-6-IPACCESSLOGDP: list 101 permitted icmp 192.168.1.2 -> 192.168.1.2 (0/0), 1 packet

L3SWITCH2#ping
Protocol [ip]:
Target IP address: 192.168.1.2

Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface: 172.16.0.2
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
Packet sent with a source address of 172.16.0.2
U
Apr 30 08:33:14: %SEC-6-IPACCESSLOGDP: list 101 denied icmp 172.16.0.2 -> 192.168.1.2 (0/0), 1 packet.U.U
Success rate is 0 percent (0/5)
L3SWITCH2#

So interestingly, If I run a ping from the device itself but use a different interface then they packets are dropped and logged by the ACL. I'm still unsure as to why packets from outside the device are allowed though.

Diagramatically speaking these 3 switches are all connected together. All have L2 access to each other and are physically cabled together. I'm not sure which parts I should represent in a diagram as the configuration for these devices is quite complex and I feel I should be trying to target only the relevant parts.

Jon Marshall
Hall of Fame
Hall of Fame

Paul

Of the 3 switches which one is responsible for routing between vlans ? Or do they all route between vlans ?

Also the device you are pingng from to test this out, what is it's IP address, what is it's default-gateway and which switch is it attached to. If the default-gateway is not on any of the switches what device is it on and how does that device connect back to the switch(es) ?

Jon

There are a large number of VLANs on these 3 switches all of which are reponsible for routing these, the majority of which are handled with BGP. This particular VLAN is across all 3 L3 Switches so would be handled at L2 (within these 3 switches). Out side of these, all 3 L3 switches advertise this subnet via BGP.

The device that I'm testing from is irrelevant as I get the same results from any device outside of the 192.168.1.0 subnet. For all intents and purposes you can consider the test device attached to a subnet that is several router hops away from the 3 L3 Switches exihibiting the strange behavior.

From 2 Routers physically attached (but still routed) to the L3 Switches and with the default gateway being another IP address on one of the L3Switches I get the same results.

ROUTER1#ping 192.168.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)
ROUTER1#ping 192.168.1.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
ROUTER1#ping 192.168.1.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.3, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)

ROUTER2#ping 192.168.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
U.U.U
Success rate is 0 percent (0/5)
ROUTER2#ping 192.168.1.2

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms
ROUTER2#ping 192.168.1.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.3, timeout is 2 seconds:
U.U.U

I've attached a VSD doc to show how this is layed out.

I believe this is some kind of bug. I can't see how L3SWITCH2 is able to allow this traffic. All L3 Switches are running the same IOS image, feature set and are the same hardware.

Jon Marshall
Hall of Fame
Hall of Fame

Paul

Apologies, can you post the vsd as a .jpg as i don't have visio software.

Does the diagram show what is connected to each switch. The reason i keep on about this is that an acl applied to a vlan interface inbound only filters traffic coming from that vlan so the packet would have to be routed onto that vlan before the acl applied. So it really does matter which switch the source device is connected to and which switch routes the packet onto vlan 101. Not saying this is why you are seeing your results but i have seen this sort of behaviour before.

Also you mentioned in your original post that vlan 101 was not assigned to any physical ports on switch 2. Presumably though it is allowed on a trunk then as the L3 SVI is up and responding to pings ?

Jon

Hi Jon.

Thanks very much for your responses. JPG attached.

Whilst your statement makes sense about the packet being routed onto the VLAN, I would say that the ACL's applied to L3SWITCH1 and 3 would block the traffic at L3 before it even gets to the L3SWITCH2 if this is is the case.

You can consider the source of the tests either the 2 attached routers or anything below coming through those routers.

paulwedde wrote:

Hi Jon.

Thanks very much for your responses. JPG attached.

Whilst your statement makes sense about the packet being routed onto the VLAN, I would say that the ACL's applied to L3SWITCH1 and 3 would block the traffic at L3 before it even gets to the L3SWITCH2 if this is is the case.

You can consider the source of the tests either the 2 attached routers or anything below coming through those routers.

Paul

You are right this is confusing.

I would have said from router1 you could -

1) ping 192.168.1.1

2) not ping 192.168.1.2 AND 192.168.1.3

reason being that the acl applied inbound to vlan 101 on switch1 does not apply but once the packet has been routed onto vlan 101 then the acl applied to switch2 and switch3 vlan 101 interfaces would apply because the packet was coming in from vlan 101.

Sorry to ask this but are you sure router1 is connected to switch1 and not switch2 ?

Jon

Router1 is connected to Switch1 and Router2 is connected to Switch3. The default gateways for each router are respectively the same.

The results from router1 and router2 are consistent though, regardless of which path you are following on to the 3 switches. If you are trying to get to 192.168.1.1 from anywhere but within my ACL range (192.168.1.1-7) I would say that the packet should be dropped?

Regardless, L3SWITCH1 is working as it should. It's the ACL on L3SWITCH2 that is "broken" (maybe..;)

Results for pinging these addresses are consistent throughout the network. The only way I can get a "drop" to occur is if I ping the 192.168.1.2 address from another VLAN on the same physical switch - L3Switch2.

Paul

The routers default-gateway(s), are these routers exchanging routing information with the L3 switches or do you have static routes.

More importantly are you running HSRP for the router's default-gateways and if so which switch is the active gateway ?

Jon

Hi Jon,

The default gateway for router1 is an HSRP address (across all 3 switches) and the currently active gateway is switch1. The default gateway for router2 is an HSRP address (across all 3 switches) and the currently active gateway is switch3. I should also mention that these 2 routers are not the 2 routers I mentioned in my original post. They do not have interfaces in the 192.168.1.0/24 range.

ROUTER1 & 2 are ABR's and are exchanging BGP updates with the 3 switches. They run OSPF on the other side of the network (they perform all BGP into OSPF resdistribution)

The route to the 192.168.1.1 network is being learnt via Switch1

Traceroutes from each router:

ROUTER1#traceroute 192.168.1.1

Type escape sequence to abort.
Tracing the route to 192.168.1.1

  1 10.0.0.1 0 msec 0 msec 0 msec
  2 192.168.1.1 [AS 12345] !A  *  !A
ROUTER1#
ROUTER1#traceroute 192.168.1.2

Type escape sequence to abort.
Tracing the route to 192.168.1.2

  1  *
    10.0.0.1 0 msec *
ROUTER1#traceroute 192.168.1.3

Type escape sequence to abort.
Tracing the route to 192.168.1.3

  1 10.0.0.1 0 msec 0 msec 0 msec
  2 192.168.1.3 [AS 12345] !A  *  !A

ROUTER2#traceroute 192.168.1.1

Type escape sequence to abort.
Tracing the route to 192.168.1.1

  1 10.0.0.1 4 msec 0 msec 4 msec
  2 192.168.1.1 [AS 12345] !A  *  !A
ROUTER2#traceroute 192.168.1.2

Type escape sequence to abort.
Tracing the route to 192.168.1.2

  1  *  *  *
  2 10.0.0.1 0 msec *  *
ROUTER2#traceroute 192.168.1.3

Type escape sequence to abort.
Tracing the route to 192.168.1.3

  1 10.0.0.1 0 msec 0 msec 0 msec
  2 192.168.1.3 [AS 12345] !A  *  !A

So as an update, I've since removed the ACL's and now I have almost the inverse problem.

As mentioned previously I have two routers in the same subnet - 192.168.1.5 and 192.168.1.6

192.168.1.1 can now ping everything but 192.168.1.6


192.168.1.2 can ping everything

192.168.1.3 can ping everything but 192.168.1.6 which is directly connected (via a transparent firewall)

192.168.1.5 is also transparentently firewalled and connected to 192.168.1.1

Outside of these devices (outside of this subnet) all 192.168.1-6 addresses are reachable.

Spanning Tree is blocking one of the ports between 192.168.1.1 and 192.168.1.5 (2 ports to the transparent firewall & one to the actual Router, Untrust/Outside Firewall port blocked). The other Spanning Tree ports blocked are all inter-switch and should not affect communication between devices.

I'm going to make some changes to the Spanning Tree configuration this evening and will post results tomorrow.

I've now changed spanning tree priorties and we have blocked broadcast traffic on the firewalls and the Spanning Tree looks a lot better (Root bridges where they should be etc...) I still have the same problem reaching 192.168.1.6. I'm going to start a new thread to try and prevent any confusion.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco