Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

ASA URL filtering without a Websense or N2H2/Smartfilter server

 

 

Introduction

One of the ASA features is url filtering. It can be used to block or allow users from going to certain urls/websites. This article aims to educate the user on how to use this feature. After reading it carefully someone should be able to take full advantage of url filtering and use it for his needs.

 

In this article we will either block or allow domains in urls and words in the uri. Of course the ASA can match on other things too. They can be found in the ASA configuration guides. From now and onwards we will allow or block the cisco.com domain. In other words, any user browsing to any page that is behind cisco.com will be subject to url filtering. Such pages would be www.cisco.com/index.html or cisco.com/exampledir/page.html. Also, we will allow or block "/test/" in the uri. In other words, any page path that contains "/test/" will be url filtered. Examples are www.examplesite.com/exampledir/test/page.html or www.anyurl.com/test/examplepage.jsp or www.anything.com/onedir/seconddir/test

 

The mechanism used to apply url filtering is Modular Policy Framework (MPF). We will create regular expressions (regex) that will be matched in class-maps of type http. These class-maps will be used in policy-maps to define the drop action. Then the policy-maps will be applied with an http inspection in another policy-map that will be applied to an interface. In that way the http inspection action will be applied to the traffic that hits an interface.

 

NOTE: Though, we need to highlight that for Enterprise URL Filtering, customers should be steered toward using WebSense or N2H2 integration with the ASA. Such web filtering engines can provide much more robust filtering based on classes of sites. URL filtering directly on the ASA using regex, should be used only sparsely when broad classifications can be applied, with limited regex patterns. The ASA will not scale being used in an enterprise with large regex matches and large volumes of HTTP traffic.

 

 

Block specific urls

Lets assume that we want to block some specific websites. For example we want to block specific anything under cisco.com and uris that contain "/test/". We will create the regexes and match them in a class-map. Note that if ANY regex is matched then the class-map will actually be met. Then in the policy-map block-url-policy whatever meets the class-map (cisco.com OR uri containing "/test/") is reset. The rest are allowed (not cisco.com and not uri containing "/test/"). The policy-map block-url-policy is used for http inspection in another policy-map (global_policy) and applied with a service-policy.

 

 

 
regex blockex1 "/test/"
regex blockex2 "cisco\.com"

class-map type inspect http match-any block-url-class
match request uri regex blockex1
match request header host regex blockex2

policy-map type inspect http block-url-policy
parameters
class block-url-class
  drop-connection log
policy-map global_policy
class inspection_default
  inspect http block-url-policy

service-policy global_policy global

 

 

 

Allow every url for specific hosts, block specific urls for the rest

Finally lets say that we have a few hosts (administrators, i.e. ip address 192.168.1.2) that need access to any website and the rest of the users need to be blocked from specific websites (Note that you need to understand the example above to be able to follow the process of this example). We will use approximately the same configuration as in the example above but this time we will need an extra access-list, a class-map and a policy-map. This access-list (user-acl) will match all the users with the exception of the ones that need unrestricted access. It will be matched in a new class-map (block-user-class) which in turn will be used in a separate policy-map (block-user-url-policy). That policy-map will do the http inspection for the allowed websites policy-map block-url-policy (that part is the same as above). Thus the block-user-url-policy will be applied to an interface with a service-policy. What this policy-map is actually doing is to match on all the users except the unrestricted ones (class block-user-class) and block them from going to the specified websites (inspect http block-url-policy. The rest of the users (not matching/denied in the access-list) will be able to go anywhere.

 

 

 
regex blockex1 "/test/"
regex blockex2 "cisco\.com"

access-list user-acl extended deny tcp host 192.168.1.2 any eq www
access-list user-acl extended permit tcp any any eq www


class-map type inspect http match-any block-url-class
match request uri regex blockex1
match request header host regex blockex2
class-map block-user-class
match access-list user-acl

policy-map type inspect http block-url-policy
parameters
class block-url-class
  drop-connection
policy-map block-user-url-policy
class block-user-class
  inspect http block-url-policy

service-policy block-user-url-policy interface inside

 

 

 

Allow only cisco.com

Lets assume that we want to allow only cisco.com. We create the regex and match them (match statement) in a class-map. Note that if the match statements is matched the class-map is met. The match statement is met the regex is NOT matched by the url. Then in the policy-map allow-url-policy the connection is reset. The rest is allowed (cisco.com). In other words, if the page you browse is not cisco.com it will be reset. The policy map is used for http inspection in another policy-map (global_policy) and applied with a service-policy.

 

 

regex allowex2 "cisco\.com"

class-map type inspect http match-all allow-url-class
match not request header host regex allowex2

policy-map type inspect http allow-url-policy
parameters
class allow-url-class
  drop-connection log
policy-map global_policy
class inspection_default
  inspect http allow-url-policy

service-policy global_policy global

 

 

 

Allow specific urls

Lets assume that we want to allow only specific websites. For example we want to allow only cisco.com pages and only uris that contain "/test/". We create the regexes and match them in a class-map. Note that if both match statements are matched the class-map is met. Both the match statements are matched if and only if NEITHER regex is matched by the url browsed by the user. Then in the policy-map allow-url-policy the connection is reset. The rest is allowed (cisco.com or uri containing "/test/"). In other words, if the page you browse is neither cisco.com nor contains "/test/" in the url, it will be reset. The policy map is used for http inspection in another policy-map (global_policy) and applied with a service-policy.

 

 

regex allowex1 "/test/"
regex allowex2 "cisco\.com"

class-map type inspect http match-all allow-url-class
match not request uri regex allowex1
match not request header host regex allowex2

policy-map type inspect http allow-url-policy
parameters
class allow-url-class
  drop-connection log
policy-map global_policy
class inspection_default
  inspect http allow-url-policy

service-policy global_policy global

 

 

 

Allow every url for specific hosts, allow only specific urls for the rest

Now lets say that we have a few hosts (administrators, i.e. ip address 192.168.1.2) that need access to any website and the rest of the users need to be able to go only to specific websites (Note that you need to understand the example above to be able to follow the process of this example). We will use approximately the same configuration as in the example above but this time we will need an extra access-list, a class-map and a policy-map. This access-list (user-acl) will match all the users with the exception of the ones that need unrestricted access. It will be matched in a new class-map (allow-user-class) which in turn will be used in a separate policy-map (allow-user-url-policy). That policy-map will do the http inspection for the allowed websites policy-map allow-url-policy (that part is the same as above). Thus the allow-user-url-policy will be applied to an interface with a service-policy. What this policy-map is actually doing is to match on all the users except the unrestricted ones (class allow-user-class) and allow them only to go to the specified websites (inspect http allow-url-policy. The rest of the users (not matching/denied in the access-list) will be able to go anywhere.

 

 

 
regex allowex1 "/test/"
regex allowex2 "cisco\.com"

access-list user-acl extended deny tcp host 192.168.1.2 any eq www
access-list user-acl extended permit tcp any any eq www

class-map type inspect http match-all allow-url-class
match not request uri regex allowex1
match not request header host regex allowex2
class-map allow-user-class
match access-list user-acl

policy-map type inspect http allow-url-policy
parameters
class allow-url-class
  drop-connection
policy-map allow-user-url-policy
class allow-user-class
  inspect http allow-url-policy

service-policy allow-user-url-policy interface inside

Same domain allow/deny

The requirements are as follows:

1. deny facebook.com

2. allow developer.facebook.com which gets redirected automatically to developers.facebook.com

3. and all other domains like yahoo.com and google.com

 

regex block-fb ".*facebook\.com"
regex allow-fb "developer[s]*\.facebook\.com"

!
class-map type regex match-any block-fb
   match regex block-fb
class-map type regex match-any allow-regex-class
   match regex allow-fb 
!
policy-map type inspect http http-inspect-pol
   parameters
match not request header host regex class allow-regex-class
match request header host regex class block-fb
    reset log

class-map http-class
   match port tcp eq www

policy-map http-traffic
   class http-class
    inspect http http-inspect-pol
!
service-policy http-traffic interface inside

 

Alternatively, one could also tie the http policy-map to the default inspection under the global_policy.

 

Bear in mind, every time you add remove or change regex, you need to 
remove the service-policy applied to the interface and add it again.

http://www.cisco.com/en/US/docs/security/asa/asa82/configuration/guide/inspect_basic.html#wp1514315

If you need to change a match command for HTTP inspection after configuring the inspection, you must
remove the attached service policy by using the no service policy command and then reconfigure the
service policy. Changing the class map by removing a match command causes HTTP inspection to block
all HTTP traffic until you remove and reconfigure the attached service policy so that all the match
commands are reprocessed.
Comments
Community Member

Hello Panagiotis,

Thanks for posting this article, it is very interesting. I wonder if you had a chance to try this feature with HTTPS (SSL/TLS encrypted) traffic. All examples I found are for clear text HTTP traffic.

Another question – does this feature work if the client is configured to use a web proxy? This is probably standard config in many companies, the users have to pass through a Microsoft IIS proxy server in order to access the Internet.

I will test the configs in the lab. You obviously spent time testing this feature, you may have the answers.

Thank you,

Cristian

Community Member

Hello Cristian,

HTTPS filtering is not supported on ASA. ASA cannot do deep packet  inspection or inspection based on regular expression for HTTPS traffic, because  in HTTPS, content of packet is encrypted (SSL).

You can also use URL filtering to direct specific traffic to an external  filtering server, such an Secure Computing SmartFilter (formerly N2H2) or  Websense filtering server. Long URL, HTTPS, and FTP filtering can now be enabled  using both Websense and Secure Computing SmartFilter for URL filtering.  Filtering servers can block traffic to specific sites or types of sites, as  specified by the security policy.

For more information on URL filters, refer the following URLs:

http://www.cisco.com/en/US/docs/security/asa/asa81/config/guide/filter.html#wp1042606

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_configuration_example09186a008088517b.shtml

http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a0080940e04.shtml

Thanks

Sathesh

Community Member

It is a god scenario, but, there is a bug for this solution which is attachments for any hotmail account only can be accessed either to be opened nor to be saved to the desktop.

If you please, if you have any idea how to solve this issue

Community Member

hmmm, i'm a bit confused on the url filtering. Would the communication from an internal user be on the resolved IP address of the url instread of the url ? ( assuming i have an internal dns server) So when user1 is going to www.cnn.com, the firewall sees the url ? or the IP address of cnn.com (which was internalyl resolved) ?

Community Member

ahmedchohan,

I was wondering the same thing.  I did a simple packet captue on my machine..  Note the "Host: www.microsoft.com".  I am by no means an expert, but it would make sense if this is what we're matching with "request header host"

GET / HTTP/1.1

Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/x-silverlight, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/csm-desktop, */*

Accept-Language: en-us

User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)

Accept-Encoding: gzip, deflate

Host: www.microsoft.com: Keep-Alive

Connection

Community Member

Hey Ahmed,

I couldn't figure out how to stop it from letting you save the attachment.  However, I did figure out how to block access to all attachments from hotmail (even the little picture preview thing).  Use at your own risk:

regex GET_ATTACH ".*([Gg][Ee][Tt][Aa][Tt][Tt][Aa][Cc][Hh][Mm][Ee][Nn][Tt])\.([Aa][Ss][Pp][Xx]).*"
regex SCAN_ATTACH ".*([Ss][Cc][Aa][Nn][Aa][Tt][Tt][Aa][Cc][Hh][Mm][Ee][Nn][Tt])\.([Aa][Ss][Pp][Xx]).*"
regex HOTMAIL_URL ".*([Mm][Aa][Ii][Ll])\.([Ll][Ii][Vv][Ee])\.([Cc][Oo][Mm])"

class-map type inspect http match-all HOTMAIL_SMACKDOWN
match request header host regex HOTMAIL_URL
match request uri regex SCAN_ATTACH

class-map type inspect http match-all HOTMAIL_SMACKDOWN_THE_SEQUEL
match request header host regex HOTMAIL_URL
match request uri regex GET_ATTACH

policy-map type inspect http HOTMAIL_SMACKDOWN
parameters
class HOTMAIL_SMACKDOWN_THE_SEQUEL
  drop-connection log
class HOTMAIL_SMACKDOWN
  drop-connection log

policy-map global_policy
class inspection_default
inspect http HOTMAIL_SMACKDOWN

Community Member

First, I must warn everyone, I'm a Storage CCIE with little background in R&S and Security.  If it doesn't deal with FCIP or iSCSI with static routes, I'm at a loss but I'm reading and trying to understand all of this.

Q1.  As for the document at the start of this post, I'm having a hard time understanding the statement:

     access-list user-acl extended deny tcp host 192.168.1.2 any eq www.  Doesn't this statement prevent the administrator at 192.168.1.2 from going anywhere via http?

Q2.  I'm doing volunteer work at my children's local school (pre-K to 8th grade) and trying to clean up the mess the previous administrator installed.  I've acquired 4 - Cisco 3548XL (I know they are EoL/EoS) and I want to secure them behind a secure router/firewall with URL filtering.  I've been looking at a Cisco PIX 501 (its free, that's why I'm looking at it), Cisco SR520, Cisco ASA5505, and a Cisco 851.  The staff has asked me to deny ALL student PC's access to anything on the Internet except ~10 URL's but allow the teachers and staff access to anywhere they want to go (with certain exceptions of course).  The cost of a Websense or N2H2 server and licensing is beyond what they can afford and whatever secure router/firewall is deemed a correct fit, I'll end up purchasing it myself and donating it to the school.  I keep bouncing between the SR520 and the ASA5505.  Does anyone have any advice for me as I'm looking at trying to move on this fairly quickly?  I've tapped a couple of local Cisco resources with no response yet.

Thanks.

Gary

Community Member

Hello Gary,

The URL filtering may be overkill for your requirements. For 10 URLs, it may be simple to limit the access based on destination IP addresses. For example, if you want to allow access to www.wikipedia.org use a rule like “access-list acl-inside permit tcp student_ip_address host 208.80.152.2 eq 80”. This solution will not allow you to limit the access to certain URLs per destination (allow access to wikipedia.org/school, but deny access to wikipedia.org/politics). Enterprise solutions like N2H2 allow restrictions for a large number of sites, with URLs that change (you pay for updates with lists of restricted sites).

How do you separate the teachers from students? The most flexible solution would be to create two VLANs, let’s say network 10.1.1.0/24 for teachers and network 10.1.2.0/24 for students. In your PIX, define three interfaces: outside facing the Internet, inside-teacher for network 10.1.1.1/24, and inside-student network 10.1.2.1/24.

The config would be something like this:

access-list acl-inside-student remark à Allow access to DNS

access-list acl-inside-student permit udp 10.1.2.0 255.255.255.0 host dns_server eq 53

access-list acl-inside-student remark à Allow access to wikipedia

access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 208.80.152.2 eq 80

access-list acl-inside-student remark à Allow access to cnn.com

access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 157.166.226.25 eq 80

access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 157.166.226.26 eq 80

access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 157.166.255.18 eq 80

[implied deny at the end]

access-list acl-inside-teacher remark à Allow access to DNS

access-list acl-inside- teacher permit udp 10.1.1.0 255.255.255.0 host dns_server eq 53

access-list acl-inside- teacher remark à no restrictions for web browsing

access-list acl-inside- teacher permit tcp 10.1.1.0 255.255.255.0 any eq 80

[implied deny at the end]

The PIX can be configured as DHCP servers for both internal networks.

The PIX may not support three interfaces at the same time, depending on what license you have. If this is the case, a solution would be to use only one network for teachers and students and assign static IP addresses to the teacher’s workstations.

access-list acl-inside remark à DNS

access-list acl-inside permit udp 10.1.1.0 255.255.255.0 host dns_server eq 53

access-list acl-inside remark à teacher 1

access-list acl-inside permit tcp host 10.1.1.254 any eq 80

access-list acl-inside remark à teacher 2

access-list acl-inside permit tcp host 10.1.1.253 any eq 80

access-list acl-inside remark à Students access to wikipedia

access-list acl-inside permit tcp 10.1.2.0 255.255.255.0 host 208.80.152.2 eq 80

access-list acl-inside remark à Students allow access to cnn.com

access-list acl-inside permit tcp 10.1.1.0 255.255.255.0 host 157.166.226.25 eq 80

access-list acl-inside permit tcp 10.1.1.0 255.255.255.0 host 157.166.226.26 eq 80

access-list acl-inside permit tcp 10.1.1.0 255.255.255.0 host 157.166.255.18 eq 80

[implied deny at the end]

A  tech savy student may go around this restrictions by configuring a static IP address on his workstation (hopefully the pre-K kids won't do it).

Regards,

Cristian

Community Member

Cristian,

Thanks for the reply.  It was very helpful!!  Yes, I'm planning and creating 2 VLAN's.  One for students and one for teachers and staff making my job easier.    I'm assuming if I put the students on their own VLAN that even if they set their IP address statically, they would still be restricted.  The only way I can see them getting around the restriction is to swap network cables with the teachers PC or use the teachers PC.

Thanks again!

Gary

Community Member

Sadly, this can't be used to block visiting websites by IP address (and thus ensureing every site is resolved by DNS for security reasons), tried...

regex blockex3 "\d*\.\d*\.\d*\.\d*"

regex blockex4 "(0|1|2|3|4|5|6|7|8|9)*\.(0|1|2|3|4|5|6|7|8|9)*\.(0|1|2|3|4|5|6|7|8|9)*\.(0|1|2|3|4|5|6|7|8|9)*\."

regex blockex6 "^http://\d*\.\d*\.\d*\.\d*(/|$)"

regex blockex7 "http://\d*\.\d*\.\d*\.\d*"

regex blockex8 "\d+\.\d+\.\d+\.\d+"

...either it had no effect, or it ended up blocking everything :-(

Community Member

Hi,

I'm very newbie - and vey dumm - in cisco stuff...

Can anyone explain me or give me any link to the appropriate

documentation about how to implement the point "allow every url for specific host, block specific urls for the rest" using ASDM 6.x tool. I've found a useful document:

http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a0080940e04.shtml#prereq

about url filtering

but no hints about how to grant evey url for some ip and I believe that it's a very common scenario.

Thank you very much for your help

filippo

Community Member

Hi,

If I want to block this url:    www.123.nl/index.html                    www\.123\.nl\/index\.html does not work!

How do I block this then, what will be the expression?

thx,

Marc

Community Member

Hi Gary,

Probably this topic is history but if you have similar issues in the future, you can have a look at SquidGuard - I know it is Linux and it is both not ok to speak about it on a CISCO forum and it requires some strong knowledge but if you combine this with your CISCO PIX cappability to allow egress www connections only from the squid machine - you may achieve a strong web filtering.

Community Member

I know this is may be outdated discussion, but I could not understand how

access-list user-acl extended deny tcp host 192.168.1.2 any eq www

Does allow user access to internet when it is a deny not permit

 

 

Cisco Employee

The ACL 'user-acl' is used in a class-map. By using a 'deny' we prevent that traffic from matching that class-map and as a result prevent that traffic (sourced from 192.168.1.2) from being subjected to the http inspection policy defined.

114173
Views
18
Helpful
15
Comments