Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

Really Slow web surfing through ZBF with IOS Content filter

Edited: attached partial output of "sh policy-map type inspect zone-pair urlfilter"   

       

Hey, all

We have a 1921 router with IOS Content filter subscribsion and it is also configured as ZBF running latest IOS v15.1. End-user keep complaining about slow web surfing. I connected to network and tested myself and found intermittent surfing experience.

For example, access to www.ibm.com or www.cnn.com hangs 7 times of 10 attempts and maybe only loads reasonablly quick in 1-2 time of the 3. This also affects the speed of download from websites.

I have the case openned with Cisco TAC and CCIE checked my configure but nothing caught his eyes...

I decide to post the issue here in case we both missed something:

           

Current configuration : 18977 bytes
!
version 15.1
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname abc_1921
!
boot-start-marker
boot system flash:/c1900-universalk9-mz.SPA.151-4.M4.bin
boot-end-marker
!
!
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication login NONE_LOGIN none
aaa authorization exec default local
!
!
!
!
!
aaa session-id common
!
clock timezone AST -4 0
clock summer-time ADT recurring 3 Sun Mar 2:00 2 Sun Nov 2:00
!
no ipv6 cef
ip source-route
ip auth-proxy max-login-attempts 5
ip admission max-login-attempts 5
ip cef
!
!
!
ip dhcp excluded-address 192.168.1.1 192.168.1.9
ip dhcp excluded-address 192.168.1.111 192.168.1.254
!
ip dhcp pool DHCPPOOL
import all
network 192.168.1.0 255.255.255.0
domain-name abc.local
dns-server 192.168.10.200 192.168.10.202
netbios-name-server 4.2.2.4
default-router 192.168.1.150
option 202 ip 192.168.1.218
lease 8
!
!
ip domain name abc.locol
ip name-server 8.8.8.8
ip name-server 4.2.2.2
ip port-map user-port-1 port tcp 5080
ip port-map user-port-2 port tcp 3389
ip inspect log drop-pkt
!
multilink bundle-name authenticated
!
parameter-map type inspect global
log dropped-packets enable

parameter-map type urlfpolicy trend cprepdenyregex0
allow-mode on
block-page message "The website you have accessed is blocked as per corporate policy"
parameter-map type urlf-glob cpaddbnwlocparapermit2
pattern www.alc.ca
pattern www.espn.com
pattern www.bestcarriers.com
pattern www.gulfpacificseafood.com
pattern www.lafermeblackriver.ca
pattern 69.156.240.29
pattern www.tyson.com
pattern www.citybrewery.com
pattern www.canadianbusinessdirectory.ca
pattern www.homedepot.ca
pattern ai.fmcsa.dot.gov
pattern www.mtq.gouv.qc.ca
pattern licenseinfo.oregon.gov
pattern www.summitfoods.com
pattern www.marine-atlantic.ca
pattern www.larway.com
pattern www.rtlmotor.ca
pattern *.abc.com
pattern *.kijiji.ca
pattern *.linkedin.com
pattern *.skype.com
pattern toronto.bluejays.mlb.com
pattern *.gstatic.com

parameter-map type urlf-glob cpaddbnwlocparadeny3
pattern www.facebook.com
pattern www.radiofreecolorado.net
pattern facebook.com
pattern worldofwarcraft.com
pattern identityunknown.net
pattern static.break.com
pattern lyris01.media.com
pattern www.saltofreight.com
pattern reality-check.com
pattern reality-check.ca


parameter-map type ooo global
tcp reassembly timeout 5
tcp reassembly queue length 128
tcp reassembly memory limit 8192

parameter-map type trend-global global-param-map
cache-size maximum-memory 5000
crypto pki token default removal timeout 0
!
crypto pki trustpoint Equifax_Secure_CA
revocation-check none
!
crypto pki trustpoint NetworkSolutions_CA
revocation-check none
!
crypto pki trustpoint trps1_server
revocation-check none
!
crypto pki trustpoint TP-self-signed-3538579429
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-3538579429
revocation-check none
rsakeypair TP-self-signed-3538579429
!
!
!! CERTIFICATE OMITED !!
!
redundancy
!
!
!
!
ip ssh version 2
!
class-map type inspect match-any INCOMING_VPN_TRAFFIC_MAP
match access-group name REMOTE_SITE_SUBNET
class-map type inspect match-all PPTP_GRE_INSPECT_MAP
match access-group name ALLOW_GRE
class-map type inspect match-all INSPECT_SKINNY_MAP
match protocol skinny
class-map type inspect match-all INVALID_SOURCE_MAP
match access-group name INVALID_SOURCE
class-map type inspect match-all ALLOW_PING_MAP
match protocol icmp
class-map type urlfilter match-any cpaddbnwlocclasspermit2
match  server-domain urlf-glob cpaddbnwlocparapermit2
class-map type urlfilter match-any cpaddbnwlocclassdeny3
match  server-domain urlf-glob cpaddbnwlocparadeny3
class-map type urlfilter trend match-any cpcatdenyclass2
class-map type inspect match-all cpinspectclass1
match protocol http
class-map type inspect match-any CUSTOMIZED_PROTOCOL_216
match protocol citriximaclient
match protocol ica
match protocol http
match protocol https
class-map type inspect match-any INSPECT_SIP_MAP
match protocol sip
class-map type urlfilter trend match-any cptrendclasscatdeny1
match  url category Abortion
match  url category Activist-Groups
match  url category Adult-Mature-Content
match  url category Chat-Instant-Messaging
match  url category Cult-Occult
match  url category Cultural-Institutions
match  url category Gambling
match  url category Games
match  url category Illegal-Drugs
match  url category Illegal-Questionable
match  url category Internet-Radio-and-TV
match  url category Joke-Programs
match  url category Military
match  url category Nudity
match  url category Pay-to-surf
match  url category Peer-to-Peer
match  url category Personals-Dating
match  url category Pornography
match  url category Proxy-Avoidance
match  url category Sex-education
match  url category Social-Networking
match  url category Spam
match  url category Tasteless
match  url category Violence-hate-racism
class-map type inspect match-any INSPECT_PROTOCOLS_MAP
match protocol pptp
match protocol dns
match protocol ftp
match protocol https
match protocol imap
match protocol pop3
match protocol netshow
match protocol shell
match protocol realmedia
match protocol rtsp
match protocol smtp
match protocol sql-net
match protocol streamworks
match protocol tftp
match protocol vdolive
match protocol tcp
match protocol udp
match protocol icmp
class-map type urlfilter trend match-any cptrendclassrepdeny1
match  url reputation ADWARE
match  url reputation DIALER
match  url reputation DISEASE-VECTOR
match  url reputation HACKING
match  url reputation PASSWORD-CRACKING-APPLICATIONS
match  url reputation PHISHING
match  url reputation POTENTIALLY-MALICIOUS-SOFTWARE
match  url reputation SPYWARE
match  url reputation VIRUS-ACCOMPLICE
class-map type inspect match-all CUSTOMIZED_NAT_MAP_1
match access-group name CUSTOMIZED_NAT_1
match protocol user-port-1
class-map type inspect match-all CUSTOMIZED_NAT_MAP_2
match access-group name CUSTOMIZED_NAT_2
match protocol user-port-2
class-map type inspect match-any INSPECT_H323_MAP
match protocol h323
match protocol h323-nxg
match protocol h323-annexe
class-map type inspect match-all INSPECT_H225_MAP
match protocol h225ras
class-map type inspect match-all CUSTOMIZED_216_MAP
match class-map CUSTOMIZED_PROTOCOL_216
match access-group name CUSTOMIZED_NAT_216
!
!
!
policy-map type inspect OUT-IN-INSPECT-POLICY
class type inspect INCOMING_VPN_TRAFFIC_MAP
  inspect
class type inspect PPTP_GRE_INSPECT_MAP
  pass
class type inspect CUSTOMIZED_NAT_MAP_1
  inspect
class type inspect CUSTOMIZED_NAT_MAP_2
  inspect
class type inspect CUSTOMIZED_216_MAP
  inspect
class class-default
  drop
policy-map type inspect urlfilter cppolicymap-1
description Default abc Policy Filter
parameter type urlfpolicy trend cprepdenyregex0
class type urlfilter cpaddbnwlocclasspermit2
  allow
class type urlfilter cpaddbnwlocclassdeny3
  reset
  log
class type urlfilter trend cptrendclasscatdeny1
  reset
  log
class type urlfilter trend cptrendclassrepdeny1
  reset
  log
policy-map type inspect IN-OUT-INSPECT-POLICY
class type inspect cpinspectclass1
  inspect
  service-policy urlfilter cppolicymap-1
class type inspect INSPECT_PROTOCOLS_MAP
  inspect
class type inspect INVALID_SOURCE_MAP
  inspect
class type inspect INSPECT_SIP_MAP
  inspect
class type inspect ALLOW_PING_MAP
  inspect
class type inspect INSPECT_SKINNY_MAP
  inspect
class type inspect INSPECT_H225_MAP
  inspect
class type inspect INSPECT_H323_MAP
  inspect
class class-default
  drop
!
zone security inside
description INTERNAL_NETWORK
zone security outside
description PUBLIC_NETWORK
zone-pair security INSIDE_2_OUTSIDE source inside destination outside
service-policy type inspect IN-OUT-INSPECT-POLICY
zone-pair security OUTSIDE_2_INSIDE source outside destination inside
service-policy type inspect OUT-IN-INSPECT-POLICY
!
!
crypto isakmp policy 10
encr 3des
authentication pre-share
group 2

crypto isakmp key password address 11.22.3.1
!
crypto ipsec security-association lifetime seconds 28800
!
crypto ipsec transform-set TunnelToCold esp-3des
!
crypto map TunnelsToRemoteSites 10 ipsec-isakmp
set peer 11.22.3.1
set transform-set TunnelToCold
match address TUNNEL_TRAFFIC2Cold
!
!
!
!
interface Embedded-Service-Engine0/0
no ip address
shutdown
!
interface GigabitEthernet0/0
description OUTSIDE_INTERFACE
ip address 1.1.1.186 255.255.255.248
ip nat outside
ip virtual-reassembly in
zone-member security outside
duplex full
speed 1000
crypto map TunnelsToRemoteSites
crypto ipsec df-bit clear
!
interface GigabitEthernet0/1
description INSIDE_INTERFACE
ip address 192.168.1.150 255.255.255.0
ip nat inside
ip virtual-reassembly in
zone-member security inside
duplex full
speed 1000
!
ip forward-protocol nd
!
ip http server
ip http access-class 10
ip http authentication local
ip http secure-server
!
ip nat inside source static tcp 192.168.1.217 5080 interface GigabitEthernet0/0 5080
ip nat inside source route-map NAT_MAP interface GigabitEthernet0/0 overload
ip nat inside source static tcp 192.168.1.216 80 1.1.1.187 80 extendable
ip nat inside source static tcp 192.168.1.216 443 1.1.1.187 443 extendable
ip nat inside source static tcp 192.168.1.216 1494 1.1.1.187 1494 extendable
ip nat inside source static tcp 192.168.1.216 2598 1.1.1.187 2598 extendable
ip nat inside source static tcp 192.168.1.213 3389 1.1.1.187 3390 extendable
ip nat inside source static tcp 192.168.1.216 5080 1.1.1.187 5080 extendable
ip route 0.0.0.0 0.0.0.0 1.1.1.185
!
ip access-list standard LINE_ACCESS_CONTROL
permit 192.168.1.0 0.0.0.255
!
ip access-list extended ALLOW_ESP_AH
permit esp any any
permit ahp any any
ip access-list extended ALLOW_GRE
permit gre any any
ip access-list extended CUSTOMIZED_NAT_1
permit ip any host 192.168.1.217
permit ip any host 192.168.1.216
ip access-list extended CUSTOMIZED_NAT_2
permit ip any host 192.168.1.216
permit ip any host 192.168.1.212
permit ip any host 192.168.1.213
ip access-list extended CUSTOMIZED_NAT_216
permit ip any host 192.168.1.216
ip access-list extended INVALID_SOURCE
permit ip host 255.255.255.255 any
permit ip 127.0.0.0 0.255.255.255 any
ip access-list extended NAT_RULES
deny   ip 192.168.1.0 0.0.0.255 192.168.2.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.3.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.4.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.5.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.6.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.7.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.8.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.9.0 0.0.0.255
deny   ip 192.168.1.0 0.0.0.255 192.168.10.0 0.0.0.255
permit ip 192.168.1.0 0.0.0.255 any
ip access-list extended REMOTE_SITE_SUBNET
permit ip 192.168.2.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.3.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.4.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.5.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.6.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.7.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.8.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.9.0 0.0.0.255 192.168.1.0 0.0.0.255
permit ip 192.168.10.0 0.0.0.255 192.168.1.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2ABM
permit ip 192.168.1.0 0.0.0.255 192.168.10.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2Bridgewater
permit ip 192.168.1.0 0.0.0.255 192.168.8.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2ColdbrookDispatch
permit ip 192.168.1.0 0.0.0.255 192.168.2.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2ColdbrookETL
permit ip 192.168.1.0 0.0.0.255 192.168.7.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2ColdbrookTrailershop
permit ip 192.168.1.0 0.0.0.255 192.168.3.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2Moncton
permit ip 192.168.1.0 0.0.0.255 192.168.6.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2MountPearl
permit ip 192.168.1.0 0.0.0.255 192.168.4.0 0.0.0.255
ip access-list extended TUNNEL_TRAFFIC2Ontoria
permit ip 192.168.1.0 0.0.0.255 192.168.5.0 0.0.0.255
ip access-list extended WEB_TRAFFIC
permit tcp 192.168.1.0 0.0.0.255 any eq www
access-list 10 permit 192.168.1.0 0.0.0.255
!
!
!
!
!
route-map NAT_MAP permit 10
match ip address NAT_RULES
!
!
snmp-server community 1publicl RO
!
!
!
control-plane
!
!
!
line con 0
logging synchronous
login authentication NONE_LOGIN
line aux 0
line 2
no activation-character
no exec
transport preferred none
transport input all
transport output pad telnet rlogin lapb-ta mop udptn v120 ssh
stopbits 1
line vty 0 4
access-class LINE_ACCESS_CONTROL in
exec-timeout 30 0
logging synchronous
transport input all
!
scheduler allocate 20000 1000
ntp server 0.ca.pool.ntp.org prefer
ntp server 1.ca.pool.ntp.org
end

12 REPLIES

Really Slow web surfing through ZBF with IOS Content filter

Hello Shuai,

As the TAC engineer might told you the URL filter policy is going to add more inspection to any http packet as it will need to look for specific patterns.

Now as a I way to get into the bottom of this I would take out the configuration related to the URL policy and test it to make sure this is related to the URl filtering stuff.

policy-map type inspect IN-OUT-INSPECT-POLICY

class type inspect cpinspectclass1

  inspect

  no service-policy urlfilter cppolicymap-1

Regards,

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC
New Member

Really Slow web surfing through ZBF with IOS Content filter

Thanks, Jcarvaja

We actually have tested Layer 4 inspection and worked perfectly. As soon as the content filter is enabled, the slow web surfing appears.

We might need to downgrade to c1900-universalk9-mz.SPA.151-4.M3.bin to see if issue got fixed. I hate this to be another bug...

Really Slow web surfing through ZBF with IOS Content filter

Hello Shuai,

Please keep us posted on this,

Regards,

Julio

Julio Carvajal
Senior Network Security and Core Specialist
CCIE #42930, 2xCCNP, JNCIP-SEC
New Member

Really Slow web surfing through ZBF with IOS Content filter

Downgraded to c1900-universalk9-mz.SPA.151-4.M3.bin and same thing...

I just wonder why after a period of time, the router/layer 7 inspect engine/content filter engine seems to lock up the web traffic and no one inside network can surf...?? Only reboot can solve the issue.

Again this is NOT a windows server!!!!

New Member

Really Slow web surfing through ZBF with IOS Content filter

Hi,

I know this is for a different platform but have a look at this link:

https://supportforums.cisco.com/thread/2089462

Read through it to get some idea of the similarity, but in particular note the last entry almost a year after the original post.

I too am having trouble with http inspection, if I do layers 3 & 4 inspection there is no issue whatsoever, but as soon as I enable layer 7 inspection then I have intermittent browsing issues.

The easy solution here is to leave it at layers 3 & 4, which doesn't give you the flixibility to do cool things like blocking websites, IM, regex expression matching etc...  but in my opinion I just don't think these routers can handle it.

It appears to be a hit and miss affair, and going on the last post from the above link, you might be better off in having the unit replaced under warranty.

The alternative is wasting a lot of time and effort and impacting your users to get something up and running that in the end is so flaky that you have no confidence in the solution and you are then in a situation where ALL future issues users are facing MIGHT be because of this layer 7 inspection bug/hardware issue etc?

I would recommend you use the router as a frontline firewall with inbound/outbound acl's (no inspection), and then invest a few $ in getting an ASA dedicated firewall (but that's just me )

New Member

Really Slow web surfing through ZBF with IOS Content filter

Well, the reason I guess my predecessor put these routers (9 of them !!) were for the content filtering. If we go the route to replace the gears, then it will be 9 of boxed not just one...

We might want to invest on alternative software filtering, like the openDNS. But I still hope Cisco tac can help a little bit before we put in order...

New Member

Really Slow web surfing through ZBF with IOS Content filter

They are great devices for routing, vpn's, and low level inspection etc but as soon as you start doing layer 7 it has the cpu and memory running at high capacity all of the time.

From the features list of the 1921:

"Integrated threat control using Cisco IOS Firewall, Cisco IOS Zone-Based Firewall, Cisco IOS IPS, and Cisco IOS                 Content Filtering"

So I can see where your predecessor was coming from...

You can always try and downgrade to the 12 series IOS like 124-20.T3 (as suggested in the other post) or do a series of tests with different IOS versions and see if you get lucky with a particular version, but of course you will lose certain features and commands but if this is an issue at 9 sites then downgrading 1 and testing won't hurt to try.

New Member

Really Slow web surfing through ZBF with IOS Content filter

Frankly, I think down from 15 to 12 will just increase the issue. I saw posts that OoO is not really properly designed in 12 and people has to rush to 15 to get proper communication on ZBF.

New Member

Re: Really Slow web surfing through ZBF with IOS Content filter

I'm not saying that Cisco would do this, but their push from 12 to 15 came mainly for introudcing a new licensing model, sure there are new features etc, but it also introduced a number of new bugs... the later 12 series started seeing these issues that weren't appearing on earlier 12 series releases...

Conspiracy theories aside - try it on one of the routers and see if it makes a difference - you don't need ooo if the ooo errors disappear and your browsing issues are resolved while still maintaining the layer 7 inspection.

Update: Seems you don't have the option there to downgrade to the 12 series anyway - that sucks.

I just downgraded an 877w to version 124-20.T3 and this resolved the issue straight away. I find it funny also that there are NO ooo packets being logged AT ALL, on this version.

I will do a quick test with the same inspection rules and and regex expressions you have on your 1921 and see how it handles it on the 877w probably die a quick death

New Member

Re: Really Slow web surfing through ZBF with IOS Content filter

I did the url filtering config just using local instead of trend and it works as it should... no intermittent browsing issues etc, I have some load on the router but nothing to cry home about as only testing with one user.

I really think this ooo bug came in with the move to the 15 series, it's unfortunate you can't downgrade to the earlier 12 series for your routers because it means you will be stuck without a resolution until cisco can resolve the bug, or alternatively you could try swapping out 1 of the routers with a replacement unit to see if it is hardware related as the last post in the above link suggested his was? 

New Member

Really Slow web surfing through ZBF with IOS Content filter

I would think the static URL filtering would work as well, especially we have cusomer using S520 with URL filtering and we never hear complain from.

We actually have replaced router for the site but still not have better perfomance...

New Member

Really Slow web surfing through ZBF with IOS Content filter

The issue is still there...downgrading IOS did not help.

We do see alot of Out of Order packets got dropped by router but the OOO parameter is also configured properly...

Any suggestion is appreciated.

1247
Views
0
Helpful
12
Replies
CreatePlease login to create content