Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

Bloking Web Crawls

Setup:

We host over 1200 websites.  There are redundant Cisco 6509 (with FWSM) used to create the DMZs.  Port 80 and 443 are allowed only from the outside (Internet) to the Blue Coat Reverse Proxies (DMZ1).  Then the port is changed and the Blue Coat (DMZ1) and is forwarded to the the IIS Web Server (DMZ2).  There is no direct access to the Web Servers or their content.  We also block all outbound traffic from all DMZs.

Scenario:

The IIS Admin acidently turned on folder browsing on the Web Server with live sites.  This allowed Google to crawl the directory and cache the contents publicly.

Question:

Is there a way I can still all the http and https request but block Web Crawls?

Everyone's tags (1)
1007
Views
0
Helpful
0
Replies
CreatePlease to create content