Solved: One Site not reachable

townofnewmarket · ‎10-01-2010

This is so strange, I can't figure it out, maybe somene out there can help.

Clients are largely Windows XP SP3, but with some Linux and Macintosh thrown in.

Switches are 3560s POE, with one 3560G. Firewall is a PIX515-E. We have some static IP addresses, so we have a small SMC router from Comcast in the data closet too.

So here's what's happening:

I try to go to www.trythissite.com, and it times out. Pinging it resolves it to an IP, but the pings are not answered. Virtually every other website out there is fine, we get right to them. Hmm...using the DNS handed me by windows. Let me use a few well known public DNS addys (8.8.8.8, 4.2.2.1, etc.) Same results, so doesn't seem to be a DNS issue.

Doing a traceroute to the host name only gets me one hop, the default gateway (Cisco 3560G), everything else times out.

Take a PC, plug it into the SMC router, and we get to www.trythissite.com everytime. So it sure seems to be something on my end. There is nothing in the PIX configuration referencing the IP or its subnet, so the PIX would not appear to be dumping the requests (though I don't know enough of the PIX and how to confirm that).

The PC goes to a 3560G, the PIX does NAT for us, and out we go to the real world (well we plug into the SMC router on the way, but that doesn't seem to be a block). Both a Linux client and a Mac client on the same switch have the same failure (so it's not web filtering, as those clients don't web filter).

Appreciate ANY help! Wouldn't you know, it's a fairly important website, lol!

Federico Coto Fajardo · ‎10-01-2010

We can check if the problem is with the PIX or not.

i.e

access-list capin permit ip host x.x.x.x host y.y.y.y

access-list capin permit ip host y.y.y.y host x.x.x.x

capture capin access-list capin interface inside

access-list capout permit ip host z.z.z.z host y.y.y.y

access-list capout permit ip host y.y.y.y host z.z.z.z

capture capout access-list capout interface outside

The first capture is applied to the inside interface:

x.x.x.x --> Real IP of the computer or host trying to reach the website

y.y.y.y --> Real IP of the website that you're trying to reach

The second capture is applied to the outside interface:

z.z.z.z --> NATed IP for the computer

Also, just curious what is the result of a packet tracer?

packet-tracer input inside x.x.x.x 1025 y.y.y.y 80 detail

Federico.

View solution in original post

Federico Coto Fajardo · ‎10-01-2010

Hi,

If you connect directly to the SMC router it works.

The problem is if you pass through the PIX correct?

You said that when you do a lookup for the domain name you get an IP (so DNS is not the problem).

Can you open a browser and get to the website typing the IP instead than the name? http://1.1.1.1

Is there any filtering configuration on the PIX that might be causing this problem?

i.e

The PIX can filter acitveX, java on destination traffic to port 80 on the Internet.

Federico.

townofnewmarket · ‎10-01-2010

Federico,

yes you understand it correctly. If we go through the PIX, we don't get there. Plug directly into the SMC, (or connect from home) and we get there just fine. The PIX was filtering java and ActiveX, so I removed that. Still no change in behavior.

Opening a web browser and pointing to the IP fails as well, (using Firefox) the error is "The connection to the server was reset while the page was loading. The network link was interrupted while negotiating a connection. Please try again." With IE,it fails as well, and suggests DNS problems, site down externally, network issues, etc.

Very strange!

Federico Coto Fajardo · ‎10-01-2010

We can check if the problem is with the PIX or not.

i.e

access-list capin permit ip host x.x.x.x host y.y.y.y

access-list capin permit ip host y.y.y.y host x.x.x.x

capture capin access-list capin interface inside

access-list capout permit ip host z.z.z.z host y.y.y.y

access-list capout permit ip host y.y.y.y host z.z.z.z

capture capout access-list capout interface outside

The first capture is applied to the inside interface:

x.x.x.x --> Real IP of the computer or host trying to reach the website

y.y.y.y --> Real IP of the website that you're trying to reach

The second capture is applied to the outside interface:

z.z.z.z --> NATed IP for the computer

Also, just curious what is the result of a packet tracer?

packet-tracer input inside x.x.x.x 1025 y.y.y.y 80 detail

Federico.

townofnewmarket · ‎10-01-2010

I don't have packet-trace available as a command on my PIX. Must be an old release.

Once I add the lines you gave me to the config, here's the output:

show capture capin

0 packet captured

0 packet shown

show capture capout

16 packets captured

01:53:39.995996 100.101.102.103.20467 > 67.68.69.70.53: udp 39

01:53:43.172461 100.101.102.103.20465 > 67.68.69.70.53: udp 39

01:53:51.171728 100.101.102.103.20483 > 67.68.69.70.53: udp 39

01:53:51.171774 100.101.102.103.20485 > 67.68.69.70.53: udp 39

01:55:28.163458 100.101.102.103.20731 > 67.68.69.70.53: udp 35

01:55:28.163565 100.101.102.103.20733 > 67.68.69.70.53: udp 35

01:55:32.330991 100.101.102.103.20743 > 67.68.69.70.53: udp 35

01:55:36.162802 100.101.102.103.20741 > 67.68.69.70.53: udp 35

01:55:40.585861 100.101.102.103.20766 > 67.68.69.70.53: udp 39

01:55:44.161994 100.101.102.103.20768 > 67.68.69.70.53: udp 39

01:55:48.181982 100.101.102.103.20797 > 67.68.69.70.53: udp 39

01:55:52.161307 100.101.102.103.20799 > 67.68.69.70.53: udp 39

01:57:49.149238 100.101.102.103.21118 > 67.68.69.70.53: udp 39

01:57:53.150703 100.101.102.103.21116 > 67.68.69.70.53: udp 39

01:58:01.150001 100.101.102.103.21124 > 67.68.69.70.53: udp 39

01:58:01.150047 100.101.102.103.21126 > 67.68.69.70.53: udp 39

16 packets shown

101.101.102.103 is the outside IP of the PIX.

67.68.69.70 is the IP of the host that has the website.

Does this tell us anything? These commands are new to me! Thanks a bunch!

townofnewmarket · ‎10-01-2010

Some recent traceroute information:

Windows Client (internal going through PIX):

tracert [ip addr of webhost]

Goes about 14 hops, gets all the way to this host, but never the final host:

14 47 ms 50 ms 46 ms lw-dc3-dist7-po5.rtr.liquidweb.com [69.167.128.129]

15     *        *        *     Request timed out.
16     *        *        *     Request timed out.
17     *        *        *     Request timed out.
18     *        *        *     Request timed out.
19     *        *        *     Request timed out.
20     *        *        *     Request timed out.
21     *        *        *     Request timed out.
22     *        *        *     Request timed out.
23     *        *        *     Request timed out.

Linux Box attached to SMC router:

traceroute [ip addr of webhost]

13 lw-dc3-dist7-po5.rtr.liquidweb.com (69.167.128.129) 46.081 ms 46.923 ms 43.747 ms
14 host.dotnotme.com (59.137.150.188) 50.641 ms 47.774 ms 49.275 ms

So again, if we're outside the corp net,we make it all the way. Inside, we almost get there, we die on the very last step.

Does that help any??

jeanaguemon · ‎10-04-2010

Do you have any access-list applied to the vlan on the switch before the PIX firewall. Please check - thanks.

Federico Coto Fajardo · ‎10-01-2010

You're right the packet-tracer is not available.

The idea is to send traffic and capture the traffic.

https://1.1.1.1/capture/capin/pcap

https://1.1.1.1/capture/capout/pcap

The idea is to get the captures and open them with wireshark (wireshark.org)

Change 1.1.1.1 for the IP of the ASA.

You must enable HTTP.

Federico.

townofnewmarket · ‎10-01-2010

I did

http server enable

but all http(s) requests to the PIX fail with a timeout. Do I need to set anything else for the http to show me what it's getting for packets? Anyway we can dump it to a tftp server?

Federico Coto Fajardo · ‎10-01-2010

Besides enabling http:

http 0 0 inside

To allow every IP from the inside.

If still does not work, need to generate the RSA keys:

name NAME

domain-name DOMAIN NAME

crypto key generate rsa

Federico.

townofnewmarket · ‎10-01-2010

The http 0 0 inside went fine.

Generating RSA keys was a little more difficult,

name NAME doesn't work for me.

domain-name DOMAIN worked

crypto key generate rsa didn't work, so I used

ca generate rsa key 512

and got this:
% You already have RSA keys defined for xxx.yyy.com.
% Please remove the keys by issuing ca zeroize rsa command
% before generating RSA keys again.

Still can't connect. I can dump to the console, maybe I could cut and paste it into a packet capture program

I sure do appreciate all the help you've given so far Federico! Thank you!!!!!

Federico Coto Fajardo · ‎10-01-2010

I don't want to be troubleshooting something that is not the original problem but if we can get the captures via HTTP that will help to the original issue.

So,

When you open a browser and type https://1.1.1.1 (changing 1.1.1.1 for the inside IP of the PIX what do you get)?

I am assuming that you can PING that address and that you're on the internal network.

In order to get the HTTP service working on the PIX:

- http server enable

- http 0 0 inside

- hostname --> this should be the hostname of the PIX

- domain-name a domain for the PIX (could be cisco.com for example)

- username test password test123 privi 15

- aaa authentication http console LOCAL

The crypto keys are already in place.

Try accesing the PIX via web and if it does not work let me know the IP of the PIX that you're trying to connect to and the IP that you're coming from.

Federico.

townofnewmarket · ‎10-04-2010

Federico, thanks so much for sticking with me on this one!

So now I have the capture ins and outs. I have opened them with wireshark, but they don't mean a whole lot to me!

Can I mail you the captures or something?

Thanks.