GRE and Traceroute

[JohnTylerPearce]

I have a Cisco 4900m switch, which has a GRE tunnel interface configured on it.

We have two networks 172.16.0.0/21 and 172.16.8.0/16. Both these networks use the same Internet pipe.

The internet pipe includes to network ranged 71.20.111.0/26 and 71.20.111.64/26. The DG for those two

networks are 71.20.111.1 and 71.20.111.65. The default route on the Cisco 4900m switch is

the 71.20.111.65 address.

VLAN768 = 172.16.8.0/21 VLAN199 = 172.16.0.0/21

interface Vlan768

ip address 172.16.8.9 255.255.248.0

ip policy route-map ZSCALER_WEB

Anything in the 172.16.8.0/21 network uses this interface as it's default gateway.

The route-map is configured as follows

route-map ZSCALER_WEB permit 10
match ip address ZSCALER_WEB
set ip next-hop 172.17.160.146

The ZSCALER_WEB ACL is as follows

ip access-list extended ZSCALER_WEB

permit tcp any any eq www

permit tcp any any eq 443

So, anything matching the ZSCALER_WEB ACL is PBRd to a next hop of 172.17.160.146.

Tunnel0 Configuration

There is a directly connected interface, which is Tunnel0 to the 172.17.0.0/30 network.

So I would assume, once it PBRs the specified traffic it arps it out to 172.17.160.146, since it

is the next hop.

interface Tunnel0

description Zscaler GRE tunnel--primary

ip address 172.17.160.145 255.255.255.252

ip mtu 1476

ip tcp adjust-mss 1436

keepalive 10 3

tunnel source Loopback0

tunnel destination 216.52.207.65

!

interface Loopback0

ip address 71.20.111.117 255.255.255.192

On a user on VLAN199 (172.16.0.0/21) the next hop is as follows

1. 71.20.111.65 (This is the correct hop according to our routing)

2. 10.66.18.161 (This addres is inside the SP's network)

On a user on VLAN768 (172.16.8.0/21) the next hop is as follows

1. 172.16.8.9 (This is the correct hip according to our routing)

2. 10.66.18.161 (confusing because the only way to get to this network is through 71.20.111.65)

What I dont understand is, why VLAN768 doesn't have 71.20.111.65 as it's next hop after 172.16.8.9.

The only way to get to 10.66.18.161 is through 71.20.111.65?

Sorry for the long post guys.....

[cadet alain]

Hi,

what is the output from  sh ip route 172.17.160.146

Regards.

Alain

[JohnTylerPearce]

Routing entry for 172.17.160.144/30

  Known via "connected", distance 0, metric 0 (connected, via interface)

  Routing Descriptor Blocks:

  * directly connected, via Tunnel0

      Route metric is 0, traffic share count is 1

I would think that 172.17.160.144 is the other side of the virtual tunnel interface.

[Konstantin Dunaev]

according to the configuration - it's the directly connected network.

[Konstantin Dunaev]

may be I'm mistaken, but you said that you're usrin PBR for traffic from the ACL:

ip access-list extended ZSCALER_WEB

permit tcp any any eq www

permit tcp any any eq 443

but traceroute doesn't match your ACL and so doesn't follow your PBR.

Or have I missed something?

[JohnTylerPearce]

That's true, so all HTTP/HTTPS would be send to next hop (172.17.160.146), and all other traffic

depending on if it didn't match any specific routes would go out the default route of 172.16.8.1. This address

is on our wireless ASA, which then has a default route to 71.20.111.65.  What I still find interesting is if

I do a traceroute to 8.8.8.8 on a VLAN768(172.16.8.0/21) network, the first hop is 172.16.8.9 (should be), but

then it goes to the 10.66.18.161 IP (which is a BGP next hop on our internet router to our SP). But it doesn't

go from 172.16.8.9 to 71.20.111.65 and then to 10.66.18.161 like ther VLAN199(172.16.0.0/21) network does.

I'm still having trouble understanding how that's even possible.

[Konstantin Dunaev]

It can be possible only if your ASA would do additionly routing decision.

please show the trace from both VLANs. and additionly do two  the traceroute to 8.8.8.8 sourcing from  your both VLAN interfaces

[JohnTylerPearce]

VLAN768(172.16.8.0/21)

1. 172.16.8.9

2. 10.66.18.161

3. 10.75.65.102

4. 10.75.65.93

5. 10.75.65.34

6. 72.158.108.194

7.(etc etc)

VLAN199(172.16.0.0/21)

1. 71.20.111.65

2. 10.66.18.161

3. 10.75.65.102

4. 74.254.101.194

5. 65.14.210.169

6. 12.81.28.56

7.(etc etc)

[JohnTylerPearce]

According to the route-map, if traffic does not match HTTP/HTTPS and the routing table, it will match the

default route which is 172.16.8.1 (This is on our wireless ASA)

This is the route table on our wireless ASA.

route outside 0.0.0.0 0.0.0.0 71.20.111.65 1

route wireless A-71.20.111.117 255.255.255.255 172.16.8.9 1

[Konstantin Dunaev]

have you tried the traceroute directyl from your 4900m switch?

[Konstantin Dunaev]

the device with IP address 10.75.65.102 does defently additionl routing decision, you want to look whta it exactly does.

if you use a Linux server you can try to use "ping -R" command - it shows to whole way towards and backwards (but limited to 9 Hops only)

[JohnTylerPearce]

The device with IP 10.75.65.102 is on the SP's network. Otherwise, I would love to check out what it does. I'll

check out the traceroute from 4900. I've done it before, I just forgot the results.

[JohnTylerPearce]

You do appear to be right Konstantin.

I did a traceroute to 8.8.8.8 on the 4900 and get the follow results.

1. 71.20.111.65

2. 10.66.18.161

3. 10.75.65.102

4. 74.254.101.194

[Konstantin Dunaev]

you should try to do the traceroute but with different source IPs - from vlan768 and 199

[JohnTylerPearce]

After anything on the 172.16.0.0/21 network leaves 71.20.111.65, it appears 10.75.65.102, is changing the

routing behavior to go to 74.254.101.194.

But anything on 172.16.8.09/21 network leaves 10.66.18.161 and hits 10.75.65.102 it changes its routing behavior

to go to more 10.x.x.x IPs.

I still dont understand how the 172.16.8.0/21 network goes from 172.16.8.9 to 10.66.18.161 though. I swear

it must have something to do with tunnel0 some how.