Guys, thanks for the explanations. Let me go a step further to explain my network. We currently have 4 Checkpoint firewalls that sit at our edge. We plan to use a feature on them that will load balance connections 50/50 between our ISPs. Given this setup, would the router that is blindly delivered a packet with a given destination from the firewalls still be able to use the route tables to route the packet optimally?

Thanks.

How does the firewall know where to send the traffic for the sharing? It sounds like the firewalls are only doing LAN Load sharing. Taking the traffic it receives and passes it to the router interface it deems.

Your post is a little misleading.

Firewalls at the edge and load balance to the ISP almost tells me that the topology is ISP - Firewall - Router

I assumed ISP - Router - Firewall

Unless the Checkpoint can see beyond the router, I do not understand how the firewall will load share internet traffic other than by splitting the load between interferaces on the router.

Keep this in mind. Routing traffic is always done at the router level in your topology. The router will receive the packet, from firewall or switch or other router, check the routing table for the exit path and send the traffic. So no matter what the firewall is doing the routing is still determined on the router.

Take a look at a simplified version of how my infrastructure looks in the jpeg I've attached, and keep in mind that my knowledge of how Checkpoint firewalls work is limited. :-)

That said, we will supply the firewall with the ip address of the inside interfaces of routers R1 and R2 and it will blindly forward traffic to the routers via this ip. And you're right, our topology is isp - router - firewall.

But I believe you answered my question...even though the firewalls will deliver the traffic to either one of the bgp-speaking routers, by accepting more than default they will still be able to correctly route traffic from the R1 router over the ibgp link to the R2 router to follow the best path? Is that it?

So following that logic (if it's correct)....if we accept default only, then whatever traffic R1 gets it will only know to send it out via ISP1 correct? and same for R2 right?

/rls

You are on the right path.

I was originally thinking both peers terminated into one router. Since they are seperate you should be fine with default routes. If you accept full routes it will help, but keep in mind that you added another hop to add the routed traffic. So it may not benefit you if you go with full routes or not. Default routes given your topology should give you want you need.

Thanks for the information. Sorry if I wasn't as clear as I should have. Too add to this, not only do the connections terminate on separate routers, but they do so in geographically diverse locations. Without a complete redo of the infrastructure, and trying out letting the firewalls run bgp I think this was our best option.

I just wish there were some good way to still be able to have the ability to point traffic where we want (if need be) even with the firewalls handling the load-balancing.

/rls

ok...this changes things

your routers in different locations

Router A

-"San Jose"

-T-1

-BGP

-Default Routes

-iBGP w/ Router B

Router B

-"New York"

-T-1

-BGP

-Default Routes

-iBGP w/ Router B

Router A and B connected, via ???

so with this you are not multi-homed. You have two locations with BGP sessions. Multi-Homed would require your ASN too.

Each location has 2 firewalls which are dual attached, not sure how if they are geographically diverse.

The only thing I can gather is point-to-point lease line between the two sites, which will help you with traffic shaping but will cause a little delay.

Remember you can shape your traffic how ever you want. You can route wherever you want. Your limitations are bandwidth, which will include latency and such.

With being geographically diverse I would run static routes and do away with BGP, then run EIGRP between the two sites, with a floating static route to bounce traffic out in case one link is down you can go out the other. But I also assume that the IP scheme is different too.

Sorry, I guess that diagram was too watered-down. See the new one. Firewalls are not redundant....yet, and the routers are connected via our own DWDM connections via 15454s and our transponder cards. We do need to run bgp as we'll be advertising our space from both providers. One block will be a /24 we carve out of our own ISP-independent /16 we own, and the other will be the /24 we currently are addressed out of from ISP1.

Does this clear things up a bit hopefully?

/rls

Is your topology already nailed down? Or can you still "chop-and-change" it? What I'm thinking is maybe get rid of the internal router and use L3 switch on the inside of the firewalls, and set up eBGP between that L3 switch and the Internet routers. You can configure BGP to install multiple equal cost paths into the routing table, hence achieving outbound load balancing without having to create convoluted static routes that might lead to routing blackhole or loops.

BTW, you said that the firewalls are not redundant. Did you mean they're not clustered locally either? I think you should (if not must) cluster 'em.

As far as inbound load balancing goes, you might want to think about using GSLB device (e.g. Cisco GSS series appliance, F5, etc.). It'll allow you to dynamically load balance inbound application traffic.

Michael, our topology is still up for changes based on solid technical merit which is why I'm doing the homework to see what others have to say. I'm in no way sold on having the firewalls handling any part of the load-balancing. I'm sort of in a time crunch as the circuit we ordered from our second provider will be setup sometime mid to late July and then management will be expecting a solution. I'd just rather not implement a "goofy logic" solution first and have to make significant changes later to improve it.

The firewalls are not redundant as they all serve different types of traffic. Clustering is part of the long-range goal, but is probably not budgeted for this project. I do agree that is the way to go. Are you saying that it might be a neccessity with the L3 switch in the mix? We're pushing traffic to the appropriate firewall via pbr? Does this break when you insert the L3 switch running eBGP? BTW...why eBGP between the L3 and routers as opposed to iBGP?

Thanks.

/rls

Hi... man... To be honest, your set up reminds me very much like the DMZ network that set up for a large bank about 12 months ago. It ended up being very messy indeed! **sigh** I honestly don't know how to answer your with brief answers without being confusing, but I'll try....

First off, I'd start with design objectives. Mine would've been something like the following:

1. Avoid PBR as much as possible

2. Firewalls shall not participate in routing. They shall only have basic static and/or default routes

3. Segregate routing between Intra-Enterprise and Internet. I.e. Create 2 layers: 1 layer between your enterprise edge (i.e. your Internet routers) and the ISPs, the other between your enterprise edge and your internal networks.

4. Avoid requiring hosts to have internal routing table. I.e. Hosts should only need to point to a single default gateway.

5. Minimize the number of physical devices required as much as possible

6. Minimize static routes as much as possible

Now, to answer your question regarding eBGP vs iBGP... Reason for that suggestion was because I assumed that you had public BGP AS number from IANA, due to multi-homing to different ISPs. Although there are techniques to "host" multiple BGP AS on the same router, I strongly advice against it, unless there's no other way of doing it. And based on design objective 3 above, you'd use public AS on the Internet routers, and private AS internally.

The reason for my suggestion of using L3 switch on the inside of the firewall is to achieve design objectives 2, 4 and 5 above.

Will inserting L3 switch break PBR? Depends on how and where you configure the PBR. What I was suggesting was to replace the DMZ L2 switch and Internal router with a single (or a pair of) L3 switch. If you planned to config the PBR on the internal router, then you'd move the config to the L3 switch (should you go down that path).

One thing that I missed was.... your diagram indicated BGP connection between the two DWDM switches. Did you mean to say you're running BGP on the DWDM switches too? Or you were depicting iBGP session between the Internet routers via the DWDM? I just thought of another alternative: BGP on Internet routers, BGP on switch3, switch4, and switch2 (get rid of internal router). Then eBGP b/w the two. You get my drift? :-)

You said that the firewalls serve different types of traffic. What do you mean by that? Can you elaborate more?

Michaelchoo, could you give me a little more insight into your design objective to avoid PBR on the inside internet router? What alternative would you suggest using? We've just recently moved to PBR where we were using strictly statics previously, along with WCCP to push traffic to the caching device.

We will be using a public AS which I'm in the process of securing now, as well as breaking up our own /16 to use with the new ISP.

Regarding the DWDM switches, they will not be running BGP, like you mentioned, I am only depicting the iBGP session between the internet routers via DWDM...

/rls