Guys, can someone give me a clear description as to which works best for a multi-homed environment. Guess I don't understand what accepting directly connected routes buys you over just plain default.
Thanks in advance.
If you're in a multihomed environment you get decide what routes you accept from your neighbors, which will better aid you to route traffic optimumly. Maybe you want to send certain traffic out router A connected to ISP A, but other traffic out router B via ISP B. Having your ISPs send you routes more specific than default gives you that ability while simultaneously providing redundancy.
with having a routing table built this will determine the best path to the destination. If you only accept default routes from the peers then you cannot use the "best" path.
-Connected to ISP A
-Connected to ISP B
Traffic may be 10 hops on Router A and only 3 Hops on Router B.
With BGP metrics you may not end up using the most optimal line.
If you accept customer only routes then any customer connected to that ISP will be preferred over the other ISP. Full Routing tables will provide you with the best option for making routing decisions on which ISP to use.
It really is an option you must make based on your requirements. I prefer full routing table from two different providers. If I use the same provider default routes might be ok, unless you are getting geographically diverse paths, then you might want to consider full or even partial routes.
Guys, thanks for the explanations. Let me go a step further to explain my network. We currently have 4 Checkpoint firewalls that sit at our edge. We plan to use a feature on them that will load balance connections 50/50 between our ISPs. Given this setup, would the router that is blindly delivered a packet with a given destination from the firewalls still be able to use the route tables to route the packet optimally?
How does the firewall know where to send the traffic for the sharing? It sounds like the firewalls are only doing LAN Load sharing. Taking the traffic it receives and passes it to the router interface it deems.
Your post is a little misleading.
Firewalls at the edge and load balance to the ISP almost tells me that the topology is ISP - Firewall - Router
I assumed ISP - Router - Firewall
Unless the Checkpoint can see beyond the router, I do not understand how the firewall will load share internet traffic other than by splitting the load between interferaces on the router.
Keep this in mind. Routing traffic is always done at the router level in your topology. The router will receive the packet, from firewall or switch or other router, check the routing table for the exit path and send the traffic. So no matter what the firewall is doing the routing is still determined on the router.
Take a look at a simplified version of how my infrastructure looks in the jpeg I've attached, and keep in mind that my knowledge of how Checkpoint firewalls work is limited. :-)
That said, we will supply the firewall with the ip address of the inside interfaces of routers R1 and R2 and it will blindly forward traffic to the routers via this ip. And you're right, our topology is isp - router - firewall.
But I believe you answered my question...even though the firewalls will deliver the traffic to either one of the bgp-speaking routers, by accepting more than default they will still be able to correctly route traffic from the R1 router over the ibgp link to the R2 router to follow the best path? Is that it?
So following that logic (if it's correct)....if we accept default only, then whatever traffic R1 gets it will only know to send it out via ISP1 correct? and same for R2 right?
You are on the right path.
I was originally thinking both peers terminated into one router. Since they are seperate you should be fine with default routes. If you accept full routes it will help, but keep in mind that you added another hop to add the routed traffic. So it may not benefit you if you go with full routes or not. Default routes given your topology should give you want you need.
Thanks for the information. Sorry if I wasn't as clear as I should have. Too add to this, not only do the connections terminate on separate routers, but they do so in geographically diverse locations. Without a complete redo of the infrastructure, and trying out letting the firewalls run bgp I think this was our best option.
I just wish there were some good way to still be able to have the ability to point traffic where we want (if need be) even with the firewalls handling the load-balancing.
ok...this changes things
your routers in different locations
-iBGP w/ Router B
-iBGP w/ Router B
Router A and B connected, via ???
so with this you are not multi-homed. You have two locations with BGP sessions. Multi-Homed would require your ASN too.
Each location has 2 firewalls which are dual attached, not sure how if they are geographically diverse.
The only thing I can gather is point-to-point lease line between the two sites, which will help you with traffic shaping but will cause a little delay.
Remember you can shape your traffic how ever you want. You can route wherever you want. Your limitations are bandwidth, which will include latency and such.
With being geographically diverse I would run static routes and do away with BGP, then run EIGRP between the two sites, with a floating static route to bounce traffic out in case one link is down you can go out the other. But I also assume that the IP scheme is different too.
Sorry, I guess that diagram was too watered-down. See the new one. Firewalls are not redundant....yet, and the routers are connected via our own DWDM connections via 15454s and our transponder cards. We do need to run bgp as we'll be advertising our space from both providers. One block will be a /24 we carve out of our own ISP-independent /16 we own, and the other will be the /24 we currently are addressed out of from ISP1.
Does this clear things up a bit hopefully?
Is your topology already nailed down? Or can you still "chop-and-change" it? What I'm thinking is maybe get rid of the internal router and use L3 switch on the inside of the firewalls, and set up eBGP between that L3 switch and the Internet routers. You can configure BGP to install multiple equal cost paths into the routing table, hence achieving outbound load balancing without having to create convoluted static routes that might lead to routing blackhole or loops.
BTW, you said that the firewalls are not redundant. Did you mean they're not clustered locally either? I think you should (if not must) cluster 'em.
As far as inbound load balancing goes, you might want to think about using GSLB device (e.g. Cisco GSS series appliance, F5, etc.). It'll allow you to dynamically load balance inbound application traffic.
Michael, our topology is still up for changes based on solid technical merit which is why I'm doing the homework to see what others have to say. I'm in no way sold on having the firewalls handling any part of the load-balancing. I'm sort of in a time crunch as the circuit we ordered from our second provider will be setup sometime mid to late July and then management will be expecting a solution. I'd just rather not implement a "goofy logic" solution first and have to make significant changes later to improve it.
The firewalls are not redundant as they all serve different types of traffic. Clustering is part of the long-range goal, but is probably not budgeted for this project. I do agree that is the way to go. Are you saying that it might be a neccessity with the L3 switch in the mix? We're pushing traffic to the appropriate firewall via pbr? Does this break when you insert the L3 switch running eBGP? BTW...why eBGP between the L3 and routers as opposed to iBGP?
Hi... man... To be honest, your set up reminds me very much like the DMZ network that set up for a large bank about 12 months ago. It ended up being very messy indeed! **sigh** I honestly don't know how to answer your with brief answers without being confusing, but I'll try....
First off, I'd start with design objectives. Mine would've been something like the following:
1. Avoid PBR as much as possible
2. Firewalls shall not participate in routing. They shall only have basic static and/or default routes
3. Segregate routing between Intra-Enterprise and Internet. I.e. Create 2 layers: 1 layer between your enterprise edge (i.e. your Internet routers) and the ISPs, the other between your enterprise edge and your internal networks.
4. Avoid requiring hosts to have internal routing table. I.e. Hosts should only need to point to a single default gateway.
5. Minimize the number of physical devices required as much as possible
6. Minimize static routes as much as possible
Now, to answer your question regarding eBGP vs iBGP... Reason for that suggestion was because I assumed that you had public BGP AS number from IANA, due to multi-homing to different ISPs. Although there are techniques to "host" multiple BGP AS on the same router, I strongly advice against it, unless there's no other way of doing it. And based on design objective 3 above, you'd use public AS on the Internet routers, and private AS internally.
The reason for my suggestion of using L3 switch on the inside of the firewall is to achieve design objectives 2, 4 and 5 above.
Will inserting L3 switch break PBR? Depends on how and where you configure the PBR. What I was suggesting was to replace the DMZ L2 switch and Internal router with a single (or a pair of) L3 switch. If you planned to config the PBR on the internal router, then you'd move the config to the L3 switch (should you go down that path).
One thing that I missed was.... your diagram indicated BGP connection between the two DWDM switches. Did you mean to say you're running BGP on the DWDM switches too? Or you were depicting iBGP session between the Internet routers via the DWDM? I just thought of another alternative: BGP on Internet routers, BGP on switch3, switch4, and switch2 (get rid of internal router). Then eBGP b/w the two. You get my drift? :-)
You said that the firewalls serve different types of traffic. What do you mean by that? Can you elaborate more?
Michaelchoo, could you give me a little more insight into your design objective to avoid PBR on the inside internet router? What alternative would you suggest using? We've just recently moved to PBR where we were using strictly statics previously, along with WCCP to push traffic to the caching device.
We will be using a public AS which I'm in the process of securing now, as well as breaking up our own /16 to use with the new ISP.
Regarding the DWDM switches, they will not be running BGP, like you mentioned, I am only depicting the iBGP session between the internet routers via DWDM...
Just plain default from both (all) upstreams will mean that all of your outbound traffic will go to only one neighbor as there is nothing to indicate a preferred destination path for any specific route. Inbound traffic to you should load-balance, however.
Taking connected routes means that destinations on a specific upstream will go to that upstream, but routes to any third-party destination will go out a default. This takes more memory and overhead in your routers to handle the additional routes, but gives you some load balancing and ensures that traffic to destinations on any of your directly-connected ASes is optimally routed.
Taking full routes means that you can now make decisions on the best path to third party destinations as well. This takes MUCH more memory and overhead in your routers.
You can take full routes plus default and filter on input to get somewhat the best of both worlds.
If your goal is a failover solution (fat pipe to a primary provider, skinny pipe to a backup) then just a default is easiest and you can set any of several BGP knobs to prefer the primary. If the primary peer goes down, the remaining default is the backup and you're good.
If your goal is to balance traffic and select the best route to multiple destinations, then the more the merrier as long as you have the horsepower to handle it in terms of memory and CPU cycles.
Very good information hennigan...just what I was looking for! The goal that's been set for us is load-sharing in whatever form that may take. Both of our circuits will be 100meg and we'd like to take advantage of them both all the time, as opposed to a backup scenario. That introduces a more complicated config, but I can understand a management's view towards the circuit lying idle.
That said, based on the last diagram that I posted that if I accepted anything more than default that I would end up with traffic being pushed to a given router via the firewall load-balancing scheme, then the router determining that the best route is actually out the other path and sending it out the other router...thus increasing my ibgp link traffic. I guess that's not so bad as I would want optimal traffic routing either way but just one of my initial concerns. I do like the L3 switch solution posted above and would appreciate hearing other opinions on this and how it would be set up.
You're correct, the firewalls and hosts not speaking BGP will have a default pointing to one router, which will route via IGP to the preferred eBGP egress router for that destination.
From your diagram, I'd set up HSRP/VRRP between int-router-2 and int-router-3 with int-router-2 as the primary with pre-empt. Point your firewall defaults to the virtual of the HSRP pair. This will minimize traffic over the DWDM to only that entering/leaving via ISP2. Should int-router-2 fail, you'll still be up via DWDM to int-router-3.
L3 switching could also be used but unless the L3 switches have the full BGP tables of the border routers (usually not practical due to memory constraints) you'll still be hairpinning some traffic through one of the borders. From the information provided, in my opinion L3 switching in this scenario doesn't offer any advantages and may just complicate things.