BGP and internet bandwidth balancing

[tgroth]

We have 4 full T1’s for all of our internet traffic (i.e. browsing, VPN, remote user), two with AT&T and two with Global Crossings. On our side these T1’s are terminated in a 7204 running BGP and we are receiving full routes from both ISP’s The two T1’s with AT&T are ebgp-multihop so they load balance (between each other) all traffic that comes in and out from AT&T. Our two Global Crossings T1’s are both links to two different physical locations on Global Crossings network so I don’t know of a way to do load balancing these two T1’s. For a while now most our incoming traffic has been hitting one Global Crossings T1 and most of the outgoing traffic has been hitting our other Global Crossings T1, meanwhile the AT&T T1’s are not getting utilized much at all. Both the Global Crossings T1 are getting maxed at peak times and end users (mostly VPN users) are seeing a noticeable slowdown. What I need to be able to do is load balance between all 4 T1’s. I realize that load balancing on in traffic would probably be harder because it is harder to control. But I would think load balancing out traffic should easier because our router could control what interface it is sending traffic out on. What I need to figure out is at least a short term fix come up with a plan for a long term fix. Is it as easy as adding routes and force some traffic to go out a different interface? Is there something I can enable on the 7204 that will load balance out traffic, something like CEF? Should I be looking into local preference attribute and pre pending our as paths? Any suggestions would be greatly appreciated.

[t.baranski]

As both of your ISP's are tier-1's, it seems strange that your traffic distribution is so one-sided. 50/50 is almost always an unreasonable expectation, but if you're not manually adjusting the attibutes of the routes you're receiving, outbound traffic should be at least somewhat balanced. Inbound traffic should also be "not so one-sided" if you're not adjusting the attibutes of the routes you're advertising. This is odd.

Perhaps you can improve the outbound traffic distribution this way: tell AT&T and Global Crossing that you want to receive only their customer routes instead of the entire Internet routing table. This should help even out the load, as traffic destined for an AS directly connected to AT&T will go out the AT&T T1's, and vice-versa for Global Crossing. To handle destinations that don't fall into either of these categories, you configure default routes pointing out each T1 (4 default routes for 4 T1's). Both fast switching and CEF (don't know what you're using) will then load balance per-destination by default (this is important -- per-packet would be a disaster here) out each of the 4 T1's. You should up with a fairly even outbound traffic distribution between your two providers this way. Load balancing the GC-bound traffic out the two GC T1's could be an issue since one already seems to be preferred over the other, but one step at a time...

Inbound traffic is harder. You could try path prepending to AT&T to see if that helps things, but again, it's odd that there's such a large disparity if attributes aren't currently being changed by your router.