We have 3 internet connections as follows:
1 T1 from AT&T
1 T1 from 360 Networks
1 DS3 from XO Communications
I am using BGP with these 3 providers but the problem I have struggling with (for years) is that if I advertise my networks with the T1 providers over BGP a bunch of inbound traffic ends up routing inbound over the T1's. Even though the DS3 is fully operational I still get certain traffic going through the T1's, particuarly the AT&T one since the internet seems to think they are a better path even though the bandwidth is 1/28th as much.
I have tried prepending my advertisements to AT&T and 360 Networks (as many as 10 of my AS#'s prepended) but I still get traffic coming from the T1 links which really sucks when you are trying to download a file and instead of it coming inbound over the quick DS3 it comes in over the T1s at a snails pace.
The workaround I have now is that I have setup a route map for my T1 connections that only advertises my networks to those AS's if a major internet route is in my routing table from XO's AS on my DS3 so normally I'm only advertising my networks to the DS3 provider and not the T1 providers unless the DS3 drops then I start advertising out the T1's.
This works ok, but does have a couple caveauts:
1. Conversion when the DS3 drops is slow as when the DS3 drops I have no routes advertised on the internet anymore and the internet has to wait for routes to propagate through the T1 circuits.
2. If XO Communications is having internal routing issues but doesn't drop my DS3 circuit entirely and keeps sending me internet routes my route-map doesn't take over and my T1's stay idle even though XO is dropping traffic on their network causing a full or partial internet outage. The only way for me to fix this is to either manually remove my route-map for the T1 circuits or shutdown the DS3 so the T1's can take over.
Any thoughts on a better way to do this?
