BGP Peering w/ Three Service Providers

[eknell]

Currently, I BGP load-share across an OC-12 Link and a Gig-E circuit. I also use weights to help influence my outbound traffic across these two links.

My questions are.....First, I am fixing to hook up another Gig-E circuit from another service provider, in which I will have to peer with. Are there any issues with peering with 3 providers??

Second...I need to load share my Gig-E connections, while only utilizing the OC-12 for inbound and outbound traffic if both Gig-E circuits were to fail. These were my thoughts on doing this....I would just set lower weights on all the routes coming from both providers off of the Gig-E circuits via route maps, to manipulate traffic flow outbound through both the Gig-E circuits first. I would also "poison" my routes via as-path prepend towards my OC-12 peer as to prevent inbound traffic, unless both Gig-E circuits were down.

Does any of this sound right? My thinking may be way off? Any help would be greatly appreciated! Thanks!!!

[Giuseppe Larosa]

Hello Ethan,

1) Are there any issues with peering with 3 providers??

If you plan to receive full Internet BGP table from each of them you need to check if your router / multilayer switch can handle them:

memory on the route processor must be enough to handle three internet routing tables.

the CEF process needs to have enough memory to install all the resulting best paths.

What platform are you using ?

If using a C6500/C7600 it is wise to have a Sup720 3BXL or better like Sup720 3CXL.

2) to make the two ISPs using GE links preferred:

a) use an higher weight for neighbors on the GE links use the same value example 1000 on both (higher weight is preferred)

b) to have both routes installed in the routing table and in CEF tables you need

router bgp yourASN

maximum-paths 2

neigh ISP1 weight 1000

neigh ISP3 weight 1000

neigh ISP2 weight 500

! where ISP2 is the one using the OC-12 link

load-balancing will be fair but not perfect different BGP internet tables are different point of views of the internet.

The prefix length is the first criteria used: most specific networks are preferred.

c) you can use AS prepending towards ISP2 but results can vary and some return traffic will come via ISP2: you can decide how to send out your outbound traffic but you can only suggest your preferred return path.

Hope to help

Giuseppe

[eknell]

Giuseppe,

Yes, I am running redundant WS-SUP720-3BXL's in my 7609, because I need the routing table capacity, considering I am getting the full routing table from each....and soon to be third, so I should be okay...I hope?!

As far as the traffic engineering is concerned for my 3 links, it looks like we are on the same page, and I am glad to hear some of my thought process's were in the right direction.

I do currenlty weight the upper and lower half of the IPV4 address space towards different providers to help compensate with some of BGP's load sharing short comings.

The one thing I am a little confused on and is not currently configured on my router is the "maximum-paths" command? I show all the routes in my routing table as well as my cef table from both providers? What is happening considering I am not using this currently? My router shows a total of External: 261464 routes...

[Giuseppe Larosa · Accepted solution]

Hello Ethan,

the maximum-paths 2 allow to install two BGP advertisements in the routing table if they share the first "N" parameters in the BGP path selection.

Actually, there could be a big impact on the CEF table because all routes with two next-hops will have a bigger entry to accomodate the two paths so I apologize it should be tested in a lab not in production.

If these memory issues don't happen you could get a more fair load-balancing.

Without using this command for all the routes that are "in common" between ISP1 and ISP3 only one path is chosen.

If I've understood correctly you are doing load-balance by using different weights for different prefixes in the IPv4 space with a route-map for each neighbor.

This can provide good results for the high number of IP prefixes and without the risk to increase CEF entries sizes with double paths.

Currently we have three upstream providers on a GSR and the two 6500 with sup720 3BXL receive the result of choices made by the two 12000 border routers so they are a little more protected.

On another thread someone was suggesting sup720 3CXL for a similar scenario (but I don't remember if the providers were 5 or 3).

Hope to help

Giuseppe

[eknell]

Well, we "load share" between the connections because we are peering with 2 different AS's from a single node...and yes,you are correct, I am setting the weights with route-maps.

I didn't think it was possible to actually "load balance" BGP, unless my router was using diverse connections to the same AS/peer, in which case I could load balance across those connections to that particular peer.

Yeah....I can't see much of a difference in the 3BXL versus the 3CXL, besides the two 10Gb ports and routing another 50Mpps...

Anyhow, I really appreciate your help on my BGP peering configuration concerns. It looks like I will be able to peer with 3 providers from my 7609 just fine. Hopefully I don't run into any processor issues. Thanks again!!