Okay BGP gurus, here's one for you:
Currently, we have 4 T-1's going into our 7206VXR Router. We just turned 2 of the 4 up this past week.
The new 2 T-1's go to one ISP, while the 2 others go to 2 separate ISP's. We run BGP between all three ISP's. The new ISP is providing us with full routing tables for the 2 T-1's. (We have 512MB of memory, plenty for this scenario). Another ISP is announcing just their routes to us, while the last ISP (which we have problems constantly with) is announcing just their routes as well. I believe we'll eventually want the other 2 ISP's to announce to us full routing tables as well, which would be ideal.
Here's the problem:
The configuration of BGP is exactly the same for the two T-1's from the new ISP, so apparently most/all traffic is going to one T-1 since they're distinguished only by IP address. The ISP provided us with a configuration to help load balance/load share the two T-1's, but they do so by spliting up our IP blocks equally. This assumes that half the traffic will go to the fist half of the block, and so on.
The other option is to mess with the attributes for BGP, which I'm quite taken aback with, since many documents says tweaking the BGP attributes is very risky.
The larger problem is load balancing among all four T-1's.
Any help would be greatly appreciated!!!
if you have 2 T1 from 1 of your router to 1 router of your isp (sth like R1 ===== R2), configure a loopback address on both router and establsih a bgp session between the loopback.
Then, make sure R1 and R2 knows each other loopback over the 2 T1 (you can use 2 static routes - 1 for each T1).
This is the best way to achieve loadbalancing for this scenario.
If your 2 T1 are going to different routers at your ISP, then you won't be able to control incoming traffic.
For outgoing traffic you can use the command max-path 2 to loadbalance over the 2 T1.
Thanks for the reply. Unfortunately, our 2 T-1's going to different routers at the one ISP. There is no way to change the attributes on BGP to control both incoming and outgoing traffic?
Oh another question for you:
When I issue the command max-path 2 to loadbalance over the 2 T-1's, is it possible to apply this to my overall bandwidth which includes all 4 T-1's?
If you issue the command max-path 2, for a single prefix/route, bgp will select 2 best route in the bgp table inject to the routing forward table, certainly, if you recieve this prefix/route from all 4 bgp neighbors, it's really possible.
you can use local preference to control your outbound traffic, for inbound traffic, you should negotiate with your isp, maybe they permit you use MED to control your inbound traffic policy.
If you have some address block really want to be accessed by outside use a particular link, you can advertise a more specific route out of this bgp session, they will choose this path to get your address. But this is not a recommended way because this maybe conflict with your isp's policy.
First of all you have to differentiate between incoming and outgoing traffic.
For the outgoing traffic you can use the max-path 2 as somebody said in a previous reply, and that is pretty easy to control since it is traffic that originates from your side.
For outgoing traffic is a little bit trickier in your situation.
You can use MED as someone suggested but MED is only indicator/sugestion to the neighboring router which route you preffer, and as previously mentioned it will take cooperation from your ISP. Anothier way is to play with what you advertise to whom. For example if you have lets say to IP ranges you can advertise one of the ranges to one neighbor and the other to the other neighbor. This way you'll load balance between the two connections but only with specific traffic for that connection. But I'm not sure if this is what you want to achieve. Another thing you have to check is to make sure that your ISP has set same prefference on both routes to you, since you are connected to different routers on the ISP side assuming that the ISP has one AS to which you are connected.
But the easyest way to fix this would be to ask the ISP to put both your T1's on the same router and to use the loopback interface on both sides.
Hope this helps
Here's a copy of my bgp config without IP addresses:
router bgp (my AS)
network x.x.x.x mask 255.255.255.0
network y.y.y.y mask 255.255.255.0
network z.z.z.z mask 255.255.255.0
neighbor a.a.a.a remote-as (ISP#1 AS)
neighbor a.a.a.a version 4
neighbor a.a.a.b remote-as (ISP#1 AS)
neighbor a.a.a.b version 4
neighbor c.c.c.c remote-as (ISP#2 AS)
neighbor c.c.c.c ebgp-multihop 5
neighbor c.c.c.c update-source Serial1/1:1
neighbor c.c.c.c version 4
neighbor c.c.c.c distribute-list 1 out
neighbor c.c.c.c route-map (routemap) in
neighbor c.c.c.c route-map (routemap2) out
neighbor d.d.d.d remote-as (ISP#3)
neighbor d.d.d.d version 4
neighbor d.d.d.d distribute-list 1 out
Now can I issue a "maximum-paths 4" to load balance among all 4 lines for outgoing traffic?
Notice in my config that for ISP#1 I have neighbor statment as a.a.a.a and a.a.a.b, thus each T-1 line for the ISP#1 is going to a different router on their end.
First of all lets try to answer your question about the max-path 4. To my knowledge in your case it wont do full load balancing since you are using distribute list on some of the peerings. That means that you are letting those systems to know only some information about your network. You are probably also filtering or adding weight with your route-maps to your peer c.c.c.c. All this will influence the load balancing when used withmax-path. I'm not sure but be carefull with that.
Second, looks like you are doing full exchange with your ISP#1. If you want to do load balancing between all four peers, I think that you'll have to do some modifications on your BGP settings, or you can do separate load balancing with ISP#1 and separate load balancing with ISP#2 and ISP#3.
Lastly, find out how your network is seen from the outside, do a BGP scan, here is a link to help you with that http://nitrous.digex.net/ this will give you some idea of how your network is seen/reached from the outside. Now if you see that the best path to some of your network is though ISP#1 and the second best is through your ISP#2 but with more hops for example, in that case you'll have to use prepend.
Since BGP best path selection uses number of hops as second criteria changing the max-path or local pref or MED will not do the job.
These are just some consideratoins you should have in mind
Hope this helps, correct me please if I'm wrong
Concerning your 2 T1s connecting to the new ISP, if the 2 T1s are connecting to the same ISP router, you may configure the peering using the loopback addresses (don't forget to use ebgp-multihop and any static routes or dynamic routing to let your router know how to reach the ISP's router loopback) load balancing will be achieved not by BGP itself, but via the forwarding process. It may vary, according to the forwarding process you area using. As an example, if you are using CEF, you should enable per-packet load-sharing to have a fair load balancing between the 2 circuits.
Regarding the other 2 peerings, as someone already wrote, it is tricky... You can use local-preference to influence your outbounf traffic and/or agree with your ISPs some MED or BGP community policy to influence inbound traffic.
I hope it helps.
Unfortunately, my 2 T-1s are going to separate routers to the new ISP. I'm going to find out if they can put the T-1's on one router. That way, I can implement your advice.
Also, should i use the max paths 4 command to balance between all 4 of my T-1's?
I don't have CEF, but I've been reading up on it, is it hard to implement? Any consequences regarding implementing this layer 3 switching technology?
Thanks for all the replies!
Max-Paths only have a word on the BGP selection algorithm if reaches a certain point. Please check the following link:
What I mean is, there are a lot of things that BGP checks before deciding based on max-paths if they are configured. BGP by default doesn' to load balancing.
the max-path command only works for route that you learned from 1 ISP over multiple link.
So, in your case only 2 T1.
There is a new feature called eibgp multipath but not sure if it is available yet and if it could do anything good in your case.
For the outgoing traffic, the only thing you can do is control what routes you advertise and the as-path (you can use as-path prepend to make a route look worst).
I really suggest you enable CEF. Lot of features require CEF to be enable nowadays.
To enable CEF :
ip cef ! global command
ip route-cache cef
So what you're saying is that if I put in the command
"maximum-paths 2" into my bgp config, then the router will balance outgoing traffic through the 2 T-1's going to the same ISP? The command will not affect the other 2 separate T-1s?
As for CEF, unfortunately, it will only work if the ISP implements CEF on their end. I already called our new ISP with the 2 T-1's, and they do not use it, and will not turn it on for our sake.
if the best path is learned via the ISP to which you have 2 T1, then 2 routes will be inserted in your routing table (if you configure the max-path 2 command).
If the best path is through another ISP, only 1 route will be installed.
for CEF. You're wrong.
CEF is not a protocol. This is the name given to the internal function that does the packet forwarding on the router.
In other words, this is something local to your router. It does not matter whether or not your ISP uses CEF.
Does the BGP session with two T1's go to the same ISP on the same remote router? Or is this a setup where you have a T1 going to ISP-A-Router1 and another going to ISP-A-Router2? If it is two t1's between the same two routers, peer with loopbacks, that will load balance for you. If you have ISP-A-Router 1 and Router 2 scenerio, maybe MEDS is something you want to look at.
The setup is my one router where there are 4 T-1's. 2 each go to a separate ISP, While the 2 other
T-1's terminate to two separate routers to the same ISP:
|----------| T-1 #1 _______________
| |---------------------|ISP#1 router |
| | ---------------
| | T-1 #2 _______________
| |---------------------|ISP#2 router |
| MY | ---------------
| Router | T-1 #3 ________________
| |---------------------|ISP#3 router#1|
| | ----------------
| | T-1 #4 ________________
| |---------------------|ISP#3 router#2|
What the new ISP suggested isn't wrong and it really works to load balance your incoming traffic between 2 T1 line if your IP network addressed well. Meanwhile, you can announce the whole IP block to them via each T1, that provides you redundancy. Where are your IP block getting from? Each ISP?
okey, tell me more about your network:
1. Are you using the private AS number?
2. Does each ISP announce your 3 net blocks to their peers? this question related to your contract with them.
3. Between incoming and outgoing traffic, which one is bottleneck since you only own T1 line for upstreaming.
Ok. So now you're trying to loadbalance incoming traffic.
Since you have 3 blocks, and 3 ISPs, I would suggest to advertise the 3 block to each isp the following way.
NetB + as prepend
NetC + as prepend
NetA + as prepend
NetC + as prepend
NetA + as prepend
NetB + as prepend
Like this, the prefer route to reach NetA is via ISPA, NetB via ISPB and NetC via ISPC.
This off course won't give you true loadbalancing but at least all your T1 will be used.
To prepend a route, use the following commands:
route-map ToISPA permit 10
match ip address 101
set as-path prepend
route-map ToISPA permit 20
match ip address 199
acc 101 permit ip
acc 101 permit ip
acc 199 permit ip any any