Routing Question with Multihomed Router

[jeff]

We have a 7507 router that has 2 Internet providers with 3 T1s each, both carriers (A and B) are used in balancing the traffic. We are doing BGP peering with both of them with an Arin assigned AS number and we have 64 class C networks assigned to us from Arin. Two of my biggest clients are using a lot of traffic (confirmed valid traffic) to hosts on the Internet. These hosts that both clients are communicating with are preferring one carrier (B) over the other. This is causing the one provider's (B) circuits to run at over 80%. Both of my clients have static class C addresses that we assigned to them, is it possible for me to route the traffic from one client down the one provider (A) who is not running at 80%? I'm sure if I do a static route statement of this one clients class C down the one provider (A), if that one provider goes down this client won't route down the other provider (B) unless I possibly use a second route statement using a high metric pointing down the second provider (B).

Thanks for any input to my problem.

Jeff

[ruwhite]

Without knowing a good bit about the network topology, it's kindof hard to give a positive answer. I would think the easiest thing to do is to set the weight on the router that peers with one of these customers so the routes to the specific destinations eating up the traffic will be set with a next hop of the router peered to SP A. If there aren't any intermediate hops that could redirect the traffic towards the other SP along the way, this could be the simplest solution--and if the route learned through SP A fails, it will automatically switch over to B again.

Another option would be to use PBR at the router for B, pushing the traffic for those hosts, and from one of the two customers, towards A. This would work, even giving you failover, as long as you don't mind the configuration, and your boxes can handle CEF PBR (so you don't slow switching down too much).

The problem may not be outbound, though--are you certain it's the outbound traffic that's clogging the link up? If it's inbbound, then you could try prepending the AS Path on one of the two customer's routes through B, to push the inbound traffic over to B. If that doesn't work, then you could actually use conditional advertisement to only advertise one of the customer's routes out to A, and the other out to B. Conditional advertisement allows the advertisement of destinations when the other SP link fails....

You might could cook something up with MPLS and VRF select, if it's outbound traffic that's causing the problem...

My first issue would be to find out if its outbound or inbound traffic causing the link utilization to go high, and then I'd work from there--if you can characterize the problem more, post more details here, and we'll work with you on the right solution for the problem at hand.

:-)

Russ

[jeff]

Russ,

Thank you for the reply.

We are an ISP and we have the 7507 with 6 T1s, three to provider A and three to provider B. The local Ethernet port on the 7507 is connected to our local LAN. Our LAN consists of servers and multiple 2501 and 2600 routers. Most of the servers and routers are for our clients services or networks. Our network of routers on the LAN are using static routes not a dynamic protocol and the default gateway for the servers and routers is the 7507. I checked our stats as you mentioned and provider B circuits are running at over 80% outbound (to the Internet). Inbound (to us) is at only 40%. Provider A circuits are balanced just fine, but I did a traceroute on the traffic communicating with my two largest clients and their traffic outbound seems to prefer provider B.

Thanks for any help.

Jeff

[ruwhite]

Ah, okay... Both ISPs are peering into the same router. This makes things a bit simpler, I think. I would set up a route map matching an access list, and setting the local preference on some number of routes learned from your upstream so you can control the preferred outbound route. Something like this:

access-list 10 permit 1.0.0.0 0.255.255.255

!

route-map adjust-traffic permit 10

match ip address 10

set local-preference 110

!

router bgp xxxx

neighbor route-map adjust-traffic in

You can add /8's to the access list until you see the traffic start pulling over from one link to the other. You could use /7's or something even shorter if you want, but it seems a little more difficult to get them right in my head, and a maximum of 255 entries in an access list doesn't seem all that bad to me.

On consideration, though, I would use a named access list for this, so you can edit out specific lines, etc., as needed, to adjust traffic levels.

:-)

Russ.W

[jeff]

Russ,

Thank you for the reply.

When you say "setting the local preference on some number of routes learned from your upstream so you can control the preferred outbound route" would this apply if i'm only receiving local routes from my providers? Since I'm still learning the details of BGP and such, could you give me more detail of this?

In your example you have "neighbor route-map adjust-traffic in", would this be configured for "out" since my high utilization is on the outbound? Below is a link to a post from a previous problem that was resolved with your help until recently with my two clients traffic usage and this one provider. The link will show you my 7507 router configuration.

http://forums.cisco.com/eforum/servlet/NetProf?page=netprof&CommCmd=MB%3Fcmd%3Dpass_through%26location%3Doutline%40%5E1%40%40.eea5b7f

Thanks again for all your help.

Jeff

[ruwhite]

"When you say "setting the local preference on some number of routes learned from your upstream so you can control the preferred outbound route" would this apply if i'm only receiving local routes from my providers?"

You would need to accept full routes from your provider. What I would do is to accept full routes, and set the local pref for routes originating in the provider's network to some high number, say 120, then adjust the rest with for traffic flow using a local pref of 110.

"would this be configured for "out" since my high utilization is on the outbound?"

Nope, inbound. Your inbound routes will control your outbound traffic flow.

If you want, you can email me off line, and I can probably explain better there, or point you to materials that might help.

:-)

Russ.W

riw@cisco.com