BGP Multihome - different ISPs - not honoring prepends

Unanswered Question
Oct 6th, 2011

I'm setting up a multi-home scenario:

1) One site

2) /24 IP prefix

3) Registered, public AS

4) Two links (two different ISPs) - ISP A and ISP B

5) iBGP between them

6) ISP A should be primary for all inbound and outbound traffic

7) ISP B should be backup for all inbound and outbound traffic - ie, no traffic on this link in normal operation

I've done this with the use of local-pref inbound (and we're only receiving the default from both ISPs) and as path prepends outbound on the ISP B router.

All seems to be functioning more or less as expected, except that lookups on certain public route servers out on the net show the ISP B path as being preferred.

I assume this may be due to some form of route filtering or manipulation on the part of the given provider who's route server I'm looking at (such as filtering ISP A to only use routes they originate), but I'm wondering if there's anything I can do about this.

We really want to have the backup circuit completely un-used in normal operations (for a variety of reasons), but that is currently not the case.

Here's some config snippets, obviously altered to protect the innocent:

ISP A router (Primary):

ip prefix-list default description Matches default route

ip prefix-list default permit 0.0.0.0/0

!

ip prefix-list main-subnet description Matches x.x.x.x/24 subnet

ip prefix-list main-subnet permit x.x.x.x/24

!

!

route-map ispa-in permit 10

match ip address prefix-list default  

set local-preference 200             

!

!

route-map ispa-out permit 10

match ip address prefix-list main-subnet
!
!

router bgp 12345

no synchronization

bgp log-neighbor-changes

timers bgp 15 45

network x.x.x.x mask 255.255.255.0

neighbor 1.2.3.4 remote-as <ISP-A--AS>

neighbor 1.2.3.4 soft in

neighbor 1.2.3.4 route-map ispa-out out

neighbor 1.2.3.4 route-map ispa-in in
neighbor other.local.router remote-as 12345
neighbor other.local.router next-hop-self neighbor other.local.router soft in


ISP B router (backup):

ip prefix-list default description Matches default route

ip prefix-list default permit 0.0.0.0/0

!

ip prefix-list main-subnet description Matches x.x.x.x/24 subnet

ip prefix-list main-subnet permit x.x.x.x/24

!

!

route-map ispb-in permit 10

match ip address prefix-list default  

set local-preference 150              

!

!

route-map ispb-out permit 10

match ip address prefix-list main-subnet

set as-path prepend 12345 12345 12345 12345 12345

!

!

router bgp 12345

no synchronization

bgp log-neighbor-changes

timers bgp 15 45

network x.x.x.x mask 255.255.255.0

neighbor 5.6.7.8 remote-as <ISP-B--AS>

neighbor 5.6.7.8 soft in

neighbor 5.6.7.8 route-map ipsb-out out

neighbor 5.6.7.8 route-map ispb-in in

neighbor other.local.router remote-as 12345

neighbor other.local.router next-hop-self

neighbor other.local.router soft in

ip prefix-list default description Matches default route
ip prefix-list default permit 0.0.0.0/0
!
ip prefix-list main-subnet description Matches x.x.x.x/24 subnet
ip prefix-list main-subnet permit x.x.x.x/24
!
!
route-map ispb-in permit 10
match ip address prefix-list default  
set local-preference 150              
!
!
route-map ispb-out permit 10
match ip address prefix-list main-subnet
set as-path prepend 12345 12345 12345 12345 12345
!
!
router bgp 12345
no synchronization
bgp log-neighbor-changes
timers bgp 15 45
network x.x.x.x mask 255.255.255.0
neighbor 5.6.7.8 remote-as <ISP-B--AS>
neighbor 5.6.7.8 soft in
neighbor 5.6.7.8 route-map ipsb-out out
neighbor 5.6.7.8 route-map ispb-in in
neighbor other.local.router remote-as 12345
neighbor other.local.router next-hop-self
neighbor other.local.router soft in
I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 3.2 (5 ratings)
Marwan ALshawi Thu, 10/06/2011 - 14:26

The logic of the config from your end looks ok

Try to contact ISPB and ask for a bgp community the you add it to the route advertised over ISP B so that they give it less preference within their cloud

Hope this help

Sent from Cisco Technical Support iPhone App

darren.g Thu, 10/06/2011 - 17:31

Duncan C MacInnes wrote:

I'm setting up a multi-home scenario:

1) One site

2) /24 IP prefix

3) Registered, public AS

4) Two links (two different ISPs) - ISP A and ISP B

5) iBGP between them

6) ISP A should be primary for all inbound and outbound traffic

7) ISP B should be backup for all inbound and outbound traffic - ie, no traffic on this link in normal operation

I've done this with the use of local-pref inbound (and we're only receiving the default from both ISPs) and as path prepends outbound on the ISP B router.

All seems to be functioning more or less as expected, except that lookups on certain public route servers out on the net show the ISP B path as being preferred.

I assume this may be due to some form of route filtering or manipulation on the part of the given provider who's route server I'm looking at (such as filtering ISP A to only use routes they originate), but I'm wondering if there's anything I can do about this.

We really want to have the backup circuit completely un-used in normal operations (for a variety of reasons), but that is currently not the case.

Duncan.

I ran into exactly the same scenario at my last job (and something very similar at this one), and unfortunately I couldn't find out how to do anything about - especially if the upstream ISP's are big enough.

Sometimes, no matter how much you prepend, the AS path is just going to be shorter from ISP B than it is from ISP A for inbound traffic, resultiting in traffic being routed in via ISP B - especially for other clients who directly connect to ISP B, and are either in their AS (single homed to ISP B), or who are a direct downstream from ISP B and several hops from ISP A's interconnect with ISP B.

I route all outbound traffic via my ISP A, but inbound I can't seem to do Jack about.

About the only thing you could try is to contact ISP B and see if they can damp the precedence within their AS - but they're not likely to be co-operative with that, since it introduced inefficiences within their AS.

Cheers.

Marwan ALshawi Thu, 10/06/2011 - 18:46

ISPs they use BGP community string for policy routing

each ISP has some community stings that make any route within its cloud preferred to its customers even if you prepend because they increase the local preference within their cloud

for example if you want to access prefix x.x.x.x from a client whos ISPB customer the path over ISPB will prefered rather than the path over Inter ISP link between ISPB and ISPA

thats why if you ask ISPB what the bgp community string you need to ad to the route in addition to the AS prepend to make it less preferred this mostly will fix it

Hope this Help

kishore.chennupati Thu, 10/06/2011 - 22:52

you dont need BGP community in this case as you already are using the as path prepend on your back up router. So your config should work just fine. You use communities when you want your ISP to manipulate the BGP attributes at their end. In this case you are dong everything so it should be fine

HTH

Please rate if helpful

Marwan ALshawi Fri, 10/07/2011 - 01:26

Kishore

the original poster has mentioned that although everything is configured correct the path selection is not working as desired and i suggested him to confirm with ISP B regarding the BGP community string to make sure they do not give preference to the advertised prefixes within their cloud over the inter ISP link

hope this help

kishore.chennupati Fri, 10/07/2011 - 06:39

marwan,

sorry i sent my msg using the cisco iphone app. It didnt display the whole original post so I didnt see the whole description.

Kishore

milan.kulik Fri, 10/07/2011 - 05:30

Hi Darren,

I agree: As long as you are advertising your prefixes to both ISPs, you can't be 100% sure the secondary ISP connection will not be used for incoming traffic.

So one idea:

Why not to try some kind of BGP conditional advertisement?

See  http://www.cisco.com/en/US/customer/tech/tk365/technologies_configuration_example09186a0080094309.shtml

It might be possible to advertise my prefixes to the secondary ISP only in case the BGP session to the primary ISP fails?

HTH,

Milan

Marwan ALshawi Fri, 10/07/2011 - 06:08

Well this conditional advertisement should work but it is better to have the range advertised over both as it's using public range and public AS number over the Internet ( quicker re routing over the Internet )

Sent from Cisco Technical Support iPhone App

haleoncco Fri, 10/07/2011 - 06:30

There are some good points above, thank you.

I think I'm going to end up taking a blend of all of the suggestions.

I'm going to start by contacting ISP B and seeing about a community or dampening that will make it the less-preferred route in their cloud.

Failing that, I'll just toss on a bunch of prepends and it will have to be good enough.

The conditional advertisement was something I had contemplated as a solution, but it felt kind of like using a sledgehammer where a ballpeen hammer would do.

Where I think I will use the conditional advertisement is when we stand up the DR site. It has a separate AS and I planned on having it be the third advertisement point for the subnet, but there won't be any IBGP between DR and the Main site, so I definitely can't have active traffic coming there. A conditional advertisement would probably be the right call there, though I'll need to figure out how to craft it.

I would assume I can't use the main /24 as the non-exist condition, since even if the main site went dark, as soon as the DR site started advertising the main /24, it would again see that /24, match the non-exist and pull the advertisement.

Maybe I can do an eBGP multihop between the sites and pass a private /32 loopback or something...

Need to think on that.

Thanks again!

Marwan ALshawi Fri, 10/07/2011 - 17:11

You are in the right track now

Confirm with ISPB if they can provide you with bgp comma ity string to make this route with less prefered in conjunction with AS prepending you did

Good luck

Plz rate the helpful posts

Actions

Login or Register to take actions

This Discussion

Posted October 6, 2011 at 12:35 PM
Stats:
Replies:10 Avg. Rating:3.2
Views:1758 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard