BGP design suggestion

Unanswered Question
Oct 15th, 2013
User Badges:

Hi All,

We have 3 ISPs providing us MPLS links to our Datacenter Routers as a primary and Secondary ( Kindly refer to the attached diagram ppt).These will be connected on our Primary and Secondary MPLS Routers  in Auto-failover mode. We have n numbers of remote sites connected via these ISPs ( only 2 links at a single site in primary and secondary mode).

Our requirement is to configure the BGP at DC end routers in auto-failover mode and same is at remote site end.At a time only primary links must be in use and secondary would be functional when primary goes down.

WAN IPs are differ but LAN IPs are common for all the providers ( means DC and Remote side LAN) and we have /27 WAN pool at DC end to connect ISPs  in a mannar as follows:

DC : primary : 10.x.1.2         ISP-1 Primary :  10.x.1.3

DC : primary : 10.x.1.2         ISP-2 Primary : 10.x.1.4

DC : primary : 10.x.1.2         ISP-3 Primary : 10.x.1.5

                

DC : Secondary : 10.x.2.2       ISP-1 Primary : 10.x.2.3

DC : Secondary: 10.x.2.2        ISP-1 Primary : 10.x.2.4

DC : Secondary : 10.x.2.2        ISP-1 Primary : 10.x.2.5


and at Remote end :


Only 2 ISPs can be connected as a primary and secondary using BGP and auto failover form

ISP-1 primary - 10.x.x.32/30

IPS-2 Secondary - 10.y.y.160/30


LAN pool- /27


Request you all to suggest\ provide a sample configuration on BGP.


Thanks a lot !


Regards,


Anil K.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4 (1 ratings)
Loading.
devils_advocate Tue, 10/15/2013 - 00:57
User Badges:
  • Gold, 750 points or more
  • Community Spotlight Award,

    Small Business, November 2015

I fail to see why someone should come up with a design and configuration for you, get someone in as a contractor. There are some great people on this forum but I would not be trusting my network design and config from someone I have never met.

Vasilii Mikhail... Tue, 10/15/2013 - 10:24
User Badges:
  • Gold, 750 points or more

Hello.


The configuration will be really simple, but this quick solution would not be efficient enough for production! You will have to tune it!


But several question before:

  • do you have a requirement for symmertic routing?
  • how do you see ISPs internal AS[s] - as single AS, or as several?
  • will your ISPs accept BGP AS numbers of another ISPs? Are they agree about AS numbers per site/DC?
  • how many routers do you have per remote location?
  • what if remote site A has ISP1 as primary and ISP2 as secondary, but site B has ISP2 as primary and ISP1 as secondary - then what links should be used for traffic A->B and for traffic B->A ? Should they traverse Datacenter even they have common ISPs?
  • what if remote site A has ISP1 as primary and ISP2 as secondary, but site B has ISP3 as primary and ISP2 as secondary - then what links should be used for traffic A->B and for traffic B->A ? Should they traverse Datacenter even they have common ISP2?
  • how does DC LAN know which WAN route to forward traffic to? Do you use HSRP/VRRP or some IGP?
Vasilii Mikhail... Tue, 10/15/2013 - 10:43
User Badges:
  • Gold, 750 points or more

Three more question:

  • do your ISPs have any routing policy based on community values? Do they support MED?
  • is it allowed to inject (by DC) 0.0.0.0/0 into ISPs' clouds?
  • what summaries could we use to summarize: a) DC only, b) whole network?
rite2anil Wed, 10/16/2013 - 01:22
User Badges:

Hi MikhailovskyVV,


Thanks for the replying.

Your question's answers are as follows :


  • do you have a requirement for symmertic routing?

Yes.Our Remote site's LAN should reach to DC via primary ISP( whichever is active at the time works as primary) and that ISP should respond from DC.


  • how do you see ISPs internal AS[s] - as single AS, or as several?

as a Single AS.


  • will your ISPs accept BGP AS numbers of another ISPs? Are they agree about AS numbers per site/DC?

All three ISPs have their seperate BGP AS no. and our DC and remote sites AS no are constant.As at site end remains single for all the sites for all the providers.


  • how many routers do you have per remote location?

Its around 500+ routers per provider.

  • what if remote site A has ISP1 as primary and ISP2 as secondary, but site B has ISP2 as primary and ISP1 as secondary - then what links should be used for traffic A->B and for traffic B->A ? Should they traverse Datacenter even they have common ISPs?

Yes.This is a situation where one site is using ISP1 as a primary but other site is using ISP1 as a secondary link. Only the up link should send \ receive the data. if in case, both the primary and secondary links are UP, only primary should send\receive the data.secondary remains passive.


  • what if remote site A has ISP1 as primary and ISP2 as secondary, but site B has ISP3 as primary and ISP2 as secondary - then what links should be used for traffic A->B and for traffic B->A ? Should they traverse Datacenter even they have common ISP2?

Look, we have aprox 1800 remote site's locations deivided among three ISP's as per their feasibility to the end locations. We have given WAN and LAN IP pools to each ISPs as per their no of sites they are providing connectivity.So in case where a site is having one ISP as a primary and others site the same IPS is as a secondary. doesnt matter.Individual site will follow the same rule i.e only UP and one link will be used for sending\receiving the data.


  • how does DC LAN know which WAN route to forward traffic to? Do you use HSRP/VRRP or some IGP?

We have TWO MPLS routers using HSRP. Primary is used to forward the DC LAN to the know WAN route.

  • do your ISPs have any routing policy based on community values? Do they support MED?

They will create VRFs for their respective sites LAN \ WAN pools towards DC . Yes They support MED.


  • is it allowed to inject (by DC) 0.0.0.0/0 into ISPs' clouds?

No. only 10.0.0.0\8 and 10.96.0.0\12 is allowed to inject by DC into ISPs clouds.


  • what summaries could we use to summarize: a) DC only, b) whole network?

At DC : 10.96.0.0\12 and 10.112.0.0 ( DR & DR) and 10.0.0.0\8 for the whole network.


Regards,


Anil

Vasilii Mikhail... Wed, 10/16/2013 - 10:45
User Badges:
  • Gold, 750 points or more

Here are some design notes - please let me know you thought about each of them:

- spokes should advertise only localy originated routes (as-path-filter $^);

- spokes should advertise local prefixes via primary link "as is", and with 3 prepends via secondary;

- spokes assign local preference 200 for inbound prefixes over primary link, keep default (100) over secondary;

- spokes filter out all inbound updates except of originated from HUB (by  originated AS or community);


- over primary links Hub advertises all routes it learnt from ISPs + 10.0.0.0/8 + 10.96.0.0/12 + 10.112.0.0, setting MED to 50;

(if you do not advertise spokes' specifics, then there is no way for provider to route traffic over DC, but not via site's secondary link)


- over secondary links Hub advertises 10.0.0.0/8 (only), setting MED to 100;

- Hub accepts all the routes (10.0.0.0/8 le 30), but set local preference to 200 over primary link;

- Hub routers should have iBGP.

- Hub should have unique (not equal to spoke) AS number.



Future optimizations:

- each site should have unique ID (let's say Hub has ID = 96);

- site should accept inbound route if it has community with local ID = SiteID:100 (should set local preference 100) and = SiteID:300 (should adjust local preference to 300);

- if all the spokes are using single AS number, it's expected that ISPs are using "neighbor as-override".

rite2anil Sun, 10/20/2013 - 21:32
User Badges:

Hi Mikhailovsky,


Yes, these points are required.


-spokes should advertise only localy originated routes (as-path-filter $^);

- spokes should advertise local prefixes via primary link "as is", and with 3 prepends via secondary;

- spokes assign local preference 200 for inbound prefixes over primary link, keep default (100) over secondary;

- spokes filter out all inbound updates except of originated from HUB (by originated AS or community);


- over primary links Hub advertises all routes it learnt from ISPs + 10.0.0.0/8 + 10.96.0.0/12 + 10.112.0.0, setting MED to 50;

(if you do not advertise spokes' specifics, then there is no way for provider to route traffic over DC, but not via site's secondary link)


- over secondary links Hub advertises 10.0.0.0/8 (only), setting MED to 100;

- Hub accepts all the routes (10.0.0.0/8 le 30), but set local preference to 200 over primary link;

- Hub routers should have iBGP.

- Hub should have unique (not equal to spoke) AS number.


Regards,


Anil K.

Vasilii Mikhail... Tue, 10/22/2013 - 09:49
User Badges:
  • Gold, 750 points or more

Hello, here is a draft config (had no as-override feature ion my IOS, so had to use different AS-number per remote site, but that changes nothing):


HUB-primary router:


ip prefix-list TO_BGP seq 5 permit 10.0.0.0/8 le 29


route-map BGP_IN_PRIMARY permit 10

set local-preference 200

route-map BGP_OUT_PRIMARY permit 10

match ip address prefix-list TO_BGP

set metric 50


router bgp 111

no synchronization

bgp log-neighbor-changes

network 10.0.0.0 <- you may use static to null0, so BGP would advertise it

network 10.96.0.0 mask 255.240.0.0 <- you may use static to null0, so BGP would advertise it

neighbor 10.1.1.3 remote-as 1

neighbor 10.1.1.3 route-map BGP_IN_PRIMARY in

neighbor 10.1.1.3 route-map BGP_OUT_PRIMARY out

neighbor 10.2.1.4 remote-as 2

neighbor 10.2.1.4 route-map BGP_IN_PRIMARY in

neighbor 10.2.1.4 route-map BGP_OUT_PRIMARY out

neighbor 10.3.1.5 remote-as 3

neighbor 10.3.1.5 route-map BGP_IN_PRIMARY in

neighbor 10.3.1.5 route-map BGP_OUT_PRIMARY out

neighbor 10.96.0.2 remote-as 111

neighbor 10.96.0.2 next-hop-self

no auto-summary


HUB secondary:


ip prefix-list SUMMARY_ONLY seq 5 permit 10.0.0.0/8


route-map BGP_OUT_SECONDARY permit 10

match ip address prefix-list SUMMARY_ONLY

set metric 100


router bgp 111

no synchronization

bgp log-neighbor-changes

network 10.0.0.0

network 10.96.0.0 mask 255.240.0.0

neighbor 10.1.2.3 remote-as 1

neighbor 10.1.2.3 route-map BGP_OUT_SECONDARY out

neighbor 10.2.2.4 remote-as 2

neighbor 10.2.2.4 route-map BGP_OUT_SECONDARY out

neighbor 10.3.2.5 remote-as 3

neighbor 10.3.2.5 route-map BGP_OUT_SECONDARY out

neighbor 10.96.0.1 remote-as 111

neighbor 10.96.0.1 next-hop-self

no auto-summary


Remote-site:


ip as-path access-list 1 permit ^$


ip as-path access-list 111 permit _111$


route-map BGP_IN_SECONDARY permit 10

match as-path 111


route-map BGP_IN_PRIMARY permit 10

match as-path 111

set local-preference 200


route-map BGP_OUT_SECONDARY permit 10

match as-path 1

set as-path prepend 6 6 6


route-map BGP_OUT permit 10

match as-path 1


router bgp 6

no synchronization

bgp log-neighbor-changes

network 10.6.0.0 mask 255.255.255.224

neighbor 10.6.0.33 remote-as 1

neighbor 10.6.0.33 route-map BGP_IN_PRIMARY in

neighbor 10.6.0.33 route-map BGP_OUT out

neighbor 10.6.0.161 remote-as 2

neighbor 10.6.0.161 route-map BGP_IN_SECONDARY in

neighbor 10.6.0.161 route-map BGP_OUT_SECONDARY out

no auto-summary

Actions

This Discussion

Related Content