Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Webcast-Catalyst9k
New Member

BGP Multihoming design topology

Hello,

Currently we have a 50mb pipe with our carrier SONIC. We have signed another contract with another provider here in town (Charter) to multihome our Internet connections in an active/active configuration. We have leased our /24 space through our carrier SONIC. ARIN has already approved our org-ID for an ASN and they will be sending us that once the billing portion is finished.

There a few design considerations I was hoping I could get some insight from the community on.. Before I start, the ultimate goal for us to use BOTH Internet connections in an active/active configuration - utilizing both pipes..

Disclaimer: I have gathered this design from a lot of other posts that have somewhat of a similiar topology with ASA-->3750-->router pair-->CPE--internet.. Please keep an open mind if you think im on the wrong track..

Please see the attached design topology.

Questions related to design:

  • What kind of routes should I get from each carrier? I have been told that partial/partial routes plus a default route form each carrier is the way to go. Also, I've heard mention that full routes from both carriers are preferred. My ASR1001's can support ~500k routes. I know the global table is approximately ~337k routes. My goal is to use both pipes and use the best outbound path per carrier.

  • We will be leasing our /24 space from SONIC. I plan on running OSPF on the DC-Edge-SW1 in conjunction with iBGP - so I can default originate two equal cost routes back to my ASA. My confusion is when the traffic hits DC-Edge-SW1, there will be default equal-cost iBGP routes to both ASR1001's (DC-Edge-RT1 & DC-Edge-RT2). If the switch does not have the BGP table, it will just load-share across both ASR's. When the traffic hits the ASR's, will they know which carrier has the best path and route accordingly?

  • Should the iBGP connection between both routers be directly connected ? Or will it suffice through the L3 3750 connection? Also, with the limitations on the routes for the ASR1001 at ~500k. If we end up getting full routes from carriers and create a iBGP neighborship between both routers, will this exceed the route limitations on this platform?

  • On both routes, I will have the network statement 'network 12.231.69.0 mask 255.255.255.0.' This is a leased network from SONIC, and we NAT everything on our ASA to 12.231.69.10. My question is, will this be a problem broadcasting this network from our AS to both carriers AS?

  • Refer to bgp-design.jpg - is it a requirement that I use our leased public subnet 12.231.69.0/24 for the interfaces from ASA5510 -> 3750 -> ASR1001?

Thank's in advance for any assistance/insight you can provide as this is the most advanced topology I have worked with.

1 ACCEPTED SOLUTION

Accepted Solutions
New Member

Re: BGP Multihoming design topology

Hi Cam,

There is one problem with using 2 x /30's between your edge routers and the 3750 -> ASA, when your iBGP session advertises routes to the other peer they will be blackholed / unreachable.

For example say DC-Edge-RT1 (12.231.69.2)  advertises a route for 208.67.0.0/16 to DC-Edge-RT2 via iBGP, DC-Edge-RT2 will know that 208.67.0.0/16 is via 12.231.69.2. To forward a packet to 208.67.0.0 DC-Edge-RT2 will lookup the route to 12.231.69.2 and will find a route via OSPF which is via the 3750.

DC-Edge-RT2 will then spit out a packet destined for 208.67.0.0/16 out the interface to the downstream 3750, this is where the problem occurs, the 3750 doesn't have a clue how to get to 208.67.0.0/16 because unlike the BGP routers it doesn't have the full table. Well it would use the default routes but you have two of those so that's undesirable.

A solution could be (same one I used):

Setup a HSRP group between your two edge routers so they share a virtual address say: 12.231.69.2/29, give RT1 12.231.69.3 for it's physical address and RT2 12.231.69.4 for it's physical address. Then create a Vlan on your 3750 with a Layer 3 SVI with an IP of 12.231.69.5.

Just need one default route then: 0.0.0.0 0.0.0.0 12.231.69.2

iBGP will still do some amount of load-sharing depending on the information in the various BGP tables...

If and when you get this up and running you can view the BGP path (and thus your AS Number) from the following LG:

http://lg.he.net/

Do a bgp route query for your prefix, will show the AS number and AS-PATH

Any questions, holla! 

Oh, just a thought, if there's no requirement for your 3750 to be Layer 3, then you could just create the same HSRP group but put both your Edge routers & your ASA in the same Layer 2 VLAN with no SVI and then set your HSRP group as the Default Gateway for your ASA's WAN interface.....

Dunc.

17 REPLIES
New Member

BGP Multihoming design topology

Not the most popular thread I see

Anyways, i've progressed my configuration and after researching have decided full routes on the ASR1001 platform should work fine. I found a really good white paper on this example below - I will model my configuration after this example.. It is somewhat dated (Feb.2004) I know the

soft-reconfiguration inbound take up some memory and it's not necessarily needed that much anymore..

Example 2: Single Site, Dual Routers, PA-space, Multihoming to Two ISPs

Background: This example is the same as Example 1, except the customer is using two routers, and

connecting to one ISP on each router. The purpose of this example is to show how iBGP is used to

internally propagate BGP routes between the two routers.

Router1Router2

! Router 1: Dual-router, PA-space, Basic Config

!

autonomous-system 900

!

ip route 198.18.1.0 255.255.255.0 Null0 200

!

ip prefix-list ANNOUNCE description Our External Netblocks

ip prefix-list ANNOUNCE seq 10 permit 198.18.1.0/24

!

ip prefix-list BOGONS description Bad Routes to Block In

ip prefix-list BOGONS seq 10 deny 0.0.0.0/8 le 32

ip prefix-list BOGONS seq 15 deny 10.0.0.0/8 le 32

ip prefix-list BOGONS seq 20 deny 127.0.0.0/8 le 32

ip prefix-list BOGONS seq 25 deny 172.16.0.0/12 le 32

ip prefix-list BOGONS seq 30 deny 192.0.2.0/24 le 32

ip prefix-list BOGONS seq 35 deny 192.168.0.0/16 le 32

ip prefix-list BOGONS seq 40 deny 224.0.0.0/3 le 32

! Prevent someone else from announcing your own prefix(es)

! back to you, for security: update this with YOUR

! actual announced block(s)!

ip prefix-list BOGONS seq 1000 deny 198.18.1.0/24 le 32

ip prefix-list BOGONS seq 9999 permit 0.0.0.0/0 le 27

!

router bgp 900

no synchronization

bgp log-neighbor-changes

bgp dampening

network 198.18.1.0 mask 255.255.255.0

neighbor 1.1.1.1 remote-as 200

neighbor 1.1.1.1 description BGP Transit to ISP-1

neighbor 1.1.1.1 version 4

neighbor 1.1.1.1 send-community

neighbor 1.1.1.1 soft-reconfiguration inbound

neighbor 1.1.1.1 prefix-list BOGONS in

neighbor 1.1.1.1 prefix-list ANNOUNCE out

neighbor 1.1.1.1 maximum-prefix 140000

! define iBGP session

neighbor 198.18.1.2 remote-as 900

neighbor 198.18.1.2 description iBGP to Router2

neighbor 198.18.1.2 version 4

neighbor 198.18.1.2 send-community

neighbor 198.18.1.2 soft-reconfiguration inbound

! make sure you are using a Loopback0 /32 on each

! router, and propagating this route internally.

! This is important to the stability of your iBGP

! sessions, so they are not tied to a physical

! interface address.

neighbor 198.18.1.2 update-source Loopback0

autonomous-system 900

!

ip route 198.18.1.0 255.255.255.0 Null0 200

!

ip prefix-list ANNOUNCE description Our External Netblocks

ip prefix-list ANNOUNCE seq 10 permit 198.18.1.0/24

!

ip prefix-list BOGONS description Bad Routes to Block In

ip prefix-list BOGONS seq 10 deny 0.0.0.0/8 le 32

ip prefix-list BOGONS seq 15 deny 10.0.0.0/8 le 32

ip prefix-list BOGONS seq 20 deny 127.0.0.0/8 le 32

ip prefix-list BOGONS seq 25 deny 172.16.0.0/12 le 32

ip prefix-list BOGONS seq 30 deny 192.0.2.0/24 le 32

ip prefix-list BOGONS seq 35 deny 192.168.0.0/16 le 32

ip prefix-list BOGONS seq 40 deny 224.0.0.0/3 le 32

! Prevent someone else from announcing your own prefix(es)

! back to you, for security: update this with YOUR

! actual announced block(s)!

ip prefix-list BOGONS seq 1000 deny 198.18.1.0/24 le 32

ip prefix-list BOGONS seq 9999 permit 0.0.0.0/0 le 27

!

router bgp 900

no synchronization

bgp log-neighbor-changes

bgp dampening

network 198.18.1.0 mask 255.255.255.0

! define BGP session with ISP-2 (ASN 300)

neighbor 2.2.2.1 remote-as 300

neighbor 2.2.2.1 description BGP Transit to ISP-2

neighbor 2.2.2.1 version 4

neighbor 2.2.2.1 send-community

neighbor 2.2.2.1 soft-reconfiguration inbound

neighbor 2.2.2.1 prefix-list BOGONS in

neighbor 2.2.2.1 prefix-list ANNOUNCE out

neighbor 2.2.2.1 maximum-prefix 140000

! define iBGP session

neighbor 198.18.1.1 remote-as 900

neighbor 198.18.1.1 description iBGP to Router1

neighbor 198.18.1.1 version 4

neighbor 198.18.1.1 send-community

neighbor 198.18.1.1 soft-reconfiguration inbound

! make sure you are using a Loopback0 /32 on each

! router, and propagating this route internally.

! This is important to the stability of your iBGP

! sessions, so they are not tied to a physical

! interface address.

neighbor 198.18.1.1 update-source Loopback0

!

end

New Member

Re: BGP Multihoming design topology

Hi Cam

pls can share with us this great white papers,  i m interesting, 5 Points from me

thanks

jamil

New Member

Hi,I have configured EBGP

Hi,

I have configured EBGP between to Internet service provider and configured HSRP with IBGP for High-Availability for Local traffic. In this Network Topology, I have two Firewalls behind Two Routers where I configured EBPG Multi-homing.

 

ISP01                                 ISP02

     EBGP                             EBGP

Router01         IBGP     Router02

                    HSRP

Firewall01           Firewall02

 

I have one global network block 105.X.X.X/23. My intention is to use both ISP active for 105.X.X.X/24 and 105.X.Y.X/24 while configuring EBGP Multi-homing with HSRP protocol. That means 105.X.X.X/24 will use one ISP for incoming and outgoing traffic and 105.X.Y.X/24 will use other ISP and fail-over happens in case of one ISP goes down.

 

Can you tell me how I can configure to achieve this Active/Active High-Availability configuring BGP with HSRP protocol and keeping Firewall behind for LAN Network.

 

I am looking forward to your assistance.

 

With Regards

Erfan

New Member

BGP Multihoming design topology

New Member

Re: BGP Multihoming design topology

Hi Cam

Thanks a lot cam, VERY VERY Informative Docs,thank y mate, another 5 pts

thanks

jamil

New Member

Re: BGP Multihoming design topology

Hi Cam,

I've actually just done very similar for our DC here in the UK, and it's live so I can give you hopefully a bit of assistance. My config actually looks very similar to your in terms of filtering and BGP config.

A couple of questions:

1) Your to BGP edge routers how will they be connected (physically) to the ASA, I'm guessing from your switch model via a VLAN?

2) You've not tweaked local-preference so I'm guessing you want some element of outbound load-balancing, based on the fact each border router will have two copies of the BGP table (1 via it's direct BGP partner, 2nd via the iBGP peer).

Can the kit handle the in excess of 700k routes / path entries that will be in the FIB / RIB?

Any questions you have for me, feel free to fire away as well.

Thanks,

Dunc.

New Member

BGP Multihoming design topology

Duncan I'm glad you responded and have a similiar topology!

1.) The BGP edge routers will be connected to the L3 3750 switch, the ASA outside interface is also connected to this switch. I'm planning on redistributing default routes from both BGP routers to OSPF which is running on the ASA.. (bgp is not supported on ASA currently)

2.) Correct, we deffinetely want outbound load balancing on these two circuits. However, you raise a good point, on each router we would have two copies of the BGP table.. How do I find out for sure if my ASR1001's will support this? If there is a hardware limitation we will have to re-evalutate and go with carrier/partial routes plus a default from both Carriers, to limit the size of the RIB.

-----------------------

Router#show memory summary

                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)

Processor  7F05B1231010   1217482752   262788048   954694704   954652872   954573868

lsmpi_io  7F05B0B2C1A8     6295128     6294212         916         916         916

------------------------

Dunc - did you decide to take full or partial routes from your carrier?

Thanks,

Cam

New Member

BGP Multihoming design topology

Hi Cam,

1) Are you going to use L3 interfaces to connect to the BGP routers?

2) Makes sense, it's worth noting that outbound load-balancing with two BGP tables isn't very finely controlled, like everything with BGP it depends who's better connected. (might be a bit biased)..

I've tried to look into the ASR1001 IPv4 prefix support but get mixed results, bottom end being 500,000 top-end being about 1 million...

Partial routes could be an option if it is 500,000, partial normally works quite well depending on what partial is (usually geographic (US in your case) + carrier customer routes)

You will end up with two copies if you run iBGP, the iBGP peers will exchange there BGP databases (those downloaded from the respective carriers).

Oh for noteworthy reference, your upstream carrier is AT&T. Also the 12.231.69.0/24 prefix falls as part of a larget AT&T aggregate, do you have an agreement from them to announce this to your other carrier?

I did take full routes from both providers in our case, I specced the routers specifically for this purpose, pair of 7206VXR's with NPE-G2 (2GB of RAM), supports around 3M IPv4 routes per router.

Any more questions let me know.

Time for sleep now (UK GMT), will check again in about 8 hours!

Dunc.

New Member

BGP Multihoming design topology

Hey,

1.) I will be going directly connected interfaces from DC-Edge-RT1 to DC-Edge-RT2. Since this is my iBGP connection between the two ASR's, can I use a private /30?

Also, for the ASR connections to the L3 switch, I will use the 12.231.69.0/24 space. I believe this is a requirement since my Natting takes places on my ASA, and the two OSPF default routes will be 12.231.69.2 & 69.3.

2.) I have tried to pull up the same information about the ASR1001 and full routes, I get the same mixed reports, anywhere from 500k to 1 million. This is one of my biggest hang ups because It will determine if I get partial/full routes from carriers.

We're in the process of setting up a call with at&t to go over this. I guess i'll just ask them for this agreement and that we want to anounce this IP space to our other carrier.

When you cut over to this topology, is there a way to verify that your ASN is listed for your prefix on a public lookinglass NAP? i.e. http://www.bgp4.as/looking-glasses

Thanks,Cam

New Member

Re: BGP Multihoming design topology

Hi Cam,

There is one problem with using 2 x /30's between your edge routers and the 3750 -> ASA, when your iBGP session advertises routes to the other peer they will be blackholed / unreachable.

For example say DC-Edge-RT1 (12.231.69.2)  advertises a route for 208.67.0.0/16 to DC-Edge-RT2 via iBGP, DC-Edge-RT2 will know that 208.67.0.0/16 is via 12.231.69.2. To forward a packet to 208.67.0.0 DC-Edge-RT2 will lookup the route to 12.231.69.2 and will find a route via OSPF which is via the 3750.

DC-Edge-RT2 will then spit out a packet destined for 208.67.0.0/16 out the interface to the downstream 3750, this is where the problem occurs, the 3750 doesn't have a clue how to get to 208.67.0.0/16 because unlike the BGP routers it doesn't have the full table. Well it would use the default routes but you have two of those so that's undesirable.

A solution could be (same one I used):

Setup a HSRP group between your two edge routers so they share a virtual address say: 12.231.69.2/29, give RT1 12.231.69.3 for it's physical address and RT2 12.231.69.4 for it's physical address. Then create a Vlan on your 3750 with a Layer 3 SVI with an IP of 12.231.69.5.

Just need one default route then: 0.0.0.0 0.0.0.0 12.231.69.2

iBGP will still do some amount of load-sharing depending on the information in the various BGP tables...

If and when you get this up and running you can view the BGP path (and thus your AS Number) from the following LG:

http://lg.he.net/

Do a bgp route query for your prefix, will show the AS number and AS-PATH

Any questions, holla! 

Oh, just a thought, if there's no requirement for your 3750 to be Layer 3, then you could just create the same HSRP group but put both your Edge routers & your ASA in the same Layer 2 VLAN with no SVI and then set your HSRP group as the Default Gateway for your ASA's WAN interface.....

Dunc.

New Member

Re: BGP Multihoming design topology

HI Cam

pls can y share ur configuration between DC-Edge-RT1 DC-Edge-RT2 .ASA and the 3750

thanks

jamil

New Member

BGP Multihoming design topology

Hey Dunc,

Your example makes sense. Per the HSRP recommendation, do you have a configuration example for the active/active group setup? For some reason I always think of HSRP as a active/standby mechanism. This achieves the same purpose that I was going for. My objective is to avoid multiple static default routes on the ASA. If a router dies, say 69.2, the static does not get removed..Creating an unreachable path. 

However with your design tip with HSRP, I can keep the ASA with a single static route pointing to the HSRP virtual address. I guess GLBP could work as well.

There is no requirement for a L3 setup on the 3750.. Is there a benefit going L2? Thanks for the link to the LG site, I browsed around and that is powerful tool.

Ibrahim - I will post the configs when I get back to work tomorrow

Thanks,

Cameron

New Member

BGP Multihoming design topology

Hi Cam,

Sadly HSRP will always be active / passive but in this istance the only reason it's used is so you only need a single default gateway / default route.

Unfortunately GLBP would also work in an active passive form, as you're ASA only has a single MAC address it would use the same AVF all the time.

That said though, even with HSRP you'll get a variety of active passive setup for example:

Say you had a packet destined for 210.50.0.0/16 and RT2 had the best route in BGP terms and RT1 was the Active HSRP.

Your ASA would forward the packet to RT1 (HSRP Active), RT1 would then look up 210.50.0.0/16 and as long as iBGP is configured correctly (local-preference) it will then forward the packet via RT2.

It isn't as cut and dry as Active/Active so to speak, but it is a form of load-sharing in outbound direction (depending on how well-connected each BGP Peer is, if one's better connected you might get an unequal bias).

Hope this helps.


Thanks,

New Member

Re: BGP Multihoming design topology

Hi Cam

pls can y share ur configuration between DC-Edge-RT1 DC-Edge-RT2 .ASA and the 3750, i m going with similar setup but with 2 ASAs

thanks

jamil

New Member

BGP Multihoming design topology

Hey Ibrahim, the configuration is almost an exact replica of the document I shared earlier.. The only difference is I grabbed the most updated Bogons from this site: http://www.team-cymru.org/Services/Bogons/http.html

Duncan - I ended up going with two 3750 switches in a stack configured, stricly Layer2 like you said.. There was no requirement to have them L3, basically plug-n-play with some obvious security hardening and port speed/duplex settings.

I didn't have to change anything on my firewall since its the same default route -> .1 (HSRP active router) - DC-iNet-RT1 -> iBGP to DC-iNet-RT2 (HSRP standby router). I'm tracking the gig0/0/0 interface on RT1 to ATT, if it fails, RT2 becomes primary.

I called ATT and setup carrier routes plus a default (approx. 56k) routes.. Everything is working fine. Thanks everyone for your help on this thread. If you have any questions, let me know.

New Member

BGP Multihoming design topology

We have a multihomed WAN  connectivity, the primary link is give the priority as it has 100 Mbps  and we wish to failover to secondary (30 Mbps) only when primary is  down. Attached the diagram for reference.

Previous  to the BGP link we had a static connectivity to single ISP(ISP1) and  this used to be the default routing path. Now for redundancy purpose we  have changed the default path to go via the bgp network.

After  changing the default route to BGP we have noticed that the download  speed dropped down to 20 kb/s, also the outside interface usage not  crossing 3 Mb/s. Later for testing purpose we have shutdown the  secondary ISP interface (int g0/2) and suddenly the outside interface  bandwidth gone up to 50 % of its capacity and also the download speed  increased to the normal values.

We  are continuiing with the secondary interface shutdown and planning to  manually bring this interface if primary goes down. Any idea why the  bandwidth is dropping when both ISP interfaces are active. Below are the  BGP configurations in the WAN router, please help me to fix this issue  because i am clueless about the problem.

!

interface Loopback0

ip address 2.2.2.2 255.255.255.255

!

interface GigabitEthernet0/1

description ISP-1 OUTSIDE

ip address 141.41.176.26 255.255.255.252

load-interval 30

duplex full

speed 100

!

interface GigabitEthernet0/2

description ISP-2 OUTSIDE

ip address 180.51.19.10 255.255.255.252

load-interval 30

shutdown

duplex full

speed 1000

!

interface GigabitEthernet0/1/0

description Static INT to FIREWALL

switchport access vlan 141

no ip address

duplex full

speed 100

!

interface GigabitEthernet0/1/1

description BGP INT to FIREWALL

switchport access vlan 102

no ip address

duplex full

speed 100

!

interface GigabitEthernet0/1/2

description DNS

switchport access vlan 102

no ip address

!

interface Vlan102

description ###BGP WAN Pool - VLAN ###

ip address 102.15.150.1 255.255.255.224

!

interface Vlan141

description ISP-1 WAN Pool VLAN

ip address 141.41.176.125 255.255.255.224

!

router bgp 123456

bgp log-neighbor-changes

network 102.15.150.0 mask 255.255.255.0

neighbor 180.51.19.10 remote-as 12121

neighbor 180.51.19.10 description isp2

neighbor 180.51.19.10 soft-reconfiguration inbound

neighbor 180.51.19.10 route-map isp2 out

neighbor 141.41.176.26 remote-as 45450

neighbor 141.41.176.26 description isp1

neighbor 141.41.176.26 soft-reconfiguration inbound

neighbor 141.41.176.26 route-map isp1 out

maximum-paths 2

!

ip route 102.15.150.0 255.255.255.0 Null0 220

!

ip prefix-list isp1 seq 5 permit 102.15.150.0/24

!

ip prefix-list isp2 seq 5 permit 102.15.150.0/24

access-list 10 permit 102.15.150.0 0.0.0.255

!

route-map isp1 permit 10

match ip address prefix-list isp1

!

route-map isp2 permit 20

match ip address 10

set as-path prepend 123456 123456

!

New Member

 Hi,I have configured EBGP

 

Hi,

I have configured EBGP between to Internet service provider and configured HSRP with IBGP for High-Availability for Local traffic. In this Network Topology, I have two Firewalls behind Two Routers where I configured EBPG Multi-homing.

 

ISP01                                 ISP02

     EBGP                             EBGP

Router01         IBGP     Router02

                    HSRP

Firewall01           Firewall02

 

I have one global network block 105.X.X.X/23. My intention is to use both ISP active for 105.X.X.X/24 and 105.X.Y.X/24 while configuring EBGP Multi-homing with HSRP protocol. That means 105.X.X.X/24 will use one ISP for incoming and outgoing traffic and 105.X.Y.X/24 will use other ISP and fail-over happens in case of one ISP goes down.

 

Can you tell me how I can configure to achieve this Active/Active High-Availability configuring BGP with HSRP protocol and keeping Firewall behind for LAN Network.

 

I am looking forward to your assistance.

 

With Regards

Erfan

14892
Views
11
Helpful
17
Replies
CreatePlease to create content