[BGP] - Two datacenter routes redundancy design with FW

[NicolasDemonty]

Hi all !

I hope you're well !

I'm currently trying to design a network to provide routes redundancy between two datacenters. Some points are forced. I'll explain the diagram.

• the two DC are geographically separated.
• Core1 models DC1 and Core2 models DC2.
• Each DC is protected from the outside by a firewall between CoreX - ASRX.
• The three SPx routers models the service provider network.
• The link between Core1 and Core2 is a leased line that is supposed foolproof.
• Each DC has its own public addresses range : let's say NET1 for Core1 and NET2 for Core2

The redundancy must be the following : when either ASRx or SPx fails then the DCx must be reached through the other DC and the leased line. For example, if ASR1 fails then the route " through SP ---> SP2 ---> ASR2 ---> Core2 ---(leased line)---> Core1" must be used.

So, because there are FW between CoreX and ASRX, I'm just wondering what technology I could use to provide route redundancy without any asymetrical or under-optimal routes.

For now I used :

• eBGP between ASRX and SPX : ASRX announces NETX to SPX and SPX announces the default routes to ASRX.
• iBGP between SPX so each router can be informed of the other routes.
• iBGP between ASX through the FW (TCP 179 and static routes)

What is blocking me is the two FW. Indeed, I use static default-route to the FW with different AD on CoreX switches to indicate how to go outside but if one ASR fails, because of the FW the static route does'nt fail and the traffic is forwarded to the FW ... The same problem occurs when SP1 fails : the incoming traffic flows to DC1 through SP2 and DC2 and then the return traffic will be forwarded to FW1 then to ASR1 which is aware of the return route through ASR2 (iBGP) but with the recursive table lookup there will be a loop ASR - FW or ASR-FW-ASR depending on the config.

So I need your help because I don't know how to make the route failure dectection become dynamic.

Have you any suggestions or idea ?

THank a lot in advance

[Jason M.]

Hello Nicolas,

Why not connect the leased lines between the ASR's? It it to avoid sending traffic between DC's through the firewall? In that case, if the leased line fails, is it configured to communicate through the ASR's anyway?

[NicolasDemonty]

Hello Jason,

yes, it is what I wanted to do at first but the leased line is forced to be connected to the core routers .

[Giuseppe Larosa]

Hello Nicolas,

if you cannot move the leased line between the ASR boxes, you will need to run an IGP between ASRs, FWs and core routers.

I would use OSPF as it is possible to use a route-map in default-originate command to generate a default route only if some criteria are matched ( for example a BGP default route is received from the expected eBGP next-hop).

I have successfully used this setup some years ago.

the route-map checks an IP address ( a route) and an IP next-hop  ( using an ACL that matches the BGP nexthop)

! check the syntax there may be some errors

route-map check-ebgp permit 10

match ip address 10

match ip next-hop 11

set type external-1

set metric 50

The default route has to be of type O E1 because there are two exit points in the OSPF domain.

The FWs have just to flood LSAs up and down.

DC core routers generate OSPF internal routes for NETx.

DC1 generates NET1 and DC2 generates NET2.

The leased line will have an high OSPF cost like 10000 on both ends to be used only when necessary.

In normal conditions, each column DCx-FWx-ASRx will use the vertical links to go out and traffic for NETx is received on the same column.

IF one ASR fails or the eBGP session of ASRx fails the following has to happen:

ASRx stops to generate an OSPF default route because route-map criteria are not matched anymore or because the device has failed.

DCx will foward traffic using the leased line as the only OSPF default route is that generated by ASRy, return traffic for Netx should follow the reverse path ASRy-FWy- DCy-DCx.

All this under the hyphotesis that Netx is only behind DCx as stated above,

Hope to help

Giuseppe

[NicolasDemonty]

Hi Giuseppe,

yes your answer helps me. It reinforces my idea to run iBGP between ASX - CoreX. With iBGP I don't need to run the IGP between CoreX and the FW. A FW failure will be detected by a breakdown in the TCP connection that passes through the FW.

Thx for your help

Nicolas