OSPF WAN with 200+ routers in area 0 - to BGP or not to BGP

[aronsmith]

We have a fair size wan spanning two datacenters (east cost and west coast) and have recently moved to ATM IMA to Frame interworking with 7204s at the core and 2600s at most client sites, some 1700s. Port speeds of client-side frame range from 56k to full T1s, some with multiple (up to 7) connections from various branch locations that are completely intermeshed.

I want to kill OSPF off the 56k lines, but don't want to lose quick convergence. Currently there is no problem, except convergence on the slow lines takes too long. None of the remote points have to talk to eachother via the core WAN routers, just to the datacenter. I want to keep static routes off the core routers and let remote sites advertize into the core routers the branches they connect via redist static subnets...

Should I use BGP or will distribute-lists on the core be better? Will a distribute-list out really shrink the tables at the remote sites? BGP will work of course but it may not be worth the additional complexity.

Thoughts?

[rais]

Do you have 200+ routers in area 0 or 200+ routes? Do you have any other areas other than backbone?

Regards.

[aronsmith]

I have 200+ routers in area 0... rather I inherited them. For our largest clients with multiple branches and multiple connections to the core, I built separate areas, the ASBR being at the remote site. There are about 15 areas other than area 0.

My ultimate goal here is twofold:

1) protect the core from stupid link flapping at remote sites I don't care about

2) protect the most feeble WAN segments from routes they don't need.

There are about 500 routes in the table. The network is fairly stable so even with 200+ routers in area 0 performance has not been a serious problem but we're growing and it will be a problem someday.

[vcjones]

My preference would be to stay with OSPF and do a better job of defining your area zero, moving systems which are not critical to backbone connectivity into other areas. Of course, at this point, it is probably too late to do a good job of assigning IP addresses so they can be summarized efficiently at area boundaries, but that should be your goal.

You could use BGP, but the impact on route selection and recovery would be significant compared to staying a single routing domain. But you should consider how you would do BGP, and use the few BGP speakers you would have as the starting point for your new area zero. Your problem is that you have an out of control OSPF area zero due to a lack of discipline in keeping the backbone network reserved for backbone routers and links, and you should shrink the backbone back to a manageable size rather than split it up with BGP.

Good luck and have fun!

Vincent C Jones

www.networkingunlimited.com

[aronsmith]

You're exactly right, it is a totally out of control area 0, and the ip address assignment is way too late to fix, not to mention that even if it was perfect it wouldn't help as out customers sometimes switch datacenters and reassigning a couple hundred PCs is not an option when that happens. My only excuse is that I was brought in after it was built with 3Com routers... I'm replacing with Cisco/ATM and trying to clean it up as I kill off the Frame T1s bit by bit and move to IMA/Frame interworking.

I've thought about possibly switching to EIGRP but I feel like I'd lose some of the more refined controls I have with OSPF not to mention the huge undertaking that would be.

Would a distribute-list out keep the LSAs from reaching remote routers or traversing the links in OSPF?

[rais]

No. Distribute-list....wont work for link states.

Stubs may help you. Statics seem promising too.

Thanks.

[nurinalmas]

From the discussions, I would probably be having the same problem. Need advice on how best to avoid it.

Central site has 200+ leased line links to remote offices. Each link uses a pair of routers. End up with 200+ routers at the datacenter.

How best to design the OSPF areas, especially area 0, considering the large number of routers at the central site?

[vcjones]

The easiest way to do a large hub and spokes network is to use a simple distance-vector protocol between each spoke and the hubs, and save the fancy routing protocol for between the routers/sites making up the hub. That way, you can use lots of filtering to minimize routing overhead between the hub and the spokes (but still avoid getting inundated with static routes or sending traffic down broken pipes) while getting the advantage of full powered routing inside the hub where you need it.

This can be done using EIGRP everywhere (if a Cisco-only network and you're willing to spend the time to design the configuration to get the desired feasible successors for critical routes) or by using OSPF only within the core and using RIPv2 (redistributed into OSPF) for links to spokes. Both approaches are discussed in Chapter 7 of my book High Availability Networking with Cisco, and example configurations are on me web site. You should also be able to do the job using OSPF everywhere and making each spoke a stub area, but I have not tried that with more than a few hundred spoke sites. Perhaps someone else will chime in with their experience and recommendations.

Good luck and have fun!

Vincent C Jones

www.networkingunlimited.com

[nurinalmas]

Will it be the same consideration if the setup is like this?

Backup Hub Site do not have aggregation routers accepting the 200+ redundant connections from the branch offices. So I end up having a pair of routers for each spoke link. Now, I have 200+ routers at this Backup Hub Site.

The Main Hub Site is also accepting 200+ primary connections (leased circuits) from the Branch offices, but there is a pair of 7513 Aggregation routers.

Each spoke has a small LAN with 2 access routers, one router linking to the Main Hub Site, and another redundant router linking to the Backup Hub site.

[vcjones]

With 200+ routers at the backup hub site, this is not a hub and spoke network from the router point of view. There is no problem with two routers at each spoke, but the backup site is a full mesh of all spoke sites and will not scale. This will be clear if you draw out the logical links between routing neighbors.

You need to do some rethinking of the network topology. Keep in mind that what counts is what the routers consider the topology, not what the physical links look like. For example, you could add one or two aggregation routers at the backup site and treat each of the 200+ routers going to the spokes as part of the spoke network rather than part of the backup site network. Another approach would be to consider the backup hub site a big mux connecting the main hub to all the spokes. Either way, each of the 200+ routers would only neighbor with one or two aggregation routers, rather than all their peers at the backup hub site.

Two things to consider are what you want the final network to look like (and work like), and how to get from where you are to where you want to be (with minimal down time and risk).

Good luck and have fun!

Vincent C Jones

www.networkingunlimited.com