Designing the network edge

Answered Question
Feb 17th, 2009

Hi guys,

I wonder if some of you gurus might help me. I am in the process of designing the edge of our network which connects to 5 other sites. We have purchased 2 6500 switches and to this is connected the LES connections to the other sites, the firewall and other such edge devices.

Would you advice routed p2p links with the core devices or hsrp/glbp etc. I want complete resilience but I'm not too concerned with load sharing as there does not appear to be too much traffic traversing the link.

Currently the routing protocol in use is ospf and the core devices are currently connected directly to another switch which has the LES connections to the other sites. This will be decommisioned once the edge switches are in live.

The core switches and all the core switches in the 5 sister sites are all in area 0 so all routes are seen by all the 6 sites without any further effort.

My questions here are:

1. Should all the sites be in are 0 or should they be in separate areas? Please give reasons.

2. I now need the edge devices to run ospf as well, should I just put this in area 0 too or should the core and edge be in separate areas, please advice as to how to correctly design the ospf.

3. Any tips on designing the edge and how it should connect to/work with the core.

There are currently quite a few vlans in the core but I want to change these to routed links with the distribution switches and the server farm, any pointers.

I need the two edge switches to be in a completely resilient design with all edge devices connecting to them both and automatic failover, any ideas on what to do, should i implement pbr? all the cabling is in place i just need to configure the devices.

I know all this is probably as clear as mud but please ask me any questions, I appreciate the time you've taken to read this long post.

Thanks,

Martha.

I have this problem too.
0 votes
Correct Answer by Giuseppe Larosa about 7 years 9 months ago

Hello Martha,

in an OSPF multi-area domain area 0 the backbone area must be at the center with other areas connected to it.

In your scenario is a natural choice to keep the WAN links in area 0.

Each site can then have its own area, to be used between core routers and distribution.

To be noted that all areas must connect to area 0 and that area 0 must be contiguous (not partitioned).

Taking in account these protocol constraints from the point of view of OSPF:

the edge routers connecting to other sites can be in area 0.

the core routers of each site have links in area 0 to edge routers.

client vlans and distribution can easily fit in a non zero area.

so core routers can be an ABR.

see this OSPF design guide

http://www.cisco.com/en/US/tech/tk365/technologies_white_paper09186a0080094e9e.shtml

Config suggestion:

to be multi-area ready use network area commands as specific as possible.

In this way in a later time you will be able to activate new areas with minor changes.

The reason is that network ... area commands have a logic similar to an ACL and the order counts.

example:

instead of putting a single network command like

net 10.0.0.0 0.255.255.255 area 0

put

10.10.10.0 0.0.0.255 area 0

10.100.200.0 0.0.0.255 area 0

so if in a second time you want to move net 10.100.200.0 in area 11 you can easily

no 10.100.200.0 0.0.0.255 area 0

10.100.200.0 0.0.0.255 area 11

Note about wan connections:

I hope the other switch is not only one : I mean don't terminate all the WAN links in a single device or your redundancy is just nominal:

if that device fails the site is isolated even if the edge switches are two.

if there is a single device terminating WAN links I recommend to migrate these links half on new edge1 and half on new edge2.

Don't be afraid to ask for a maintanance window for doing this if necessary.

Hope to help

Giuseppe

Correct Answer by Giuseppe Larosa about 7 years 9 months ago

Hello Martha,

LES shuld mean leased lines.

1)

if you are comfortable with OSPF in area 0 only adding two devices don't change the picture.

In the future you may want to implement an OSPF multi-area domain for scalability and for getting a finer control on routing (route filtering is possible only at area borders).

In that case the core devices can become the ABR between area 0 used on the WAN links and the area used in each site (one area in each site)

2) yes the new edge routers need to take part in OSPF area 0 see above for multi-area considerations

3) different connections schemas can be used: you can use 4 point-to-point routed links to avoid any STP interaction.

You don't need anything else then OSPF: automatic failover is provided by the routing protocol

about moving routing to distribution is a common practice.

This step if has to be done in all 6 sites can be a trigger for a multi-area OSPF implementation or not all depends in the size of the sites.

Hope to help

Giuseppe

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Paolo Bevilacqua Wed, 02/18/2009 - 00:09

Hi, go ahead and use a single area for your design. area split designs are recommended for large wan or when you have many many routes.

Do not introduce further complications, go ahead and release the network. You will have plenty of time to improve it based on real issues.

mfawehin Wed, 02/18/2009 - 04:17

Hello P.B,

Thanks for your response.

You are right, I desperately want to avoid adding further complications so will stick to using a single area for now.

I was trying to design a scalable solution but dont want to have to break the existing ospf config on all the other sites that work by making changes there.

Cheers,

Martha.

Correct Answer
Giuseppe Larosa Wed, 02/18/2009 - 00:18

Hello Martha,

LES shuld mean leased lines.

1)

if you are comfortable with OSPF in area 0 only adding two devices don't change the picture.

In the future you may want to implement an OSPF multi-area domain for scalability and for getting a finer control on routing (route filtering is possible only at area borders).

In that case the core devices can become the ABR between area 0 used on the WAN links and the area used in each site (one area in each site)

2) yes the new edge routers need to take part in OSPF area 0 see above for multi-area considerations

3) different connections schemas can be used: you can use 4 point-to-point routed links to avoid any STP interaction.

You don't need anything else then OSPF: automatic failover is provided by the routing protocol

about moving routing to distribution is a common practice.

This step if has to be done in all 6 sites can be a trigger for a multi-area OSPF implementation or not all depends in the size of the sites.

Hope to help

Giuseppe

mfawehin Wed, 02/18/2009 - 04:11

Hi Giuseppe,

Thanks so much for taking the time to respond to me.

As advised, I will keep the edge devices in area 0 for now and consider multi-area ospf at a later stage.

Cheers,

Martha

mfawehin Wed, 02/18/2009 - 04:32

Sorry Giuseppe,

one quick question regarding what you wrote about implementing a multi-area domain. Currently all the distribution switches, the core devices and the core devices at all the other 5 sites are in Area 0.

If I wanted to make the edge switches and all the other site core devices (WAN links) into a different area and make the core an ABR so both areas can be seen, could I do that?

Please give some pointers as to how I can configure this.

I think I might just stick to the single area but I need to convince my boss that this is the best solution for now or at least show how we can alternatively configure multi-area domains but explain taht we should probably do the less complicated implementation for now.

I appreciate your help.

Regarding the use of 4 point-to-point routed links to the sister sites (if thats what you meant), that is not possible as the inter-site connections go into a separate switch and there is a trunk port connecting this to the edge switch.

Much obliged,

Martha.

Correct Answer
Giuseppe Larosa Wed, 02/18/2009 - 04:52

Hello Martha,

in an OSPF multi-area domain area 0 the backbone area must be at the center with other areas connected to it.

In your scenario is a natural choice to keep the WAN links in area 0.

Each site can then have its own area, to be used between core routers and distribution.

To be noted that all areas must connect to area 0 and that area 0 must be contiguous (not partitioned).

Taking in account these protocol constraints from the point of view of OSPF:

the edge routers connecting to other sites can be in area 0.

the core routers of each site have links in area 0 to edge routers.

client vlans and distribution can easily fit in a non zero area.

so core routers can be an ABR.

see this OSPF design guide

http://www.cisco.com/en/US/tech/tk365/technologies_white_paper09186a0080094e9e.shtml

Config suggestion:

to be multi-area ready use network area commands as specific as possible.

In this way in a later time you will be able to activate new areas with minor changes.

The reason is that network ... area commands have a logic similar to an ACL and the order counts.

example:

instead of putting a single network command like

net 10.0.0.0 0.255.255.255 area 0

put

10.10.10.0 0.0.0.255 area 0

10.100.200.0 0.0.0.255 area 0

so if in a second time you want to move net 10.100.200.0 in area 11 you can easily

no 10.100.200.0 0.0.0.255 area 0

10.100.200.0 0.0.0.255 area 11

Note about wan connections:

I hope the other switch is not only one : I mean don't terminate all the WAN links in a single device or your redundancy is just nominal:

if that device fails the site is isolated even if the edge switches are two.

if there is a single device terminating WAN links I recommend to migrate these links half on new edge1 and half on new edge2.

Don't be afraid to ask for a maintanance window for doing this if necessary.

Hope to help

Giuseppe

mfawehin Wed, 02/18/2009 - 12:37

Hi Giuseppe,

You are absolutely fantastic. I really appreciate you taking the time to answer all these questions.

It is really difficult to get downtime so I will try to make as few changes as possible.

The idea of putting all the site edge switches (WAN links) in area 0 appeals to me but i am not in a position to change any but the main site at this point.

I will add the additional area to the core devices so they can be easily changed over when I get to that stage.

I will also read the document on the link you sent.

At the moment, there is only a sole switch for the intersite connections but I cant see anyway around that.

Again, many thanks.

Martha.

mfawehin Thu, 02/19/2009 - 00:08

Hi Giuseppe,

This should be the last email on this subject, honest :)

Just to be sure I understand your advice, please review how I have currently scripted my changes.

on the Core router

(recall this is currently in Area 0 and has connection to the 5 other sites whose networks are also currently defined in Area 0. The 6 distribution blocks are defined as stub areas in separate areas 100-105)

router ospf 1

log-adjacency-changes

area 100 stub

area 101 stub

............. (all other DB's)

network local loopbk int 0.0.0.0 area 0

network other local int 0.0.0.0 area 0

network ext site 1 0.0.0.0 area 0

network ext site 2 0.0.0.0 area 0

network (other ext sites) 0.0.0.0 area 0

network DB1 0.0.0.3 area 100

network DB2 0.0.0.3 area 101

.............. (all other DB's)

Now I need to add another area to this core device and make it the ABR with links in both area's??

As the core routers are still part of area 0, presumably the distribution blocks are still fine and I have not formed any contiguous networks??

Please let me know if I have understood correctly, I would as advised like the core routers at every site to eventually be in a different area and to have the edge devices at all sites in Area 0 but as we have designed the DB's in separate areas, I hope this does not complicate things.

Many thanks,

Martha.

Giuseppe Larosa Thu, 02/19/2009 - 00:24

Hello Martha,

>> Now I need to add another area to this core device and make it the ABR with links in both area's??

Actually, a router becomes an ABR when it has multiple network .. area commands specifying two different areas.

your core is already:

ABR between area 0 and area 100

ABR between area 0 and area 101

and so on.

You don't need to do else in OSPF.

note:

the most important action even before using OSPF multi-area is to remove the single point of failure represented by that switch terminating all links to other sites.

This is really important and needs to be done to make a good job.

So I would introduce a section on migrating these links to new edge nodes

It may take two/four hours of night time to do that but without this the design is not fault-tolerant.

Hope to help

Giuseppe

mfawehin Thu, 02/19/2009 - 01:07

Thanks again Giuseppr, I'll work on getting them to fork out for an additional switch.

Would you advice changing the distribution blocks so they are all part of the same area e.g change all to 100(as I mentioned, there are 6 DB's, each connected to about 4 or 5 switches per cabinet) and the have the core as an ABR for this this single amalgamated area plus the backbone - area 0?

Each cabinet does not need to know about the other and that is whay I have made them stub areas. I need to ensure any change I make does not mess things up.

Cheers again.

Giuseppe Larosa Thu, 02/19/2009 - 02:51

Hello Martha,

>> Would you advice changing the distribution blocks so they are all part of the same area e.g change all to 100

yes, this is reasonable otherwise the work for the ABR is 6 times worse:

it is common to use a single stub area for a site.

To be noted that even if you filter inter-area routes using totally stub areas the default route allows communication between the different cabinets.

you may want to use a single totally stub area to keep area 100 database size to a minimum.

in this case you need

area 100 stub no-summary on ABR nodes ony

the distribution switches need

area 100 stub

only

in the future you can deploy other stub areas in the other 5 sites.

Hope to help

Giuseppe

gesadmin1 Thu, 02/19/2009 - 09:22

I apologize for intruding on this conversation, but I am wondering one thing. If the OP had connections to the Internet from each independent location, making the other areas stubs would not be a wise choice am I right?? The reason I ask this is the default route that gets installed on the stub routers would forward all traffic back over and through the backbone when it may actually need to go out via the Internet connection. This is more of a hypothetical design question more so than a question related directly to this issue. Please tell me if I'm understanding this correctly. Thanks.

Mohamed Sobair Thu, 02/19/2009 - 09:41

Hi,

Cisco Recommended the OSPF design to be as follows:

1- The Distribution layer should be in (Area-0), and all Sites Blocks connected to ARE-0 should be in different Areas.

For example , if you have 3 Block/ Sites, recommended every one to be in seperate Area 1,2,3 and all has to connect to AREA0 (Backbone Area).

why? because each OSPF AREA maintains its own link state database, this will reduce number OF LSAs , Number OF SPT , And the less of CPU usage, Only the Boarder router would have to be capable enough interme of performance.

The Second answer, Cisco recommends Applying all Summarizations and filtering at the Distribution Layer (Area 0), This will maintain a small amount of routes at the access layer , therfore , the routing table is reduced and the least CPU consumption.

3rd Answer, Cisco highly recommends complete Layer-3 Design even in the Access layer, Transfering All layer-3 in the Access layer would speed convergence and will rapidly increse failure detection.

HTH

Mohamed

Actions

This Discussion