Core to Access Layers - routed or switched?

[rebecca.richards]

Hi Guys,

I had a philosophical discussion with a contractor the other day about his desire to use L3 routed links between the core and access. At present, we're using L2 VLAN trunks between the core and access.

Some concerns I have with his argument are:

* We're used to using L2 VLAN trunks

* The L2 design is fairly simple

* The end users are not "sensitive" enough to feel a failover of links from one core switch to another when a trunk fails (i.e. STP topology changes)

* The configuration of the access layer switches is fairly easy

What has made me think about L3 links between the core and access layers are the potential fast convergence (is OSPF convergence faster than STP at layer 2?), and that our current network design is approaching 3+ years old. The downside is that we would have to start using /31 blocks of IP addresses, rather than have unnumbered trunks.

One concern of mine is probably a bit dated now. We have VoIP being slowly rolled out (read over months...) within the building. My thought was that the VoIP traffic over L3 routed links would have some latency added, whereas L2 trunks is purely switched.

With new equipment going into the building as part of the rollout of VoIP, maybe my (dated) thought is moot - if we use CEF, would latency continue to be a factor?

What are your thoughts on L2 VLAN trunks vs L3 routed links between the core and access layers?

[lamav · Accepted solution]

Rebecca, I hope you're sitting down because this is gonna be long! :-)

Seriously, though, there is a lot that can be said about this, so let me start by saying that the trend is to migrate to a routed access layer, especially if you have a collapsed backbone, which it seems you have, since you make no mention of a distribution layer.

The reason for migrating to a routed access layer is to enjoy the benfits of L3 isolation and minimizing the reach of the switched domain. This minimizes the possibility of suffering from a L2 spanning-tree loop while maintaining fast convergence through L3 switching and deploying dynamic routing protocols, like OSPF and EIGRP, with adjusted timers and stub router configurations.

Moreover, in a collapsed core, it is definitely desirable to keep it a L3/routed layer and prevent the possibility that a spanning tree loop will actually adversely effect your core.

A routed access layer may not be feasible in a server farm because of L2 adjacency requirements.

There is a lot to take into consideration. Here are two excellent links that you will definitely benefit from:

http://www.cisco.com/application/pdf/en/us/guest/netsol/ns107/c649/ccmigration_09186a008073377d.pdf

http://www.cisco.com/application/pdf/en/us/guest/netsol/ns432/c649/ccmigration_09186a00805fccbf.pdf

HTH

Victor

[Jon Marshall]

Rebecca

Just to add to Victor's points.

You would presumably connect your access-layer switches with redundant links to your core/distro layer. As such OSPF/EIGRP should see 2 equal cost paths to the core. So it is not really a question of faster convergence because if a link fails the routing protocol will automatically transfer all traffic over the other link.

The L2 design is fairly simple but L2 can be quite difficult to troubleshoot compared to L3 IMHO.

VOIP - ironically this was what convinced us to move to a routed L3 access-layer as voice is very delay sensitive. Admittedly we were influenced by the use of Nortel phones and Nortel do not use spanning-tree on their switches so we decided to eliminate STP from the uplinks. The latency is absolutely minimal as L3 switches route packets in hardware.

Your QOS design would need to be adjusted slightly as you are now running over L3 links and hence can use DSCP markings from the access-layer.

Whichever one you choose will not be a "bad" decision and you need to be able to support it once it is in. We run a mixture of L3 routed access-layer and L2 switched access-layer in our major sites and both work well. All things being equal i would go with L3 but that may not meet all your requirements.

The key limiting factor to be aware of with L3 routed access-layer is that you cannot extend a vlan across access-layer switches. So if your building has 3 floors and you need the same vlan to be present on the switches on all 3 floors L3 would present you with problems.

Jon

[Joseph W. Doherty]

As Jon notes, either will work. However, I sometimes wonder if there's thinking now that we can means we should. (I'm concerned about ". . . our current network design is approaching 3+ years old.")

As nice as having L3 switches at the edge might (or not) be, don't forget then tend to cost more than L2 switches, both for the hardware and sometimes for a routing feature license. (Maintenance for them is often higher too.)

Many years ago, a mainframe systems programmer told me, when the hardware vendor makes recommendations, never forget what the business of the hardware vendor is. (I.e. Hopefully the contractor that you had this "philosophical discussion" has no interest in new hardware or implementation/supporting same.)

Your question of convergence latency between L2 and L3 is germane. Assuming L3 is faster, is the decreased convergence time really needed? How often has there been a hardware failure that caused convergence? How many users would be interrupted and what's the value to the organization to avoid those interruptions. (Often not all users require the same performance level, e.g. no VoIP drops on sales lines; support lines, perhaps some VoIP drops might be allowed.) What's it going to cost to meet service levels, using L2 vs. L3? Etc.

With regard to your question about latency, L3 switching, usually, is as fast as L2 switching.

Don't think I believe L3 at the edge is bad, just it needs to be rationally analyzed against business requirements. You can also mix both, if that's the best solution for you.

[rebecca.richards]

Awesome and very valid comments guys! Thank you very much!

I didn't think about the cost factor of L3 switches v L2 switches, plus maintenance. I was looking more at the benefits and complexities of future support - we don't have many engineers, and we're all sort of learning as we go - thus the KISS principle.

In the 4 years that we've had L2 trunked circuits within the building, there's only been one real instance where we were left scratching our heads. A link from a floor to the core died, and the users on the floor were complaining that the network was very slow. We investigated the failure, and the slowness, and discovered that the failure was caused by a fibre patch cord break, which caused STP to converge across to the redundant circuit, which just so happened to pass through a faulty GBIC (lots of errors across the circuit).

The two PDF's linked to within Victor's posts were very good, and showed the plus/minuses to trunking v routing between the core and access.

... and yes, I was sitting down!

Cheers,

- Rebecca