Routed Ports / L3 SVI / Network Design

Unanswered Question
Mar 11th, 2008

I know the title says a lot but I am currently working on designing a new network with cisco's traditional 3 tier model except with the option of using the routed access model, i'm a bit confused on how I should approach it.


First off after looking at the diagram you will notice a bunch of trunk ports. The reason they are trunk ports and not routed ports is because each server will be dual homed. I'm not sure I have an option here to use the "routed access layer" because of this.


Secondly I am unsure of whether I should be using SVI or routed ports for the distribution to core connectivity. I believe routed ports is the way to go but I would greatly appreciate opinions and the reasoning behind them.


I would also appreciate anyone willing to comment on this portion of the design.


There are other pieces I didn't include in this diagram (i.e internet, extranet, vpn, etc)


Thanks to anyone willing to assist.



Attachment: 
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Joseph W. Doherty Tue, 03/11/2008 - 20:27

If your going have a L2 edge, and have multiple VLANs on the edge, trunks are fine unless you want to dedicate an uplink per VLAN.


If you run L3 on the edge, there would be no VLAN trunks because the uplinks will be L3, like distribution to core. VLAN will only be defined on the edge devices, distribution won't see them, it will see downstream 3750s as routers. (Also you won't need the L2 trunk between distribution routers.)


Since you diagram shows 3750s, you could consider stacking them. Can be stacked for either L2 or L3 edge.


If your distribution/core pairs are 6500s (with sup720c), and if this is a new design, you might consider VSS (which "stacks" a pair of 6500s).


On your question about SVI or routed ports between distribution and core, SVI would allow easy reassignment of the "routed" port to a physical port.


SVI would also allows shared subnets between distribution and core. Most usually only recommend logical p-2-p links. Avoids multicast flooding issues between routers. With routed ports, you can't have a shared subnet, which avoids this issue.

lamav Tue, 03/11/2008 - 20:59

Amo:


"The reason they are trunk ports and not routed ports is because each server will be dual homed. I'm not sure I have an option here to use the "routed access layer" because of this."


You're correct. With a routed access layer, the VLAN is confined to each switch, so you lose the option of L2 adjacency for dual-homing the servers. The routed access layer has some benefits, but it can pose problems in a data center. You can extend the VLAN across the access layer, but that sort of defeats the purpose of the routed access layer and minimizing the switched domain.


"Secondly I am unsure of whether I should be using SVI or routed ports for the distribution to core connectivity. I believe routed ports is the way to go but I would greatly appreciate opinions and the reasoning behind them."


Just to add what Joseph shared with us, I would recommend routed p-2-p uplinks between the distribution and core layers using routed ports, not SVIs. The practice of using SVIs to create a routed interface has its roots in the days when CatOS was the only show in town. Each switch port in a CatOS switch was an L2 port and the only way to simulate a routed connection was to put the port in a VLAN and create an SVI for that VLAN. With IOS, that is no longer necessary, and in fact SVIs can add a layer of complexity that is not necessary.


Lastly, if you ever decide to deploy data center services, like application optimization, firewalling with the FWSM, SSL offloading and load balancing with a CSS (HSRP, too, since you'll be running a switched access layer), you will need the L2 trunk between distribution layer switches to provide the L2 adjacency necessary to run the active/standby topology.


HTH


If so, kindly rate this post


Victor

Jon Marshall Wed, 03/12/2008 - 00:35

"You can extend the VLAN across the access layer, but that sort of defeats the purpose of the routed access layer and minimizing the switched domain"


That's a very interesting point actually. Made me wonder if there is any benefit to have a pair of switches in the access-layer connected via a Layer 2 trunk providing redundancy for the servers and then uplinking the access switches to the distribution with L3 routed links.


Benefits are that you get equal cost paths from the access-layer to the distro layer so no manually setting STP priorities / HSRP active gateways on distro switches.

Also you have kept the STP diameter down to 1 in the access-layer and if you run RSTP you have very quick failover.


Agree if you are looking to deploy service modules routed access-layer can be a hinderance.


Just thinking out loud..


Jon





Joseph W. Doherty Wed, 03/12/2008 - 05:04

Even, some more thinking out loud.


Victor is correct, with VLANs normally restricted to one L3 edge device, you lose the option for L2 adjacency across multiple edge device, but yet this shouldn't be an impediment for dual homing. Dual homing, at least to me, implies different subnets (also as seen in the OP's attachment). On the other hand, if you're thinking NIC teaming, then you have a problem.


Victor mentions that you can still extend the L2, at least easily with L3 switches. Yes you can. As to defeating the whole purpose of an L3 edge, maybe not.


If you maintain the classic L2 with a L3 edge, what you now have is routers on a shared segment(s). If you configure the gateway on the edge device, all routing between VLANs on the device can be done on the device. I.e. traffic need not flow via an uplink to jump edge VLANs. Traffic destined beyond the edge device will be routed to the distribution routers (must decide whether you'll permit redirection). Return traffic, from distribution, will not transit the edge router as L3.


The edge router doesn't even need to peer with the distribution routers, it can use default routes for the uplink paths. The distribution routers don't need to peer with the edge L3 to know the networks since you've extended the L2 to them.


Why bother? You still get most of the edge advantage of L3. The rest of the network topology remains one hop less. (Oh, and cough, your edge device might not need a more expensive feature license.)

amohabir1 Wed, 03/12/2008 - 05:14

Right dual homing does mean different subnets however, It is undetermined on whether they will use nic-teaming vs. dual homing. From what I hear this morning it will be nic teaming, so now I have no option but to use the switched access layer.


I also agree with your argument for the L3 Edge. I plan on having the edge device handling most of the traffic flows and routing here.


I agree with the edge/distro design as well.


In the edge we have the 4507 (because of cost) with dual sup5s 10GE.



Joseph W. Doherty Wed, 03/12/2008 - 05:27

If you're going to use 4507 with dual sup5-10GE, then on the same chassis you can extend the VLAN NIC teaming across blades and still do L3 on the edge. If you need to extend the VLAN and NIC teaming across devices, which I discourage because of unicast flooding issues with dual distribution, then you preclude L3 at the edge.

amohabir1 Wed, 03/12/2008 - 05:45

Well the access layer switches are the 3750's and the way it seems now is it will have to be across devices because they are not stacked.


I can most certainly stack them if need be and work use different ports on each 3750 for the teaming.


The only thing that cisco recommneds is to adjust the aging timers for any arp/mac entries.



Joseph W. Doherty Wed, 03/12/2008 - 05:57

If using 3750s, I would stack them (also increases the bandwidth between VLAN members).


With regard to arp/mac timers, if this is in reference to unicast flooding, if you don't span VLANs across multiple edge devices from dual uplink routers, it's not an issue.

amohabir1 Wed, 03/12/2008 - 05:59

Okay thats what I thought. I don't plan on spanning vlans across any edge devices. The server VLANS will be localized to the access layer with no need to span them across any other device.

amohabir1 Wed, 03/12/2008 - 06:00

Okay thats what I thought. I don't plan on spanning vlans across any edge devices. The server VLANS will be localized to the access layer with no need to span them across any other device.

amohabir1 Wed, 03/12/2008 - 05:05

Thanks Victor


The other question is can I use a L3 etherchannel between the distribution switches and still have the L2 adjacency for the access layer.

lamav Wed, 03/12/2008 - 07:11

Hi, Amo:


You're welcome.


One point before I answer your question: when I mentioned dual-homing earlier, I was speaking in generic terms. That's why I didn't capitalize the words Dual Homing. I was using it as a verb phrase, if you will. The concern I raised had to do with L2 adjacency.


And the answer to your question is yes, connecting your routed distribution layer switches, you will have both: an L3 inter-switch-link [not to be confused with Cisco Inter Switch Link -- there go those capitals again ;-)], which will support certian multicast environments, like MSDP (financials use this a lot), and route summarization -- and then you will also have the L2 etherchannel between routed distribution layer switches to extend the VLAN across the distro layer.


[edit] As far as defeating the whole purpose of the routed access layer by placing an L2 trunk between them, I said it "sort of" defeats the purpose WITH REGARD to "minimizing the switched domain." Read carefully. The main rationale for creating a routed access layer is to leverage L3 isolation and minimize and mitigate the occurence of an L2 loop and its potentially disastrous effects. Jon stated it perfectly.


Sorry, Im joining this discussion "late" -- had to work this morning. I gotta stop doing that...! ;-)


HTH


if so, kindly rate my posts


Victor

amohabir1 Wed, 03/12/2008 - 07:30

No problem.


I thank everyone for their suggestions and opinions and trust me it is making it a lot easier for me.


Feel free to join whenever. That's the point of these forum's. We can't always be on all the time.

lamav Wed, 03/12/2008 - 07:37

I'm glad to know we were able to help you.


Please feel free to come back with more questions/concerns.


Victor

amohabir1 Wed, 03/12/2008 - 07:38

Just to give you guys some insight to what I have gotten so far from a visual perspective here is the updated diagram. Please provide any critique you have as it is greatly appreciated.



Actions

This Discussion