new distrubution layer design

Unanswered Question
Jul 31st, 2008

Hi all, we are in process of designing a new distrubution layer for the servers etc, however we have 2 computer rooms. we want to use layer 3 between the distribution and core. But we want to keep the same ip subnets in both computer rooms for the servers etc. What would we do here? use the dist switches as a gatway for the servers, using hsrp,vrrp etc? then would we link the 2 dist switches direct together using layer 2 or 3, I would think layer 2 so we can have the same subnets still in both rooms? then would we use l3 ports to the core switch say on a /30 network and advertise the vlans etc from the dist switches?

can anyone tell me the best design for this?


I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 3 (1 ratings)
4rmorris Thu, 07/31/2008 - 06:50

I would use layer 3 connections between the distribution and core, as you described with /30 subnets. Then I would run a separate physical connection either:

1) Directly between the distribution switches

2) Between the core and distribution switches

And this connection would carry only layer 2 traffic (i.e. it would be an 802.1q trunk). With option 1, you need not configure the vlans on the core switches, but with option 2 you would have to configure the VLANs on the core switches to make sure they pass between the distribution switches.

With option 1, the default gateway would be the distribution switches (using HSRP or VRRP). With option 2, the default gateway would commonly be the core switches. If you have a requirement to keep only layer 3 to the core, than connection the distribution switches at layer 2 is the way to go.

Good luck,


carl_townshend Thu, 07/31/2008 - 07:04

is number 1 the best design here ?

what does the link in between the dist switches do, why do we need this ?

and whould I advertise the /30 subnet on each seperate switch ?

4rmorris Thu, 07/31/2008 - 08:13

I prefer #1 based on your requirements. It allows the core to focus on high speed routing and not worry about layer 2 (i.e. spanning tree) issues.

Because you want the same IP subnet in both server rooms, you need to span the VLAN that contains that subnet across the distribution switches. If you only have layer 3 connections to the core, than you need to add a layer 2 connection between the switches to carry the intra-vlan traffic.

The /30s are necessary to establish IP routing to the core. They could be carried in a summary outside the core. The important thing is that the core switches and the distribution switches have a routing relationship, so you can advertise the routes in the distribution up to the core.

Does this make sense?


Marwan ALshawi Thu, 07/31/2008 - 07:31

the best way for ur case as follow

as u said make the two dist layer as a gate way

u can use hsrp but if u can use GLBP it would be better because u gonna loadbalance the traffic

then the connection between the dist layer and core layer should be L3 only and the route from the dist to core layer should be summry route for more effitioncey and better performance because u gonna advertise summry route from the dist switches to the core any link fauilor between the access layer and dist layer will not be notced by the core

to solve this issue make the link between the two dist layer as layer 3 link

good luck

please, Rate if helpful

carl_townshend Thu, 07/31/2008 - 09:26

Hi, where am i summarizing, on the dist switch or the core switch? what would it look like if I use ospf?

ronbuchalski Thu, 07/31/2008 - 10:02

Carl, some additional questions that may help guide you to the best design:

1) Where do you currently have your L3 gateways for these server subnets?

2) Do you have a dist. layer today, or do you route at the core today?

3) If you make the dist layer the L3 gateway, then you can easily support both computer rooms having servers on the same IP subnets. Just be very careful with your design so that spanning tree won't be a problem. BPDUGUARD is a good feature to enable on the host ports of your access layer switch.

4) Suggestion was made to summarize routes at the dist layer, but before you go through the effort of designing this, ponder these questions:

A) Are you IP addresses allocated in such a way that they can be summarized?

B) How many IP subnets are you talking about? If it is a small number, you really don't gain much by summarizing. Keeping route flaps from the core is a good design goal, but if you're only talking about summarizing 8 subnets into one summary advert, I would say that the gain is marginal.

5) Regarding route summarization with OSPF, you need to define an area (or areas) for your access level subnets, and define Area 0 for your core, and then summarize for routes advertised across the border (your dist routers are your ABRs). See:

Hope this helps.


carl_townshend Tue, 08/05/2008 - 00:16

hi there, I have a question about the routing, If I have the layer 2 access, then layer 3 from the distribution layer, ie use hsrp for on the distribution routers, If I use a routing protocol, how will it route to the routers? will it load balance? as this maybe tricky as one of the gateways will be active, the other will not,

can anyone help me with this?


ronbuchalski Tue, 08/05/2008 - 05:24

Carl, the hosts on the access layer will be pointed to the gateway router address (say, As is standard in HSRP, you designate which router is active and which is standby, and the active router will handle the routing of these packets.

There is no load balancing on the inbound packets (from access layer hosts to HSRP routers). If you want inbound load balancing, you'll need to use GLBP:

However, both HSRP routers will be sending packets outbound to devices on the LAN.

Hope this helps.


carl_townshend Tue, 08/05/2008 - 06:20

Hi There

I am more interested in the routing between the core and dist layer switches, ie if I use a routing protocol etc, how will it work if I have 1 router active and standby in the dist layer, and will I have to advertise the same routes on both dist switches as they need to route the same.

please help

ronbuchalski Tue, 08/05/2008 - 06:45

Carl, routing between the core and dist. routers will work in the normal way, and is not affected by HSRP. If your two dist. routers are connected to two core routers, your network will show two equal cost paths between each core router and each dist. router, and traffic will be load balanced across the links.

As I said previously, HSRP only determines the gateway that hosts on the LAN will use to send packets to other subnets. The upstream from your dist. routers will be standard IP routing, and will include redundant paths for load balancing if they are present. Return traffic from the core to the access layer (via the dist. routers) will be load balanced to both dist. routers, and will be sent out from both dist. routers to the local LAN. There is no 'standby' HSRP for outbound packets from the dist. router to the access LANs. They will both route packets to this access LAN.

Hope this helps.


carl_townshend Tue, 08/05/2008 - 06:59

can you show me what this config would look like, im confused as when the core routes back to the dist, if there are equal cost routes, only one of the dist routers is active, so how will traffic reach that router ?

ronbuchalski Tue, 08/05/2008 - 08:03

Carl, see the attached diagram.

Again, the important point to note here is that HSRP only determines the active gateway that will be used for hosts to get off of the access LAN. For traffic flowing back to the access LAN, BOTH routers are fully active and will route packets to the access LAN.

Hope this helps.


carl_townshend Tue, 08/05/2008 - 11:09

hi there, im still confused here, if I advertise the same network on my dist routers, the core will load balance all traffic between them, am i right? if this is the case, if traffic hits the non active router, how will this get to the other router which is active ?

ronbuchalski Tue, 08/05/2008 - 11:35

Carl, I'll try one more explanation for you. Please refer to the diagram I posted previously...


HSRP is designed for networks where hosts cannot choose between multiple gateways, or are generally configured to point to a single IP gateway address. It "invisibly" allows multiple routers to be able to accept packets sent to a single gateway address, without the host needing to know which physical device actually routed the packets.

When you have two (or more) routers participating in HSRP, the HSRP protocol and your router configurations determine which router accepts the packets which are PUT ON THE ACCESS LAN, destined FOR THE GATEWAY.

HSRP only determines which router acts as the gateway for packets that need to leave the Access LAN.

All of the other routed interfaces of your Distribution switches act in the way you configure, according to the routing methods you use. RIP, OSPF, EIGRP, STATIC...whatever.

Traffic coming from your Core routers to your Dist. routers can go to EITHER Dist. router, because BOTH OF THEM can then route the packets onto the Access LAN. Your routing protocol will load balance across equal cost paths. Looking at the diagram, each Core router has TWO equal cost paths to network and

The Dist. routers are using HSRP on the Access LAN interfaces. REPEAT...ACCESS LAN INTERFACES. This does NOT mean that either Dist. router is non-active when it comes to routing packets from the Core to the Access LAN. BOTH Dist. routers are fully capable of doing so.

So, again...HSRP is only configured on the router interfaces connected to the Access LAN, and it is intended to aid the hosts on the LAN by "invisibly: providing a redundant gateway.

If you want to check it out in further detail, set up a lab using the diagram I provided as a model. Send pings, traceroutes, etc, and use a sniffer to see how the Dist. routers respond and participate with the hosts on the Access LANs.

Again, I hope this helps to clear up any confusion.


carl_townshend Tue, 08/05/2008 - 12:59

hi there, i see what your saying but, the only thing is, if the return traffic is load balanced back to the dist routers, will some of the packets go to the other dist router and never reach the client? hope you know what i mean ? i.e both dist routers will be advertising the same vlans and ip networks, not different.


ronbuchalski Wed, 08/06/2008 - 11:05


If a packet is routed to a router, and the router has an interface directly connected to the destination network, it will not route the packet to another router. Instead, it will arp for the destination host and then deliver the packet to the connected network.

You still seem to be confused by this fact.


carl_townshend Thu, 08/07/2008 - 00:46

Hi , I understand that, but what im saying is that we have 2 distribution routers, 1 active one standby, these connect to the core routers via ospf, I want to know how come the traffic manages to get back to the client as it is load balanced back from the core to the non active distribution router also.


ronbuchalski Thu, 08/07/2008 - 06:08

Carl, your comments indicate that you still do not understand what I have been trying to explain to you.

I will try one more time to explain this, using a more 'personal' example...

You have a letter to mail. There are two mail carriers, named 'Act' (for Active) and 'Stan' (for Standby), standing side by side, waiting to accept your mail. As you approach, you hear them repeatedly telling each other, "I'm alive". When you hand off the letter, Mail Carrier 'Act' tells you, "I'll take that" and accepts the letter from you. THIS IS HSRP.

Meanwhile, two bags of letters are delivered to the mail carrier's area, and one bag is given to each mail carrier. As they each look through their individual mail bags, they find letters addressed to you. BOTH of the mail carriers independently deliver these letters to you as they find them. They work totally independent from each other, and do not tell each other that they are delivering letters to you. THIS IS LOAD BALANCED ROUTING.

I hope this helps you to understand what I have been posting.


carl_townshend Thu, 08/07/2008 - 12:49

hi there, I know what load balanced routing is, the question was more of what the other distribution router does with the packet, i think as it arrives to the other dist router, the interface is on the same subnet as the client and would just use arp to resolve the client, it would use the link in between the dist routers to get there as the other direct link would be blocked by spanning tree in my config, i think i know what happens now

thanks for your help

ronbuchalski Thu, 08/07/2008 - 13:05

Carl, that is correct. From a spanning tree perspective, the frames sent from the distribution routers will be sent to the spanning tree root first and then down the appropriate branch. So the non-root dist. switch will send the frames across the interconnecting link to the root dist. switch.



This Discussion