Solved: Collapsed Distribution Design

Nicholas Beard · ‎11-24-2010

Hi all, just a few quick questions regarding a collapsed distribution scenario -

To give you a brief outline of the setup, I have two 3750 layer 3 switches setup as the distribution layer with HSRP and a layer 3 link between them. I then have two 2960G switches setup as the access layer with redundant layer 2 (4 port) channels to the distribution layer. I have load balanced the 2960G VLANs; with three VLANs on the first 2960G and three VLANs on the second 2960G. The two 3750 switches have SVI's configured for each VLAN and the desktops/servers have been appropriately assigned the SVI for their default gateway. My questions are as follows -

1) Am i right in saying VTP is a global command and therefore, all switches would learn of all VLAN's should this be configured, and i would want to avoid this?

2) Would i setup two instances of Rapid-pvst based on the load balancing of the VLANs?

3) Within this scenario where would the best placement be for the servers? Out of the distribution layer or maintained within the access layer?

4) Would i setup a HSRP group for the multiple SVI's?

Thanks for your time

Nick

Jon Marshall · ‎11-24-2010

ttgmaverick wrote:

Jon, thanks for the reply you are without doubt a fountain of Cisco knowledge.

Regards question 1, I would not want both 2960G switches learning of each others VLANs otherwise they would span the entire network end to end. Am i right in saying, there is no fundamental requirement for this?

Regards question 2, the spanning tree design is wiping the floor with me (I dont have an excuse, its not even a Friday). I realise you would have an instance of rapid-pvst running within each VLAN but what is really confusing me is the spanning tree design based on the load blancing of the VLANs between switches. I realise i would setup port priorities for each of the channel uplinks, but ive hit a comlpete mental block beyond that.

Thanks

Nick

1) If you don't need to span vlans then the advice is don't. The requirement for spanning vlans is more often seen in a DC where it provides flexibility in relocating servers to different switches/racks etc. However in a non-DC environment it is a good idea to limit each vlan to a single access switch if at all possible.

Bear in mind that often you have at least one vlan that spans all access switches and that is the management vlan used to manage the switches themselves.

2) You don't need to use port priorities. What you do is -

set 3750_1 as the STP root for all odd vlans and STP secondary for all even vlans

set 3750_2 as the STP root for all even vlans and STP secondary for all odd vlans

make sure the HSRP active gateway matches the STP root. Bear in mind this is only really relevant if you have multiple vlans per access-layer switch. If you don't and each switch only has one user vlan then there is little point in doing it.

In a more common setup (although not necessarily better), the distro switches are connected via a L2 link. So if you have an access switch connected to both distro switches with L2 links then something has to block because you have a L2 loop. One of the uplinks will block (note block per vlan) and that is when you manually set about "load-balancing" the vlans as i suggested above. Again it should be stressed this is only worthwhile if you have multiple user vlans per access switch.

But you have a L3 link between your distro switches so both etherchannel uplinks from an access switch can be forwarding, providing you have limited the vlans to one per switch or more specifically no vlan spans more than one switch. Sounds great and if you had GLBP it would be. Unfortunately only one uplink for traffic from the access-layer switches to the distro switches will actually be used per vlan. (Note this is not a critcism of the design at all).

The reason this happens is -

sw1 is connected to both 3750s. Both of sw1's uplinks are forwarding traffic. sw1 has vlan 10 for user devices. HSRP is setup for sw1 and 3750_1 is the active switch. Note that HSRP messages actually go via sw1 between 3750_1 and 3750_2 because they cannot go direct as your 3750 interconnect is a L3 link not L2 trunk.

If a device on sw1 arps out for the default-gateway then the arp will go to both 3750_1 and 3750_2. But only 3750_1 will respond so that is recorded on sw1 ie. the mac-address for the VIP is recorded as being reachable via the etherchannel link to 3750_1.

So you still only use one uplink from the access-layer to the distro although it's worth noting that traffic from distro back to access-layer could actually use the other link.

An alternative to GLBP is MHSRP where you could utilise both uplinks for the same vlan but this does require extra manual config and is debatable whether or not it is worth it. If your'e interested let me know. SImply put it involves having more than one VIP per vlan.

Basically, if you limit the user vlans so each vlan only goes to one access switch then STP is only used as a failsafe and you don't really need any "load-balancing" config although you should still set STP root/secondary. You should still run it though.

Jon

View solution in original post

Jon Marshall · ‎11-24-2010

ttgmaverick wrote:

Hi all, just a few quick questions regarding a collapsed distribution scenario -

To give you a brief outline of the setup, I have two 3750 layer 3 switches setup as the distribution layer with HSRP and a layer 3 link between them. I then have two 2960G switches setup as the access layer with redundant layer 2 (4 port) channels to the distribution layer. I have load balanced the 2960G VLANs; with three VLANs on the first 2960G and three VLANs on the second 2960G. The two 3750 switches have SVI's configured for each VLAN and the desktops/servers have been appropriately assigned the SVI for their default gateway. My questions are as follows -

1) Am i right in saying VTP is a global command and therefore, all switches would learn of all VLAN's should this be configured, and i would want to avoid this?

2) Would i setup two instances of Rapid-pvst based on the load balancing of the VLANs?

3) Within this scenario where would the best placement be for the servers? Out of the distribution layer or maintained within the access layer?

4) Would i setup a HSRP group for the multiple SVI's?

Thanks for your time

Nick

1) If the switches are all in the same VTP domain and you have setup VTP server/client then yes all switches would learn all vlans. If you don't want this use VTP transparent.

2) No

3) Servers can be either access or distribution. In a 3 tier design servers are generally on their own access switches which connect back to distro. However there may be bandwidth issues if you connect to access-layer so you need to be aware of these. You do see many collapsed designs with servers connecting straight to distro switches.

As a side note, not a good idea in a DC.

4) Not sure what you mean. You can assign multiple SVI's to same group but if you don't have to then don't bother.

It's a shame your distro switches don't support GLBP because with a L3 link between the 3750s GLBP would be the perfect choice.

Jon

Nicholas Beard · ‎11-24-2010

Jon, thanks for the reply you are without doubt a fountain of Cisco knowledge.

Regards question 1, I would not want both 2960G switches learning of each others VLANs otherwise they would span the entire network end to end. Am i right in saying, there is no fundamental requirement for this?

Regards question 2, the spanning tree design is wiping the floor with me (I dont have an excuse, its not even a Friday). I realise you would have an instance of rapid-pvst running within each VLAN but what is really confusing me is the spanning tree design based on the load blancing of the VLANs between switches. I realise i would setup port priorities for each of the channel uplinks, but ive hit a comlpete mental block beyond that.

Thanks

Nick

Jon Marshall · ‎11-24-2010

ttgmaverick wrote:

Jon, thanks for the reply you are without doubt a fountain of Cisco knowledge.

Regards question 1, I would not want both 2960G switches learning of each others VLANs otherwise they would span the entire network end to end. Am i right in saying, there is no fundamental requirement for this?

Regards question 2, the spanning tree design is wiping the floor with me (I dont have an excuse, its not even a Friday). I realise you would have an instance of rapid-pvst running within each VLAN but what is really confusing me is the spanning tree design based on the load blancing of the VLANs between switches. I realise i would setup port priorities for each of the channel uplinks, but ive hit a comlpete mental block beyond that.

Thanks

Nick

1) If you don't need to span vlans then the advice is don't. The requirement for spanning vlans is more often seen in a DC where it provides flexibility in relocating servers to different switches/racks etc. However in a non-DC environment it is a good idea to limit each vlan to a single access switch if at all possible.

Bear in mind that often you have at least one vlan that spans all access switches and that is the management vlan used to manage the switches themselves.

2) You don't need to use port priorities. What you do is -

set 3750_1 as the STP root for all odd vlans and STP secondary for all even vlans

set 3750_2 as the STP root for all even vlans and STP secondary for all odd vlans

make sure the HSRP active gateway matches the STP root. Bear in mind this is only really relevant if you have multiple vlans per access-layer switch. If you don't and each switch only has one user vlan then there is little point in doing it.

In a more common setup (although not necessarily better), the distro switches are connected via a L2 link. So if you have an access switch connected to both distro switches with L2 links then something has to block because you have a L2 loop. One of the uplinks will block (note block per vlan) and that is when you manually set about "load-balancing" the vlans as i suggested above. Again it should be stressed this is only worthwhile if you have multiple user vlans per access switch.

But you have a L3 link between your distro switches so both etherchannel uplinks from an access switch can be forwarding, providing you have limited the vlans to one per switch or more specifically no vlan spans more than one switch. Sounds great and if you had GLBP it would be. Unfortunately only one uplink for traffic from the access-layer switches to the distro switches will actually be used per vlan. (Note this is not a critcism of the design at all).

The reason this happens is -

sw1 is connected to both 3750s. Both of sw1's uplinks are forwarding traffic. sw1 has vlan 10 for user devices. HSRP is setup for sw1 and 3750_1 is the active switch. Note that HSRP messages actually go via sw1 between 3750_1 and 3750_2 because they cannot go direct as your 3750 interconnect is a L3 link not L2 trunk.

If a device on sw1 arps out for the default-gateway then the arp will go to both 3750_1 and 3750_2. But only 3750_1 will respond so that is recorded on sw1 ie. the mac-address for the VIP is recorded as being reachable via the etherchannel link to 3750_1.

So you still only use one uplink from the access-layer to the distro although it's worth noting that traffic from distro back to access-layer could actually use the other link.

An alternative to GLBP is MHSRP where you could utilise both uplinks for the same vlan but this does require extra manual config and is debatable whether or not it is worth it. If your'e interested let me know. SImply put it involves having more than one VIP per vlan.

Basically, if you limit the user vlans so each vlan only goes to one access switch then STP is only used as a failsafe and you don't really need any "load-balancing" config although you should still set STP root/secondary. You should still run it though.

Jon