Layer 2 Ethernet WAN Link - MST

[TimLaffan]

Hi All,

I hope someone could give me some recommendations, we are having a 100MB Layer 2 Ethernet Link installed within two sites. The plan is to have a 3750 at each end that will terminate to them. Until now the sites have been two completly seperate LANs connected together by MPLS Links (Routed)

As i understand it, A single instance of Spanning Tree will now be running across both networks with a single Root Bridge, we do not have the capability to run RPVST as we use HP switches at the access

One of my thoughts is to run MST at both sites, as all switches support this protocol, aswell as the capability to run two different regions, so one region for one site and another region for another - But this bring up a question... Both networks have been configured so a VLAN on one site matches the VLAN on another site - there is no dynamic VLAN assignment, its all assigned on a port basis. If we run two regions will the VLAN's be an issue and is this solution ideal? I have attached a Visio drawing of our proposed configuration

Site 1 - Running Region 1           Site 2 - Running Region 2

VLAN 10                                   VLAN10

VLAN 20                                   VLAN 20

VLAN 30                                   VLAN 30

etc                                            etc

If this sounds link the right configuration, is it just a case of configuring MST regions on both sites and then let it do the work?

Thanks

Tim

[Peter Paluch]

Hi Tim,

I do not believe that it is necessary to split your network into two MSTP regions unless you really need to have different root bridge for a single VLAN in two locations (which is not a good idea anyway - a single VLAN, be it local or spanning through L2 VPN, should have a single root bridge). As opposed to areas in link-state protocols which are used quite often, splitting a network into several MST regions is only rarely required by intent. In most cases, a single-region MSTP configuration is the best way to go.

What is necessary, though, is to ensure that your provider of Layer2 VPN service transmits the MSTP BPDUs transparently, i.e. that the L2 VPN does not actively participate in the MSTP. Make sure that this is the case (ask your provider whether it provides tunneling of MSTP BPDUs).

Can you describe your requirements in more detail?

Best regards,

Peter

[TimLaffan]

Hi Peter,

Thanks for the quick reply,

We have purchased a LES100 Circuit between the two sites rather than a VPN Link.

The requirements are:

Configure a Trunk for VLAN 103,104 (Server Farm) across the LES100 circuits

The idea being, if the Tub site is taken down (As my boss likes to call it "Hit by a VC10"), then we could bring up Virtual Machines's at the other site with as little configuration as possible.

The Issues,

Both sites have different IP ranges, Tub 172.16.x.x, Wyc, 172.17.x.x, also each site has the same VLAN ranges created 10,20,30 etc adding 10 each time

The IP ranges have been broken down into /24 address - 172.16.10.x, 172.17.20.x etc - with SVI's assigned and InterVLANrouting enabled

172.16.10.254, 172.17.254

When the LES100 link goes in, we are only going to span across the 103,104 vlans, and these will only be in the 172.16.103.x, 172.16.104.x range - no 172.17.x will exist on the other side

As i understand it, MST creates different instances within regions and each instance is assigned to a group of VLANs, so if i put VLAN 10 into an instance it will assign a root bridge for that instance which will be the root for VLAN 10 on each site - The root will be the 3750 at Tub - so all wycombe switches will have the switch at Tub as the root - Will this be an issue, as no loop link exist across the links?

Also if we are allowing only VLAN 103 and 104 to cross the link how will MST talk to the other switches?

Mind has gone blank, so if you need any more information let me know... End of working day

[Peter Paluch]

Hello TIm,

The MSTP indeed works by running a configurable count of instances inside a region. Each instance can have its own spanning tree and thus its own root bridge, and all VLANs mapped to the same instance share that same spanning tree. Note that the instances are defined first, and only then, VLANs are mapped to instances. The MSTP can actually run several instances without any VLANs being mapped to them although that is not useful, but it shows the fundamental property of MSTP - it allows many VLANs to share the same spanning tree, and all VLANs mapped to a single instance are controlled by this instance. The MSTP instance is what is dictates the spanning tree - not the VLAN.

The MSTP runs untagged even over trunk links - it essentially runs in the native VLAN. Allowing only a subset of all VLANs on a trunk does not impair the functionality of MSTP itself, as it describes all instances in a single BPDU. MSTP essentially disregards allowed VLANs on trunks as it deals with instances, not VLANs. Of course, the disallowed VLANs will stay disallowed on a trunk - the MSTP cannot make a trunk carry a disallowed VLAN. However, the MSTP decisions about forwarding and discarding ports are uncoupled from individual VLANs and this must be taken into consideration when electively allowing and disallowing VLANs on trunks. However, you are obviously going to have only a single LES link interconnecting your two locations so these issues should not matter to you.

Having a root switch for one or more MSTP instances on a particular location should not pose a problem.

Best regards,

Peter

[j-marenda]

Tim,

iff you just have ONE Ethernet Link between the sites and you do not plan to add another one which may form a loop

i strongly recommend you not to run spanning tree over that link ("bpdufilter enable" on the port of the cisco switch)

If you want to add a second link with same speed later i would create a port-channel for that.

MST is allmost allways painfull if you add vlans not planned in your mst configuration;

changing the mst configuration (assignement vlan to mst-instance, identifier and verison--number) will cause downtime.

I also wish you good luck and a spanning-tree-bug-less firmware on the non-cisco-switches you mentioned,

(multiple-)spanning tree and interoperability on each side.

So when you start configureing MST,

build instances with lots of spare-vlans (on cisco, the must not exist!)

so adding vlans will not generate a new MST configuration which will produce downtime.

Juergen.

[Peter Paluch]

Hello Juergen,

Regarding not running the MSTP over the LES circuit - well, if there is really only a single circuit going to be deployed between the two sites then of course no loop can be created and the STP is not required to run here. I agree with you on this and I see this as a viable alternative. I usually try to run things safely but in this case, not running the STP over the link may actually make things easier.

Regarding your comment about less-than-perfect MSTP implementation on non-Cisco switches, I am personally not so pessimistic. I have a network running MSTP on HP ProCurve switches, and while the implementation of MSTP on HP switches has some quirks (you cannot map a nonexistent VLAN to an instance ), it runs very nicely. Believe me, Cisco gear has its own box of unique bugs, some of them not being take care of for ages, some of them being quite fresh (sadly, this trend seems to be intensifying in the last years - but that's another topic). I would personally not suggest any vendor being more or less bug-prone.

Best regards,

Peter

[TimLaffan]

Hi both,

Thanks very much for the info, its really helped but also left me with serveral questions which i hope you could help with

When running BPDU filter this will prevent BPDU's from crossing the link so in theory will this be link running two seperate regions of MST? Even though running the same configuration

Native VLAN, would you recommend trunking this across the LES circuit?

The 3750's are stacked so we have no reason for running mutiple instances (If one fails, we will fail over to another one in the stack), i was planning to keep all VLANs in instance 0, is the any recommendations to move them all to run in instance 1

Would you be able to explain abit more why you one only run one region of MST across both sites

[j-marenda]

Tim, if the two swichtes are stacked with the 100MBit/s Ethernet-Circuit,

they form ONE Switch .

I am not sure thatyou really want this thru a leased line.

Also remember that an software update on the one switch

will also start this for the other one (and it will automagically reload) .

Since i do not really know the Spec. of a LES100 Circuit,

i also do not know wether the MTU is big enougth to hold VLAN TAGs etc.

For my infrastructure, i try to avoid the native VLAN.

The Dot.Q Bytes also carry some kinf of "QOS" Information within,

so the may be usefull.

The native VLAN is good for CDP, LLDP, spanning tree, etc.,

but for application traffic, i almost allways use the tagged form.

[Peter Paluch]

Hello Tim,

Allow me to add a few answers although Juergen has been already very helpful!

When running BPDU filter this will prevent BPDU's from crossing the link
 so in theory will this be link running two seperate regions of MST? 
Even though running the same configuration

Correct. With the BPDU Filter configured on both ports of the LES circuit, the two locations of your switched network will become basically MSTP-independent. While each of them can and assumingly will be running MSTP, these two MSTPs will not interact with each other, in essence becoming two independent regions (although this is not terminologically quite correct because two regions are created by two switches actually talking to each other but each having a different MSTP configuration). Let's perhaps say more properly that these two locations will become two distinct and independent MSTP domains.

Native VLAN, would you recommend trunking this across the LES circuit?

This actually depends on what data you carry in the native VLAN. Exactly as Juergen suggested, the native VLAN is used between Catalyst switches to transport several Layer2 service protocols including VTP, CDP, STP and perhaps some others, so it is the best to put hands off the native VLAN and use other VLANs for user traffic. If you are not aware of any user traffic being carried by the native VLAN and you do not expect to use any of these special protocols then you don't have to transport the native VLAN across the LE circuit.

The 3750's are stacked so we have no reason for running mutiple 
instances (If one fails, we will fail over to another one in the stack),
 i was planning to keep all VLANs in instance 0, is the any 
recommendations to move them all to run in instance 1

MSTP instances have nothing to do with switch stacks. An MSTP instance is a particular spanning tree governed by a single MSTP process shared by a predefined set of VLANs. You create multiple MSTP instances if you want to define different spanning trees in your network (by, for example, selecting a different root switch for each instance) and if you in consequence want different VLANs to use different spanning trees in your network, resulting in load balancing (a link blocked for a particular MSTP instance can be forwarding for a different instance). The MSTP instances provide you with a compromise - the PVST runs a separate STP instance for each and every VLAN while the former 802.1Q specified a single STP for all VLANs. An MSTP instance is something between - you can have a number of instances (up to around 64 on modern switches) - not as many as all VLANs but more than a single STP - and divide the VLANs between these instances so that all VLANs mapped on the same MSTP instance share the same spanning tree topology.

Would you be able to explain abit more why you one only run one region of MST across both sites

Well, an MSTP region is basically a way of MSTP to interact with non-MSTP world or to interact with MSTP world that is configured differently. Having two MSTP regions does not significantly decrease the complexity of MSTP processing except in very large networks, therefore, there is nothing to gain by running two MSTP regions in a smaller network besides having things more complicated. The interoperation of two MSTP regions is not intuitive and if there are switches that implement their own version of per-VLAN STP on an MSTP boundary, there can be different issues arising from this translation of MSTP into per-VLAN STP.

You may be interested in reading these two articles about MSTP - they are very detailed but they are by far the best descriptions of MSTP I've been able to find in years:

http://blog.ine.com/2008/07/27/mstp-tutorial-part-i-inside-a-region/

http://blog.ine.com/2008/09/24/mstp-tutorial-part-ii-outside-a-region/

Best regards,

Peter

[j-marenda]

Peter,

i have tried the Procurve switches (since some customers are keen of them

and pricing for managed just-layer-3 switches looked ok, BUT.

BUT

- there are problems with the spanning tree implementaion.

  They get bug-fixed, and the re-occur.

For one of our customer this means:

a PC or network-printer is powered ON and Spanning tree recalculates

and temporarily blocks all ports. Since they have a big modular Model,

it stoppes the network working and each working day's start is a pain.

Beeing unable to add undefined vlans to an MST Instance is also

not so good since changing MST configuration gives also some down-time.

Delivering that configuration to a lot of swichtes may be a problem

if you do not have out-of-band access.(this applies to _all_ Manufactuers of switches).,

The Region Concept is quite hard to understand (or just bad explained?) and implement.

For me, the desierd functionality is not given.

- have one spanning-tree instance to ethernet-connected customre site

- have multiple of this on the central switches

- have one ore to spanning-tree instances to access-switches in server-housing.

- no need to reconfigure every switch to the full MST instance konfiguration

- on customre site, there may be things like the embeded swithce sof 870 ore 1812 Routers

  which just speak (per-vlan)-nonrapid-spanning-tree without any chance to do MST config.

-eventually, i have a remote switch(-pair) with few vlans. This shall speek MST for that set of vlans to the central site

thru it's redundant links and (pv)-(r)stp(+) to the customers.

And i would like to be able to add customer, vlans, remote-sites, WITHOUT downtime .

This was possible with Cisco's old per-vlan-spanning-tree,

but there are some cpu-limits so on the smaller access-switches, you get problems

if you insist of having more than 32/64 VLANs ; the "bigger" Models ar for Layer2-access to expensive;

the alternatives do not have per-vlan-(rapid)-spanning-tree.

So i am very unhappy with all that, independant from manufacturer.

I also see that MST is the only spanning-tree version common to the swiches of cisco, hp, 3com/h3c/huawai

so you have a chance to get them working together.

Juergen.

[Peter Paluch]

Hello Juergen,

Thank you a lot for your reply. Let me discuss some of your points.

a PC or network-printer is powered ON and Spanning tree recalculates and temporarily blocks all ports. Since they have a big modular Model, it stoppes the network working and each working day's start is a pain.

I am sorry about that and I agree that it is an improper behavior. According to this description, I would say that the ports connected to end devices are not properly configured as MSTP Edge Ports - but I assume you have already ruled out that possibility. In any case, I am running a number of HP ProCurve 2650 and 2510 switches running MSTP against a Catalyst 3560 and I have never observed an improper MSTP behavior.

Beeing unable to add undefined vlans to an MST Instance is also not so good since changing MST configuration gives also some down-time.

Agree absolutely. This is something I should start complaining about to HP to get fixed. It is a stupid limitation to require that the VLAN must exist before mapping it to an instance.

The Region Concept is quite hard to understand (or just bad explained?) and implement.

It's both, I believe. It is well designed but poorly understood and thus poorly explained. A never-ending circle.

And i would like to be able to add customer, vlans, remote-sites, WITHOUT downtime
[ ... cut ... ]
So i am very unhappy with all that, independant from manufacturer.

Well, there are many to blame here, including Cisco. I have noted that if two Cisco switches running MSTP are placed into different MSTP regions and are interconnected via a trunk, they revert to PVST+ operation on the trunk instead of running RPVST+. I don't know why is that - I have even posted a question about this recently here on NetPro but so far, nobody has answered. The result is that if an MSTP region is partitioned, the downtime in Cisco switched network is going to be up to 30 seconds. If the Catalysts used RSTP/RPVST+ as they should, the downtime with any MSTP region reconfiguration would be less than 1 seconds thanks to the Proposal/Agreement mechanism. Even more, because of cumbersomeness of the PVST Simulation when interworking the MSTP and (R)PVST+, Cisco Catalyst switches can actually block the link to a different MSTP/(R)PVST region for an unlimited time until certain criteria are met. In this case, the PVST Simulation actually does more harm than good. Even with Cisco devices, while you are running MSTP in a single region then it's fine, but running it in several regions can be very displeasing. The MSTP concept of region is absolutely fine, and pure MSTP implementation would not have these issues even with several regions but in the particular case of Cisco switches, it is the PVST Simulation, added originally as a good-willed mechanics to interoperate with (R)PVST, that shows itself to be rather counterprodictive (and it cannot be even turned off).

The best thing that every vendor should do is to implement the pure MSTP as standardized in the most recent 802.1Q standard properly, and provide an option of deactivating any proprietary form of PVST or PVST-alike protocol on a MSTP region boundary. The MSTP itself is well-defined including the operation on a region boundary so for a good interoperability, nothing else is required.

One of the obvious problems is that the IEEE came up with the MSTP being a very nice protocol but it has not standardized any open management protocol for MSTP region configuration. Apart from VTPv3 on Catalyst switches, I don't know if there is any other scalable way of maintaining an MSTP region configuration across a switched domain. This is obviously discouraging many people to deploy the MSTP. I find myself to be quite a proponent of MSTP but I understand that all these issues including the relatively little knowledge about the guts of MSTP is alienating most administrators from seriously considering using it, sadly.

Best regards,

Peter