I am building a highly available but relatively small flat network with no single point of failure. At the core of the network are 2 Ethernet switches. All servers and clients are dually attached using NIC teaming. I also have 2 routers used by users in remote locations to access the network.
Is it best practice to dually attach each router to the switches? I would like to still have my 2 routers available even in the case of a switch failure.
If I do dually attach the routers, what is the best practice? Bridge both Ethernet interfaces into a virtual bridged IP interface? I would configure Spanning Tree to prefer the inter-switch Link Aggregation bundle as the preferred path between the switches.
Comments and questions welcomed...
You don't say whether the switches are responsible for inter-vlan routing or not.
If they are then i would dual connect each router to each switch. I would use L3 Point-to-point links from the routers to the switches and use a routing protcol such as EIGRP/OSPF to exchange routes between the switches and the routers. For all of the above you need L3 switches.
Are your switches L3 switches or L2 only.
If you have to have both routers still available with a switch failure and the routers are responsible for the inter-vlan routing then yes you will have to use BVI to dual connect each router.
If the routers service different remote sites then this is probably the way to go.
"All servers and clients are dually attached using NIC teaming."
As in link aggregation across both switches? If so, are they something like 3750s in a stack configuration?
"Is it best practice to dually attach each router to the switches?"
Usually not for L2, since routers usually won't allow two interfaces on the subnet.
A couple of things you can do. Some Cisco routers can link aggregate across different stacked 3750s (or line cards in a chassis). Or, you can define a new transit route between the routers. Physically, the routers could be directly interconnected or you can define another VLAN across both switches and connect routers to it. (Second interface connected to other switch.) With the latter routed solution, you need to be mindful of routing path costs. I.e., although the second router will be available with the loss of a switch, end-to-end path cost might preclude it from being used. This might be addressed, if currently using something like GLBP, to migrate to HSRP and adjust active router to equally use both it's own path and transit path via the other router.
Jon, just saw your post. Good suggestion on BVI! I tend to overlook such solutions on routers.
Funnily enough just about to say good suggestion on transit vlan using separate IP addressing.
I have used the transit vlan solution and it generally does work but i agree that we would need some info re. remote sites to fully understand routing issues.
BVI is something i often shy away from as well - not sure why as i'm quite comfortable with switching.
With apologies to Dan for this somewhat off-topic reply; Jon guess we'll have to form a mutual admiration society. ;)
If you have the equipment, the solution I like is the link agregation from the router to couple of switches. Learned, here on the forums, you can aggregrate some ISRs Ethernet ports when running some of the later IOSs.
As I mentioned in my original post, the network is a flat, single VLAN deployment on the LAN side. The routers are used to reach remote users over redundant WAN links. VRRP is used to virtualize the routers to the devices on the LAN.
The switches are non-stackable L2 switches (2960 for example). I cannot cost-justify using stackable switches.
The servers and clients implement an active-backup Ethernet teaming. They will prefer a particular NIC and only fallback to the other NIC only in a case of a switch or cable failure.
Since the servers and clients implement an active-backup Ethernet teaming and since you're using VRRP, do you actively (as in pass traffic) use both routers now, or is there an active and backup path (both upstream and downstream)? Reason I ask, if you're not using both WAN paths to pass traffic actively, VRRP and normal dynamic routing should move traffic to the redundant router with loss of the switch that had active VRRP router.
If both WAN links are actively passing traffic, then since the switches are not stackable, you'll be unable to use the router to switches link aggregation technique, but both the transit subnet or BVI approaches might be used.
Sorry for not getting back earlier...
I do want the routers to load balance in order to fully utilize the links. On the LAN side, I would implement 2 VRRP addresses, each primarily hosted by a different router. Half of my devices would point to the first VRRP address, the others to the other VRRP address.
How do you implement this transit subnet? What are the pros and cons of transit subnet versus BVI?
(BTW: Instead of using dual VRRP addresses, have you considered GLBP?)
A transit subnet could be implemented by defining a new VLAN for it that's on both switches to which you connect one of the router interfaces. Or, you can just connect one router's interface to the other router's interface. Either approach could define the new subnet as a /30.
The pros and cons boil down to L2 vs. L3. Today, with all new multilayer L3 capable switches, L3 designs are being pushed. BVI would tie you to equipment that supports it, where L3 should be doable with any L3 device. BVI might seem simpler unless what ports are actually passing traffic is important. BVI also might make it easier to maintain usage of both WAN links when one switch fails.
Without really thinking about both methods, and thinking about "what if", can't easily recommend over the other.