So I have 2 routers (cisco 3640) that each go to their own ISP and then back to the same switch. I have setup ospf and glbp, and now have pretty good redundancy. If either internet connection or routers go down everything is still golden.
So I was thinking that if an interface went down then the router would not be load balanced with glbp which got me thinking whats the best way to get interface redundancy (and I was going to add a 2nd switch with the second interface)... my thoughts
1) Setup BVI on the 2 interfaces.
2) Setup a 2nd interfaces (on each router), I would have to split the subnet, for instance:
then the machines could be on the subnet 192.168.0.0/23 and setup glbp for 1 ip across all 4 interfaces (I'm not even sure if you can do this but think it would work).
3) Is there a way to utilize etherchannel or anything like this
A negative to option 2 would be that if 1 of the interfaces went down, all the sudden 2/3 (or so) of your traffic would be going through 1 router
Just looking for the best way... I may not have even touched on the right solution... I'm open to any solution... was leaning towards trying BVI but saw a post where someone said its messy.
Okay a few points to consider -
1) if you have a pair of firewalls and run them in active/standby and insert them between the routers and the internal switches then GLBP gives you nothing. This is because the active firewall will be the only mac-address seen by the routes and so will be assigned to the same router every time. So you find that only one ISP would be used unless the router failed and then it would be the other ISP.
2) If you do insert the firewalls then you would use a switch between the routers and the firewalls. This means that if a LAN interface on one of the routers fails then the firewalls won't know about it because their connection to the switch is still up. The ways round this are -
1) use IP SLA on the firewalls if they are ASA
2) use a dynamic routing protocol between the routers and the firewalls ie. you could run OSPF between your routers and firewalls.
3) Not clear on whether you want to use both ISPs simultaneoulsy or one and then the other if the first fails. If you want to use one and only the other as backup then you need to influence the static route so one is favouted over the other.
4) Not many people worry about interface failover in the way you are. If you have 2 of everything including internet connections then you have the redundancy. The whole idea of this design is that you have removed any single point of failure.
If you do want it there are a couple of options -
i) BVI as mentioned
ii) use a L3 switch between the firewalls and the routers. Use 2 LAN interfaces on each router and connect back to the switch. These would be on different subnets. Then the firewall would also connect into this switch on a different subnet again.
You then run a dynamic routing protocol between the firewall(s) and the routers.
So each router will advertise it's route(s) via both LAN interfaces. The firewall then sees 2 equal cost paths to each router so in effect it gets 4 routes (2 from each router).
Drawback to this is that using a L3 switch on the outside of your firewalls is often frowned upon.
iii) You could directly the routers and then if the LAN interface of one router fails there is still a path to that router. But the problem here is that for outbound traffic it would never go across that path because the router with the active LAN interface has a direct route to the internet.
However where it is useful is if you are presenting servers to the internet and you do not have provider independant addressing ie. you use one of the ISPs blocks for servers. If the router that is connected to this ISP (R1) looses it LAN interface the block could still be advertised and traffic could come to R1 and then be routed across the interconnect to the other router and down to the firewall(s).
There is so much that could be covered here but it is going to confuse things even more. As you say you still have a lot of things to tackle and i would recommend concentrating on designing/configuring this before you worry too much about interface redundancy. It's not that important compared to all the other things you need to get working.
Things to be aware of -
1) ISP addressing. As you can see from the above the addressing can have a large effect on how the design works. If you have 2 blocks, one from each ISP this can affect how you advertise out servers within your network
2) firewalls - if you are looking to use firewalls then you need to decide which ones, are they going to be running in active/standby, active/active etc. Again how you run them has a big effect on the design. Bear in mind with firewalls between your internal switches and routers you probably won't be running GLBP on your routers.
You also need to make sure any firewall you purchase has the functionality you need.
3) ISP - one or 2. If you use one ISP you could have multiple circuits off them. Certainly in the UK we could order 2 circuits from the same provider to the same site but with completely different routing of the circuits. It may make things easier to begin with.
It sounds like you are going from a no redundancy situation so it can be done in steps rather than try to do the whole thing at once. Certainly the design should be flexible enough to add more redundancy but it might not be advisable to do the whole thing at once in terms of implementation. I am not trying to put you off at all but there is a lot to think about in these setups.
Please feel free to come back with further questions as you may well have more.