I think I have it riddled out... I don't have a diagram to show but I think I can explain my solution which will offer insight. I will have a switch that will have two OSPF areas defined: Area 16 (some arbitrary number) for my internal lan to know best way to route to my other internal L3 switches. Then I'll have another area, Area 0 that will be my network ports connecting to off-net devices (managed private MPLS WAN router and firewall appliance to cable connection for VPN). This will be mirrored configuration at all sites (except for that internal Area arbitrary number) so that Area 0 should be the only Area that redistributes through BGP on the MPLS managed router link and straight OSPF through the firewall cable VPN link keeping my routing table pretty clean due to the number of subnets/VLANs I have on those internal LAN Areas. Don't know if that makes sense or not.

You have two ways of doing it. Having each side with its own Area0 and having a third protocol running redistribution of these routes. (Like the BGP you mentioned). Or you may ask your L3 MPLS provider to run OSPF with you (if they support it).

OSPF is not always supported for L3 VPN MPLS due to scalability issues with the OSPF design for MPLS. But, other many carriers will support it.

For the second scenario:

In each side, the router connecting to your MPLS provider, as well as your appliance should be on AREA0 and being interconnected.

All the rest of your LAN of each side, should reside in another area. So you should end with something like:

SiteA_LAN -> Area0 ->WAN<- Area0 <-SiteB_LAN

Where your "WAN" will be either the MPLS and/or your backup path.

For this to work, you should get either L2VPNs or L3VPNs from your MPLS provider and simulate a point-to-point connection over your cable (i.e. a VPN).

The important thing is to remember that all the areas will communicate to each other over Area0 only. (yeah there are tricks around this). Also, that Area0 must be a contiguous area. This last point, is what you achieve by running the VPN over the backup cable link and L3 or L2 VPNs with your MPLS provider.

HTH.

-William

I don't think I'll have any other choice but to use BGP at the router level. This is something I think is in stone with AT&T. My follow up question is if I'm understanding this correctly, I will have my switch ports that are "off-net" going to my AT&T managed router for MPLS and my Sonicwall firewall failover (VPN & Internet); the rest of my switchports for various internal VLANs OSPF'd for my internal network. I know for this to happen, my off-net switchports have to be on some separate subnet from the rest of my internal VLAN'd subnets. My question is basically, I just need to come up with some other subnet ip address scheme for the Area 0 "off-net" subnet? I would never use VLAN1's subnet to do this with, right?