Hi Carl,
So just to be clear you are going to have an MPLS infrastructure that will centrally connect to your HQ. In parallel each site will also have internet connectivity. I guess my first question would be, what does your perceived traffic pattern look like. I assume you will be implementing an IGP (EIGRP/OSPF) and configuring it to advertise routes among the MPLS sites. Will there be any need to back haul internet traffic through the HQ site?
I have had similar infrastructures like this in the past. Its much easier when all INET traffic demarcs the same central location, usualy a colo. In my case I configured the EIGRP advertisements with a default route from the HQ. At each location I had a backup RTR with INET connectivity also peering EIGRP, however, I redistributed a static default route, allowing the injection locally to implement with an AD of 170. This allowed the MPLS Default Route to natively be implemented at each site. In the event of failure, the EIGRP traffic would converge to then use the local sites INET conneciton and I had IPsec tunnels configured in a hub-spoke off HQ. Each tunnel used Reverse-Route Injection (RRI) to allow all downstream routing to also converge and continue using best path.
When the primary MPLS link came back online, EIGRP would again converge and automatically add the correct default route back in.
One snag was the SA Lifetimes would leave the injected static route from the established tunnel in the Colo, so you may want to tweak lifetime settings very aggressive, or modify keepalives and understand you may manually have to wash the SA.
This is a thought and example of how I have configured this type of toplogy in the past. Your description above is a little vague. I hope this helps some.
Thanks.
