Question on Multi site, Multi Default gateway

Unanswered Question
Oct 10th, 2008


Not sure how to explain this exactly, but I attached a drawing.

We have 2 sites, 1 site with a couple of 3800's and a 6509.

the other site has 2 6513's and a couple of 3800's aswell.

Both sites serve the same VLAN's and the vlans go across using a trunked LAN Extension.

What I would like to do is have PC's on Site A have a default gateway going out via the Site A Router. If not available, then go to Site B as a alternate gateway.

And vice versa for the Site B users, meaning there primary gateway is out Site B's router, and secondary is out Site A's router.

I do not want to make any changes to the systems themself, but be able to do this either on the 6500's. The Fortigate Firewalls are presently doing the inter-vlan routing.

Please let me know some ideas and/or feedback as to what should be done.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 3 (1 ratings)
Jon Marshall Fri, 10/10/2008 - 10:42

Could you clarify a couple of things

1) 6500 switches in both sites are only acting as L2 devices ?

2) What is the addressing in SiteA and SiteB on the firewalls for vlan 100. Are they running failover between them.

3) What is the traffic flow for each site at present.

If the 6500 switches are only L2 there is not a huge amount you can do on them - could they perhaps be responsible for routing vlan 100 ?


pipsadmin Tue, 10/14/2008 - 04:12

1) Yes, the 6500's are only L2, intervlan routing is done on the firewall.

2) The vlans cross into both sites, so Vlan 100 on SiteA is the same IP Schema as SiteB. Same thing for everysingle VLAN in the network.

3) There is a 1Gig Lan extension and we use about 20% so far.

The issu I have with the 6500's doing the intervlan routing is that it is going to be a nightmare to manage all ACL's required, as the firewall currently have about 1000 different acl's. Most of the ACL's are host based, or so I have been told.

I would rather have the ACL done on the 6500 or beter yet, add a 3845 on each side to do this, but the 6500 is far capable of doing this on it's own.

I just need to provide more then 1 option, as bringing the vlan routing down to the 6500 is going to be a major task and might take a whole weekend to do, the directors are giving me 3-5 hrs to work with...

pipsadmin Tue, 10/14/2008 - 04:13

Note that the diagram does not mention the other 6513. On siteA, there is 2 6513's. They are noth linked between eachother with 10Gig links and share the same vlans.

Jon Marshall Wed, 10/15/2008 - 04:54

Oops, sorry i missed your reply. If it is the same vlan and across both sites and you want site A to use Site A firewall and only use Site B firewall if A fails and vice-versa then if the firewalls are responsible for the routing the obvious question is what can the Fortigate firewalls do for you - sorry but i have no experience with Fortigate.

Now if you migrated the routed interface for vlan 100 to the 6500 switches you then could either

1) use PBR with 2 next-hops for each site with the order you want them used ie. on site A 6500's next-hop would be fortigate firewall in site A and then fortigate firewall in site B.

2) Use 2 default-routes on each 6500. Site A default-route would point to Site A firewall and then you add a floating static pointing to site B eg.

ip route

ip route 250

and do the reverse on site B's 6500's.

For this to work the vlan that is used to connect the fortigate firewalls to the 6500 switches in each site must be the same vlan and must be allowed across the LAN extension.

I think 2 is a better option if your topology supports it.

Of course if inter-vlan traffic within and between sites is controlled on the fortigate firewalls for security reasons only then migrating to the 6500 is not an option. ACL's on the 6500 are not the equivalent of a stateful firewall. But if the 6500's need to stay as L2 only then you have to rely on your firewalls for L3 failover.


pipsadmin Wed, 10/15/2008 - 09:26

Thanks for the reply Jon,

I was thinking of option 2. But the Fortigate firewall has a connection to each vlan. The actual box has 10 Ports, and on some ports, they are setup as trunk ports carrying multiple vlans. Only the big vlans, like Production, Corp and Voice have their own interface Switched ports on the firewall, not trunked.

But by doing option #2, The 6500 would have to be doing the intervlan routing right? So I would have access lists on each vlan interface, which would make it to complicated to manage due to the amount of acl's we would require.

One thing I'm thinking is how would the second gateway be used in both cases? Only if the IP of the gateway itself is not reachable?

What if the link from the 3845 to the carrier goes down?

Example of gateway config:

Firewall at Site A

Corp vlan GW

Prod vlan GW

QA vlan GW

Voice vlan GW

The vlan interfaces on the 6500's do not have an IP assigned to them, hence does not do inter-vlan routing.

Jon Marshall Wed, 10/15/2008 - 09:53

"The vlan interfaces on the 6500's do not have an IP assigned to them, hence does not do inter-vlan routing."

I know and that is your problem really. If your switches are only running at L2 then it's very difficult to get them to provide L3 redundancy.

Do the fortigate firewalls run VRRP or something like it where they share the virtual IP between the 2 sites. What you really need is something like MHSRP functionality on the fortigates so that you can have site A clients going to site A fortigate but failing over to site B if need and vice-versa.

I agree with you that migrating all the config to the 6500's is a lot of work and could well introduce security problems but if you want to leave the vlan gateways on the fortigates then you will have to find a solution on the fortigates.

Or to put it another way, i don't know of any other way of doing it. But others might well have suggestions - there are some very bright people in these forums :)


pipsadmin Wed, 10/15/2008 - 10:04

currently, the Site B does not have any firewalls, as everyting is sent down the Lan extension to Site A.

Would installing a Cisco ASA be able to provide us this MHSRP/VRRP?

Jon Marshall Wed, 10/15/2008 - 10:12

Don't want to mislead you. VRRP is not the answer really. MHSRP is, because of your particular requirements. VRRP would allow the firewalls to failover between each other but at any one time only one of the firewalls would be active for all clients in site A and B.

I don't believe the ASA's support MHSRP. Also need to be careful that even if they did you would need to be very careful that replacing the fortigates does not lose you some functionality that the ASA's cannot support. The 6500's support FWSM's so you could integrate the firewalling into the 6500 chassis but they are expensive and i would still recommend a separate front-end pair of firewalls.



This Discussion