Firewall Failover and ARP

Unanswered Question
Mar 25th, 2009

Hello All:

I'll try to be succinct.

A router's fastethernet interface and two firewalls' outside interfaces all sit on the same subnet - R1, FW1 and FW2.

Interconnecting them are 2 L2 switches with a trunk between them - SW1 and SW2.

So, R1 connects to SW1.

FW1 also connects to switch 1.

Sw1 has a trunk to SW2.

FW2 connects to SW2.

Imagine R1 has a static route to network 1.1.1.0 with a next hop that points to the firewall failover IP.

This means that Sw1's MAC address table will have an entry for the FW failover MAC in its table.

I don't know if SW2 will have a MAC address entry for the failover IP, too, but thats not the real question I have.

Anyway, what happens when FW1 fails and fails over to FW2?

SW1 has an entry for the failover MAC thats bound to the interface to which FW1 is connected, because FW1 was the active FW and responded to the ARP request put out by R1. Now that FW2 is the active FW, does it send out a gratuitous ARP to inform the switch fabric that the failover MAC can now be found on another switch and switchport?

If not, does R1 have to wait for an ARP timeout before it sends out another ARP request to get a response from FW2?

I hope I am being clear.

Thank you

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.6 (5 ratings)
Loading.
Giuseppe Larosa Wed, 03/25/2009 - 12:53

Hello Joe,

the FW taking the active role should sends out a gratuitus ARP telling "This ip address is reachable at my MAC address":

this updates the CAM tables of both switches and the ARP table of the router R1.

For example a pair of routers providing an HSRP VIP ip address use the aforementioned gratuitus ARP top update CAM tables on switches

otherwise if no ARP is sent as you have noted until ARP entry timeout connectivity is broken

Hope to help

Giuseppe

ex-engineer Wed, 03/25/2009 - 13:11

Hi gentlemen:

Thank you for your timely responses.

"the FW taking the active role should sends out a gratuitus ARP telling "This ip address is reachable at my MAC address""

I guess that is what can happen. I always thought of SW1's CAM getting updated the normal way, which is when R1 wants to forward traffic and broadcasts an ARP request. In this case SW1 will forward the ARP request out all ports in the vlan. FW1 will respond and this is how SW1 CAM gets updated.

Now, my first question is how does SW2's CAM get updated? FW1 is directly connected to SW, not SW2. Does SW1 forward its CAM entry to SW2?

For the second question, which is actually my original one, what happens when FW1 failsover to FW2? How do the switches, SW1 and SW2, learn that the MAC address that is associated with the failover IP can now be reached via another switch and switchport? Does FW2 send out a gratuitous ARP immediately after becoming the active FW?

Lastly, is the PIX failover like HSRP, where the virtual IP and MAC are the same no matter which device is forwarding traffic? Or does the Failover IP take on a different MAC when it fails over to the backup FW?

PS. I attached diagram just to facilitate.

Thank you very much once again. I really appreciate your indulgence.

Attachment: 
Jon Marshall Wed, 03/25/2009 - 13:19

Joe

Bear in mind that the vlan that the router interface and the 2 firewall interfaces are in is present on both switches and is allowed across the link. So any arp broadcast that is flooded out of all ports will also be sent across the link from SW1 to SW2 and then SW2 will forward it out of all ports in that same vlan.

FW2 would send out a gratuitous arp immediately and based on previous paragraph this would also go to SW1.

The mac-address and IP address are moved over to the standby firewall when it becomes active.

Jon

ex-engineer Wed, 03/25/2009 - 16:04

Jon, thank you.

I am satisfied and clear regarding the grauitous ARP from FW 2 after it becomes active due to a failover. I figured that was the case, or else the ARP entries would have had to time out and that would have taken too long.

As a quick aside, this means that if the link between FW1 and SW1 fails, R1's Fastethernet interface would still stay up, and therefore R1 would not lose its route to 1.1.1.0/24, which points to the failover IP. Correct? In other words, R1's route would still be valid and not withdrawn from its routing table?

=======================================================

The part Im a bit stuck on is the first question about flooding ARP requests and responses.

"So any arp broadcast that is flooded out of all ports will also be sent across the link from SW1 to SW2 and then SW2 will forward it out of all ports in that same vlan."

OK, so taking it step by step, please. R1 ARPs for the failover MAC by sending a broadcast to SW1. SW1 forwards the ARP broadcast out all ports in the vlan, including over the trunk to SW2, as well as its connection to FW1.

Being that FW1 is the active FW, it will respond with a unicast packet to the originator of the ARP request (R1). SW1 receives the ARP unicast response from FW1 and forwards it to R1. Great. Now R1 knows the failover MAC and SW1 knows which port it corresponds to.

Lets go back to SW2, now. It received the initial ARP broadcast from SW1 and forwarded it out all the ports in the vlan, which for SW2 is only the direct connection to FW2. FW2, however, does NOT respond because it is not the active FW. So how does SW2 ever learn about FW1s MAC when the ARP response was unicast to R1 and only passed through SW1? SW2 never heard the ARP response. No?

Am I being a nut? Or is this a legitimate question?

Jon Marshall Wed, 03/25/2009 - 16:16

Joe

"Am I being a nut? Or is this a legitimate question? "

No your being a nut :-)

Seriously though your questions are perfectly legitimate and i will try and answer them -

"As a quick aside, this means that if the link between FW1 and SW1 fails, R1's Fastethernet interface would still stay up, and therefore R1 would not lose its route to 1.1.1.0/24, which points to the failover IP"

Correct, R1 keeps it's route pointing to the same IP address which is now moved over to the standby firewall.

"So how does SW2 ever learn about FW1s MAC when the ARP response was unicast to R1 and only passed through SW1? SW2 never heard the ARP response. No? "

Remember when you setup LAN failover the 2 firewalls are aware of each other and of each others IP and mac-addresses. If the 2 firewalls didn't have a LAN failover connection then what you say would be correct.

Jon

ex-engineer Thu, 03/26/2009 - 05:44

Thanks, Jon.

I dont want to be a pain, but its still not jiving with me.

If FW1 had put out a gratuitous ARP when it came online, then I can see SW1 and SW2 having an entry in their CAM tables because the ARP would be broadcast. Much the same way FW2 sends out the gratuitous ARP after the failover - in that case SW1 and SW2 know about the change.

Anyway, I dont want to obsess over this. If I had the equipment, I would just set it up here....

Thanks

Jon Marshall Thu, 03/26/2009 - 06:05

Joe

Your'e not being a pain, it's that i am not explaining it very well.

If i understand correctly the gist of your question is that if FW1 responds with unicast to R1 then how does FW2 ever learn about FW1 mac-address so it can use that mac-address if it becomes active. Is that correct ?

Well when you setup failover between 2 firewalls the firewalls need to exchange hello packets at the very least to monitor each others status. For the 2 firewalls to exchange packets they need each others mac-address as they are communicating over an ethernet LAN. So FW1 and FW2 will always know each other mac-addresses and IP addresses.

Jon

ex-engineer Thu, 03/26/2009 - 06:09

"If i understand correctly the gist of your question is that if FW1 responds with unicast to R1 then how does FW2 ever learn about FW1 mac-address so it can use that mac-address if it becomes active. Is that correct ?"

No, incorrect. I knew that was what you were thinking when you gave me that last answer. :-)

"So how does SW2 ever learn about FW1s MAC when the ARP response was unicast to R1 and only passed through SW1? SW2 never heard the ARP response. No?"

I asked how does SW2 (switch 2), not FW2, hear the unicast ARP response from FW1 to R1. SW1 hears it because it passes right through it. But what about SW2?

Jon Marshall Thu, 03/26/2009 - 06:14

"No, incorrect. I knew that was what you were thinking when you gave me that last answer. :-)"

oops :-)

So how does SW2 ever learn about FW1s MAC when the ARP response was unicast to R1 and only passed through SW1? SW2 never heard the ARP response. No?"

SW2 doesn't hear the arp response from FW1 to R1 you are right. But for FW2 to talk to FW1 then FW2 would have to arp out for FW1's mac-address and SW2 would then know that FW1's mac-address was located via the switch interconnection.

If FW2 then becomes active when it issues it's gratuitous arp SW2 and SW1 both update their CAM tables. So SW2 no longer thinks that the active mac-address is reachable via the interconnection because it now knows it is on the port connected to FW2.

And SW1 now knows that the active mac-address is no longer on the port connected to FW1 but has moved to the interconnection to SW2.

Am i doing any better :-)

Jon

ex-engineer Thu, 03/26/2009 - 07:23

"But for FW2 to talk to FW1 then FW2 would have to arp out for FW1's mac-address and SW2 would then know that FW1's mac-address was located via the switch interconnection."

That is really interesting! FW2 ARPs for FW1's MAC. Wow! I would think that FW2 already knows what the virtual MAC is because it shares it with FW1 as a result of the fact that they are in a failover set up.

So, it begs the question, is the virtual MAC predictable? Is it based on a certain standard, or does FW1 just assign one arbitrarily and that is why FW2 doesnt know what it is and has to ARP for it?

The plot thickens...:-)

And yes, you're doing outstanding and I do indeed appreciate very much your time and thoughts. So much so that Im finally going to rate your posts. :-)

Jon Marshall Thu, 03/26/2009 - 09:34

Joe

Appreciate the ratings, thanks.

The actual mac-address and IP address are switched over to the new active firewall. It isn't like HSRP where each device has a real IP address and then share a virtual IP address/mac-address.

As far as i know the actual mac-address and IP address are transferred to the new active firewall.

Unfortunately like you i don't have 2 pix firewalls handy but that is how i have always assumed it works.

Jon

ex-engineer Thu, 03/26/2009 - 10:45

Alright, Jon.

So, I think I got it now. :-)

Thank you for all your help and attention.

Giuseppe Larosa Wed, 03/25/2009 - 13:40

Hello Joe,

as Jon has explained the gratuitos ARP is sent with a broadcast destination and source= MAC address

so SW2 will create an entry for the MAC and will associate it to the link to SW1.

L3 capable devices can update their ARP tables so this single message can update L2 switches and L3 devices

Hope to help

Giuseppe

ex-engineer Thu, 03/26/2009 - 07:26

Mr Giuseppe:

Im sorry I havent gotten back to you, but Jon and I are in deep thought. :-)

Thank you for your input and help.

Actions

This Discussion