port-fast causing arp cache problem?

wilson_1234_2 · ‎10-30-2008

I have a 6509 switch that one port hold a host type server that is used for banking transactions.

This server has sort of a stateful type failover.

During primary server failure the failover server aquires the primary server's address.

Primary server = 10.10.10.1

Failover server = 10.10.10.100

The failover process causes the ip address to change from the normal host address to a temporary one (same subnet)while the failover process takes place.

The failover server also uses a temp address until it aquires the primary server address.

This process has been failing and the tech support beilevs that portfast was causing a problem by arp caching the ip address and not letting the temp address take over.

Is this possible, if not, can I adjust arp cache on a single interface?

pkaretnikov · ‎10-30-2008

I have doubt regarding portfast being linked to an arp issue.

Do you have any kind of arp inspection set up on your 6500, and is the 6500 performing the routing? Can you view the arp table and verify that the arp table is not updating to reflect the new mac-address? Do the servers send out any gratuitous-arps as part of the failover process in order to make sure the network is aware of the change as quickly as possible?

wilson_1234_2 · ‎10-31-2008

Sorry, but I do not know if we are doing arp inspection on the 6509.

The 6509 is doing ospf routing, but there is an edge router that connects our DR site (where the failover server is).

This is also being bridged across a DS3.

Could this potentially be an issue?

My understanding was that in Cisco gear the arp timeout is 4 hours by default.

But if the server does an arp with it's new ip address, then you are saying this will be updated instantly, is this correct?

pkaretnikov · ‎10-31-2008

Yes, the as soon as it has a way to recognize the backup server as the new owner of that IP it will change the arp table to reflect that.

Most systems accomplish this through a gratuitous ARP. You may want to hook a packet sniffer up and view the traffic flowing across the network during the changeover period. If you do not see any sort of 'announcement' then it's most likely a server issue. If you see it on the wire, but it's being ignored, then it's a problem with the network.

Also, you will need to check the router local to the server, not the core to see if that ARP table it updating. ARP is a local concept.

gnijs · ‎10-31-2008

check on the 6509 (sh arp) if you see the mac address changing of 10.10.10.1 during failover. the 6509 is propably not detecting the ip change and is still sending packets for 10.10.10.1 to the old server (old mac address). do a 'clear arp' on 6509. does this recover communication ? during failover the sec server (!) must sent gratuitious arp packets to update the arp values on the router. these arp pkts are broadcast so should be bridged from the DR site to the main site.

Richard Burts · ‎10-31-2008

Richard

First of all, I do not believe that portfast is causing any of this problem. In fact I suspect that things would be worse if portfast were not enabled since without portfast there would be a period while the interface initializes, listens, learns, before it can forward and that might complicate the fail over process.

Secondly I hope that you can clarify the topology. If the fail over server is at the DR site and if there is a router in addition to the 6500 it would be nice to know how things are connected and what (and how) is being bridged.

Bottom line of it is to ask whether the primary server and the fail over server are really in the same broadcast domain? If the primary server pings the fail over server, does the primary server have an entry for the MAC address of the fail over in the arp cache of the primary server?

HTH

Rick

HTH

Rick