cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
268
Views
0
Helpful
2
Replies

Bizarre flooding issue

james4627
Level 1
Level 1

This may get rather involved and I hope I can get the setup clear in words...

I am experiencing a strange flooding issue during stress-testing of encryption devices. The devices (VRRP groups) are set up back-to-back on the red side and have iperf traffic generating PCs on the green sides. A 2950 switch connects the encryption device, iperf PC and a cisco 2621 on the green side... The 2621 would usually route between VLANs, but in this test setup it only serves as a "known good" address for the encryption device. The VRRP-similar redundancy protocol on the devices use the 2621 as a heartbeat to make sure the network is still there. The traffic flows from a "local" iperf PC, is encrypted by a encryptor group, then unencrypted by the remote group, and received directly on that remote lan by the "remote" iperf PC...

Now to the problem... After about a day, things get strange. Suddenly, and for no apparent reason, the traffic is no longer sent directly from the the encryptor to the iperf PC (same (v)lan), but is flooded out every port in the vlan. Since the 2621 is also there, it gets choked by this rather massive udp stream and no longer answers the heartbeat requests from the VRRP group. Of course the network breaks down... This takes about two minutes to recover, and happens at TEN MINUTE INTERVALS, three or four times. After which there are about two hours of stability, and the process starts over again.

I have spent close to a week recreating the problem, sniffing packets, and troubleshooting. Particularly the packets going out to the router port seem fine, i.e. source/destination macs and IPs are that of the encryptor to the PC. It would seem logical that the switch is to blame, but I have changed out switches and IOS... And the 10 minute interval every two hours does not sound like a switch process... The encryptors DO have a ten-minute ARP cache flush...

Anyone experienced this before or have an idea?? Any help is greatly appreciated!

2 Replies 2

Aaron Harrison
VIP Alumni
VIP Alumni

Hi

It's difficult to say exactly what's going on but it sounds much like the switch is 'to blame' as the destination MAC must have been aged out of the mac-address-table...

Try setting your switch to have a longer aging time -

mac-address-table aging time 14400 vlan 1

Aaron

Please rate helpful posts...

Aaron Please remember to rate helpful posts to identify useful responses, and mark 'Answered' if appropriate!

That would seemlike a logical explanation, however the traffic is bidirectional, meaning that the switch should not flush the iperf machine's MAC as it is constantly sending frames... Besides, I checked the MAC table during an acute phase of the problem, and all was as it should be.

Even though it seems irrelevant, I will up the aging time and see if there is a difference... Thanks Aaron.