Icmp between two nodes on same subnet partial fail

UHansen1976 · ‎10-27-2009

Hi,

I'm experiencing a rather strange problem, which I haven't encountered before. I have two AIX-nodes on the same subnet/vlan, both configures with the correct subnet mask. This vlan is directly connected to fwsm and no icmp packets are permitted through the fw, only a few management-ports are.

The two AIX-nodes have two interfaces, one for client traffic and one for management-traffic. When pinging between the two "client"-interfaces, everything works fine. ICMP can go on forever, withouy any packet loss.

But once I try to ping between the two managment-interfaces, I get anywhere from 25% to 80% packet loss. Sometimes the icmp goes on for a good period of time without any interruption, but suddently and for no apparent reason, it fails.

The two aix-nodes are virtual, running on IBM Power6 hardware. The Power6 is connected to the network via an 2x1Gbps etherchannel, and the two virtual adaptors on the Power6 share one physical adaptor.

I've done a trace on the switch which shows, that everytime the icmp fails, the gateway-address for the subnet responds with a 'ICMP Destination unreachable (Network unreachable)'. I just can't seem to understand, why the icmp packets would even be forwarded to the gateway, since both nodes reside within the same vlan. I'm not an authority on AIX, but so far I can't any mistakes in the ipconfig. The addresses and subnet masks are correctly configured and there are no hostroutes on the aix-nodes that would interfere in anyway.

Any suggestions?

Thanks

/Ulrich

lgijssel · ‎10-27-2009

Did you check for duplicate mac addresses?

Sounds very much like this is your issue.

regards,

Leo

UHansen1976 · ‎10-27-2009

Hi Leo,

Thanks for replying.

Well, a 'sh mac-ad int po1' shows the following:

Vlan Mac Address Type Ports

---- ----------- -------- -----

300 001a.6429.f195 DYNAMIC Po1

301 001a.6429.f195 DYNAMIC Po1

Although the same mac pops up in both vlans, there's an explanation for that. Po1 is attached to a physical NIC on the AIX, which is subsequently sliced into two virtual NIC's. And the virtual NIC in vlan301 is the one experiencing the problem. Also, this setup is used widely within our infrastructure, but only hosts on a few vlans are experiencing the problem.

lgijssel · ‎10-27-2009

Then please check your infrastructure to verify your native vlan settings.

If a trunk port somewhere has the wrong native vlan, frames may 'leak' to another vlan (presumably 301 here) and cause this problem.

regards,

Leo

UHansen1976 · ‎10-27-2009

Hi Leo,

I see a difference in the port-configuration between Po1 and the two switch-uplink ports. The uplink ports are configured with vlan1000 as native vlan, but Po1 uses default native vlan.

I'll try and change the portconfiguration and see if it help.

Thankx

/Ulrich

iyde · ‎10-28-2009

Hi Ulrich.

The native VLAN shall match for both ends of a connection. This means that if you have defined your trunk on the uplink ports with native VLAN as 1000 in both ends, that's fine. But if your AIX has native VLAN set to 1, then your Po1 shall ahve native VLAN 1 as well.

Actually, if you wanted to, you could have different native VLAN on every trunk, if you wanted to.

HTH - og hav en god dag!

Ingolf