Let's see if I can explain this coherently, sorry if I don't...
Problem:
What we're seeing is that a machine with multiple IP addresses tied to one NIC can only be reached via one of those IP addresses from a different VLAN. I look on the MSFC arp table, and I only see an entry with a MAC for that one IP address, none of the others. If I add a static ARP entry, I can then reach the other IP addresses from the other VLANs. So communication is possible, the ARP table is just not getting populated automatically.
-HOST A in VLAN A is pointing at the MSFC for it's gateway.
-HOST B in VLAN B is pointing at the MSFC for it's gateway.
-The CSM is in bridge mode. VLAN C is the client side VLAN. VLAN B is the server side VLAN.
-HOST A is trying ping HOST B. HOST A can ping HOST B on it's "main" IP address, but none of the others.
-The ARP table on the MSFC has an entry for the "main" IP address on HOST B, but no entries for any others.
-The ARP table on the CSM does have entries for the "extra" IP addresses on HOST B.
-A static ARP entry for an "extra" IP address on HOST B solves the problem. HOST A can then ping HOST B's "extra" IP address.
My thoughts:
The ARP table on the MSFC is not getting populated automatically from the CSM. As I see it, this is because HOST B is in VLAN B, which only has an interface on the CSM. The arp replies are going to the CSM successfully, but aren't getting to the MSFC because there is no gateway or route defined for VLAN B on the CSM.
The reason that anything at all works is that the hosts in VLAN B are initiating communication outbound to their gateway on the MSFC, so it's getting their MAC addresses that way. When a machine has multiple IP addresses, and it doesn't use them to communicate outbound, the MSFC doesn't learn the MAC for those addresses because the ARP replies are going to the CSM which isn't sharing.
Hopefully that makes sense, and it also makes sense why I'm thinking it's the lack of a gateway entry. Thanks for your help.
