MS NLB with WAN Network

[Hubert Mayr-Kneilling]

Hello,

we have a scenario (see attached drawing) with 2 servers connected to two Cisco 6509. These servers ar configured for MS NLB. We have done the necessary configs on the 6509 (static arp, and static multicast) according to this document:

http://www.cisco.com/en/US/products/hw/switches/ps708/products_configuration_example09186a0080a07203.shtml#mm

Now we can reach the virtual ip address (10.x.x.3) from all VLANs in the location 1, because the two 6509 are default gateway for these vlans.

We would like to reach the virtual ip address also form location 2, which is behind a mpls-cloud (hosted by a provider).

Server 1 and Server 2 can be reached by location2, but not the virtual address.

Do we need static entries in the mpls-cloud, on the location2-routers or is this scenario not possible at all.

regards

HMK

[phiharri]

Greetings Hubert,

Let's review why these configurations are necessary to determine what other changes are required on your network.

• Static ARP bindings must be applied on any devices which route traffic from a different subnet into the subnet with NLB servers. This is because Cisco devices will not honour an ARP reply associating a unicast IP with a multicast Ethernet MAC address.
• Static CAM entries are recommended (but not mandatory) on all switches in the subnet(s) containing NLB servers to avoid flooding traffic destined for the cluster on unnecessary ports. This is because by default NLB servers do not generate IGMP reports which are the primary mechanism switches use to constrain multicast traffic. Newer Windows versions can be enabled for IGMP, but this may not work for all switches as some models (notably Catalyst 3750 and similar) forward multicast at L2 based on destination IP (which we can't determine from the NLB IGMP reports) rather than MAC. For a network with a small number of routers touching the NLB subnet I recommend staticly configured entries.
  

Which extra config is needed at Location 2 depends on the type of WAN connection you have. If this is a transparent Layer 2 service which extends the NLB subnet between locations then gateway(s) at Location 2 will require the static ARP binding, but the MPLS routers likely don't need extra config. If the connection between the MPLS routers and 6500s at Location 1 is L3 then the MPLS routers don't require a static ARP binding as they aren't routing traffic into the NLB subnet.

If you're unsure, check if there hosts at Location 2 with IPs in the  same subnet as the NLB servers, whether ARP gets resolved for the  virtual IP, or ask your MPLS provider.

P routers within the MPLS cloud won't require any configuration as forwarding decisions there are made based on the MPLS label stack, not the underlying IP or Ethernet destination.

Hope this helps,

/Phil

[Hubert Mayr-Kneilling]

Hello Phil,

thanks for your detailled answer.

MPLS network is providing L3 service, so Location 2 is a different subnet. MPLS-Router at location 1 routes all traffic destined to location 1 to the 6500s at location 1. Nevertheless pinging to the virtual address stops at the MPLS-Router at location 1. (Traceroute) while pinging the physical addresses works.

We assigned the mac-address of the virtual ip to the interface where the server is connected and to the interface of the interlink of the 6500s. Do we have to assign the mac-address also to the interfaces connected to the MPLS-Router to get this work?

regards

HMK

[phiharri · Accepted solution]

Hi Hubert,

You don't need to point the static CAM towards the MPLS routers as that's the opposite direction that we need to send this traffic.

It sounds like the MPLS routers are directly connected to the VLAN with NLB servers. Are you sure that the connection between MPLS routers and 6500s is Layer 3 - are the interfaces on the 6500 side configured with an IP address?

If you traceroute a physical server address from Location 2, do you see three final hops at Location 1 (MPLS router, 6500 SVI, server)? If you just see MPLS router followed by the server, you'll need to ask the MPLS provider to add static ARP bindings on those routers.

/Phil

[dmitry_gudkov]

Hello, Phillip.

Additional question:

Must we add the static CAM entries for the Etherchannel ports if we have an Etherchannel link between Catalysts 6500 in this topology?  Will we get a loop with flooding the ARP packet out of these ports?

With hope for help,

Dmitry

[Hubert Mayr-Kneilling]

Hello Dmitry,

with my configuration the link between the two 6500 is an etherchannel. There is no problem!

regards

HMK

[phiharri]

Exactly, the static CAM entries should cover every L2 path to reach the physical servers. Regardless of the static CAM, traffic is discarded on spanning-tree blocked ports preventing a loop.

/Phil

[dmitry_gudkov]

It is possible that I don't understand the scenario fully.

But it can happen so that these servers will be connected to different Catalysts (for example, result of network adapters fault tolerance working). In this case we get two L2 paths to reach the physical servers at the same time. Packets will be going back and forth between Catalysts.

Best regards,

Dmitry.

[phiharri]

Hi Dmitry,

I think I understand your point that with two server access switches you will have a static CAM entry on each switch which includes the interface to the other switch, so it may seem that traffic would loop. But regardless of a static CAM entry, at L2 we never forward frames back out the interface they arrived on, nor on STP blocking ports.

Hope this clarifies!

/Phil

[dmitry_gudkov]

Hello, Phillip, Hubert!

Thanks you for opening and discussing the theme.

Now I'll employ the knowledge base for my practice.

Good luck!

[Hubert Mayr-Kneilling]

Hello Phil,

you are right, the MPLS-Router is connected to the VLAN where the servers are.

So I instructed the provider to implement the appropriate entries. But this still lasts. So hopefully this will solve the problem.

Thanks again for your support.

[Hubert Mayr-Kneilling]

Hello again,

sorry for comming back with this problem. I thought it could be solved by adding the appropriate commands in the MPLS-Routers.

The MPLS-Provider could set the "arp  x.x.x.x H.H.H ARPA" command but not the mac-adress static command.

I tested the commands with a 2821 router:

The arp-command works well and you can find it in the config

the command "mac-address-table static H.H.H interface gi0/0" is accepted in config-mode but there is nothing in the configuration or with the "show mac-address" command.

Any ideas?

regards

HMK

[phiharri]

Hey again Hubert,

Did you still have any connectivity problems to the NLB servers across site after adding the static ARP binding?

Unless there is a bridge-group or switching module on the MPLS router then you don't need a static CAM entry there, just the ARP. What model of router is used in the production network?

Cheers,

/Phil

[Hubert Mayr-Kneilling]

Hello Phil,

yes, I still have connectivity problems. On the MPLS-Routers I can see the arp entry, but not the mac-address entry.

Traceroute from remote site stops at the MPLS-Router.

Traceroute inside LAN (from different VLAN) is ok. LAN-Router is Catalyst 6509 with SUP2

MPLS-Routers are 3845 with 2GE and 4 FE Interfaces. One of the GE Interfaces is connected to the 6509.

regards

Hubert

[phiharri]

Hi Hubert,

Strange, but I don't believe you'll need the CAM entry on the MPLS router as there is just one interface towards your core switches and it's configured with an IP address, right?

Can you ping the NLB cluster from the MPLS router?

Cheers,

/Phil