SLB Question

Answered Question
Mar 9th, 2010

Please read these 2 short paragraphs from the same chapter of a Cisco press book regarding data center architectures.

SLB Router Mode


The servers typically use the SLB inside address as their default gateway. As reply traffic
from the server to the end user passes through the SLB, the SLB changes the server’s IP address
to the appropriate VIP address.
Therefore, the end user has no way of telling that
there is an SLB device in the path, nor does the end user see the IP address of the real
server.

SLB One-Armed Mode

Inbound end-user traffic is routed to the VIP on the SLB device. The SLB device then
translates the IP destination address to a physical server IP address and forwards the traffic
to the physical server, the same as it does in routed mode.
The main difference is that
return traffic must be forced to go to the SLB device so that the source IP address of traffic
from the physical server can be translated back to the VIP that the end user device
thinks it is communicating with.

How is that a difference from routed mode? From the authors own description of the two architectures -- routed SLB and one (or two) armed mode -- the return traffic from the physical server goes back through the SLB and gets NAT'ed to the server farm VIP address.

What am I missing?

Thanks

I have this problem too.
0 votes
Correct Answer by Jon Marshall about 6 years 9 months ago
 

What do you mean that the SLB will be sitting on another vlan? Lets talk interfaces because describing the whole device lacks necessary detail, right?

I guess you mean that the single interface on the SLB that is used for the hair-pinned traffic is sitting on vlan 12. Correct?

Is it a L2 or L3 interface? Im assuming that since the SVI for vlan 12 is on the MSFC that the interface on the SLB is L2.

What about the serverfarm VIP? Is the VIP bound to any interface or is it just floating, so to speak? Is it on VLAN 12?

I guess what I am missing are the details regarding the traffic flow and how each interface is configured.

if you have sample configurations for a router/LB/switch that is using SLB in one armed mode, that would be great....

Thanks

I don't have a config but lets assume we are dealing with a standalone load-balancer because in effect it doesn't really matter.

In one-armed mode the load-balancer (LB) will be on it's own dedicated vlan eg vlan 12. There would be a L3 SVI on the MSFC for vlan 12 and the LB would have one interface connecting to a port on the switch. This interface would have an IP address from the vlan 12 subnet. So yes in effect the traffic is hairpinned out of this interface.

So traffic comes from a client. It arrives at the 6500 on vlan 10. The packet is destined to a VIP. The MSFC routes the VIPs to the LB interface IP address so the LB is in effect a L3 next-hop from the MSFC.

The packet arrives at the LB. The LB then selects a server and NATs the VIP address to the real server address. It then NATs the client IP address to an address that the MSFC routes to the LB. The packet is then sent back out of the LB interface to the SVI for vlan 12 on the MSFC. The MSFC then routes this onto the server vlan which is vlan 11.

The return flow is from server to vlan 11 SVI, then to LB on vlan 12 which does the necessary NAT on both src/dest, then back to the SVI for vlan 12 on the MSFC and from there the packet is routed back to the client.

Now i have to be honest all this explanation has been done from memory and it was with a CSM-S module in a 6500 switch. But that is virtualised to an extent ie. there are no physical interfaces on the CSM-S so the same should apply to the SLB. Whether or not you actually configure it the same way, i'll need to check so when i get a moment i'll have a read of the SLB docs.

Jon

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (4 ratings)
Loading.
Jon Marshall Tue, 03/09/2010 - 13:46

What am i missing ?

Oh don't tempt me, where to start

Routed mode = destination address is VIP from client and changed to real server IP address by SLB device.

One armed mode = destination address is VIP from client and changed to real server IP address by SLB device.

So far the same. But it is what happens to the client addresses that is significant.

Routed mode -

clients -> (VIPs)  SLB -> real servers

real servers default-gateway is set to SLB ie. the return traffic from the servers must go to the SLB device which then routes the packets back to the clients.

One armed mode - (apologies for crap diagram)

                     SLB

                    (VIPs)

                       ^

                       |

                       |

clients -->                <---- real servers

notice how the SLB is not in the path between the clients and the servers. Traffic from the clients go to the SLB  because the SLB handles the VIPs. But when the servers send the packets back to the clients they do not need to go to the SLB ie. they can route straight back to the clients. In routed mode this isn't possible, see above diagram of routed.

But traffic cannot be routed straight back to the clients because the SLB needs to see both traffic flows. So the SLB NATs the source IP addresses of the clients to an address so that the servers see the source address as coming from the SLB. And as your example says the return traffic must go back to the SLB so it can NAT the real address of the server back to the VIP. The servers then send the return traffic to the SLB which then NATs it back to the real client address.

Key thing to understand is that with one-armed mode the SLB device is not in the path between the clients and servers, it is in effect hanging off to the side.

Hope this makes sense.

Jon

lamav Tue, 03/09/2010 - 14:43

Jon, I'll get to the business of giving you a zing right back in a minute, but first lets get SLB out of the way


OK:


Router Mode SLB:


Client traffic is destined for the advertised server farm's VIP address, typically a public address routable over the Internet (lets assume the client is somewhere on the Internet). CORRECT?


The server farm's VIP resides on the SLB, so, given the load balncing and NAT policies configured on the SLB, it will perform a destination NAT, replacing the VIP address with the real (private) address of the actual server that it decides to forward the client request to. CORRECT?


GIVEN THAT THE default gateway on the physical server is the SLB's inside address, the traffic will be forwarded to the SLB, which will then perform a source NAT, replacing the server's real IP address with the VIP.CORRECT?


So, in routed mode, there are 2 floating addresses, so to speak: the advertised public VIP address that the client sends traffic to, and the default gateway on the private subnet that the actual servers sit on. CORRECT?

One Armed Mode:


Client traffic is destined for the advertised server farm's VIP address, typically a public address routable over the Internet (lets assume the client is somewhere on the Internet). CORRECT?


The server farm's VIP resides on the SLB, so, given the policy configured on the SLB, it will perform a destination NAT, replacing the VIP address with the address of the actual server that it decides to forward the client request to. CORRECT?


Now, I understand the connectivity implications of one -or two-armed mode, as you described, but isnt that path the return traffic takes really a function of the server's default gateway, and not necessarily how things are physically connected? In other words, its the logic flow, rather than the physical connectivity, that truly matters. CORRECT?


I mean, where else would the server send its return traffic if not to the SLB? Remember that the routing engine that provides L3 isolation between the advertised public VIP address and the local private LAN is the SLB itself, so the server could not have any other device besides the SLB as its default gateway. CORRECT?


If the SLB were configured in one-armed BUT bridged mode, then, yes, the default gateway for the servers will be the routing engine, which in this case is NOT the SLB because it is in switched/bridged mode and the inside and outside interfaces are bridged together and belong to the same internal subnet. CORRECT?

Jon Marshall Tue, 03/09/2010 - 15:17

Victor

Router SLB mode

===============

Client traffic is destined for the advertised server farm's VIP address, typically a public address routable over the Internet (lets assume the client is somewhere on the Internet). CORRECT?

Correct but it could just as easily be an internal web server farm in the data centre accessed buy the clients in the Enterprise.

The server farm's VIP resides on the SLB, so, given the load balncing and NAT policies configured on the SLB, it will perform a destination NAT, replacing the VIP address with the real (private) address of the actual server that it decides to forward the client request to. CORRECT?

Correct.

GIVEN THAT THE default gateway on the physical server is the SLB's inside address, the traffic will be forwarded to the SLB, which will then perform a source NAT, replacing the server's real IP address with the VIP.CORRECT?

Yes assuming you are talking about the return traffic from the server.

So, in routed mode, there are 2 floating addresses, so to speak: the advertised public VIP address that the client sends traffic to, and the default gateway on the private subnet that the actual servers sit on. CORRECT?

Not sure what you mean about floating addresses. The VIP is a floating address if you like but the default-gateway isn't floating, it will be attached to a L3 SVI which will be the vlan the servers sit on.

One Armed mode

==============

Client traffic is destined for the advertised server farm's VIP address, typically a public address routable over the Internet (lets assume the client is somewhere on the Internet). CORRECT?

Correct as before with same proviso.

The server farm's VIP resides on the SLB, so, given the policy configured on the SLB, it will perform a destination NAT, replacing the VIP address with the address of the actual server that it decides to forward the client request to. CORRECT?

Correct.

Now, I understand the connectivity implications of one -or two-armed mode, as you described, but isnt that path the return traffic takes really a function of the server's default gateway, and not necessarily how things are physically connected? In other words, its the logic flow, rather than the physical connectivity, that truly matters. CORRECT?

It is a function of the servers default-gateway but see below for an explanation of where the servers default-gateway is with one-armed mode.

I mean, where else would the server send its return traffic if not to the SLB? Remember that the routing engine that provides L3 isolation between the advertised public VIP address and the local private LAN is the SLB itself, so the server could not have any other device besides the SLB as its default gateway. CORRECT?

Incorrect. The vlan that the SLB resides on in one-armed mode is neither the vlan the clients come in on nor is it the vlan that the servers sit on. In one-armed mode the SLB sits on it's own vlan. Now because the SLB "owns" the VIPs the traffic from the clients gets routed by the MSFC to the SLB. But when the return traffic comes from the server the default-gateway of the server vlan is not the same vlan the SLB is on. So in effect the server traffic could be routed by the MSFC straight to the client.

To ensure that the MSFC routes the return traffic back to the SLB the SLB NATs the client IP addresses to an address that the MSFC will route to the SLB. You don't need to do this with routed mode because as you say the default-gateway of the server is the SLB.

So you can't have one-armed bridged mode as it doesn't make sense. Both routed and bridged mode use 2 vlans, one on either side of the SLB. One-armed mode uses only one vlan.

Jon

lamav Fri, 03/12/2010 - 12:37

Jon:

Sorry for taking so long to get back to you. Been an insane week...

Anyway, so in one armed mode, how are the client-facing and server-facing interfaces configured? What do you mean by one vlan? Can you elaborate more on how the interfaces are configured?

So, in one armed mode, the servers default to the msfc?

Thanks

Victor

Jon Marshall Fri, 03/12/2010 - 13:02

lamav wrote:

Jon:

Sorry for taking so long to get back to you. Been an insane week...

Anyway, so in one armed mode, how are the client-facing and server-facing interfaces configured? What do you mean by one vlan? Can you elaborate more on how the interfaces are configured?

So, in one armed mode, the servers default to the msfc?

Thanks

Victor

Victor

No problem.

In one-armed mode the client and server vlan interfaces are just normal L3 SVI's on the MSFC.  So for example you would have -

client vlan 10        -> MSFC       ->   server vlan 11

192.168.5.0/24                                192.168.6.0/24

there is nothing stopping these 2 vlans routing between each other. The servers would have their default-gateway set to the L3 vlan 11 SVI.

The SLB device would be sitting on an entirely different vlan eg. vlan 12 and vlan 12 would also have a L3 SVI on the MSFC.  So traffic that is meant to be load-balanced has to be "directed" to the SLB.  It's also important to note that with routed and bridged mode the SLB effectively has 2 interfaces ie. outside where client traffic arrives and inside where server vlan is sent to. With one-armed mode client and server traffic goes in and out the same interface.

So as described before traffic from the clients is routed to the SLB on vlan 12 because the SLB is responsible for the VIPs. And then the SLB NATs the client IPs to ensure that when the server sends the return traffic it doesn't just get routed via the MSFC straight back to the client. Instead it gets sent back to the SLB on vlan 12 which then NATs the client address and sends it back to the MSFC to be routed back to client.

So with one-armed mode the source and destination IP addresses have to be Natted.

Jon

lamav Fri, 03/12/2010 - 15:26

In one-armed mode the client and server vlan interfaces are just normal L3 SVI's on the MSFC.  So for example you would have -

client vlan 10        -> MSFC       ->   server vlan 11

192.168.5.0/24                                192.168.6.0/24

there is nothing stopping these 2 vlans routing between each other. The servers would have their default-gateway set to the L3 vlan 11 SVI.

The SLB device would be sitting on an entirely different vlan eg. vlan 12 and vlan 12 would also have a L3 SVI on the MSFC.  So traffic that is meant to be load-balanced has to be "directed" to the SLB.  It's also important to note that with routed and bridged mode the SLB effectively has 2 interfaces ie. outside where client traffic arrives and inside where server vlan is sent to. With one-armed mode client and server traffic goes in and out the same interface.

So as described before traffic from the clients is routed to the SLB on vlan 12 because the SLB is responsible for the VIPs. And then the SLB NATs the client IPs to ensure that when the server sends the return traffic it doesn't just get routed via the MSFC straight back to the client. Instead it gets sent back to the SLB on vlan 12 which then NATs the client address and sends it back to the MSFC to be routed back to client.

So with one-armed mode the source and destination IP addresses have to be Natted.

=============================================================================================================

What do you mean that the SLB will be sitting on another vlan? Lets talk interfaces because describing the whole device lacks necessary detail, right?

I guess you mean that the single interface on the SLB that is used for the hair-pinned traffic is sitting on vlan 12. Correct?

Is it a L2 or L3 interface? Im assuming that since the SVI for vlan 12 is on the MSFC that the interface on the SLB is L2.

What about the serverfarm VIP? Is the VIP bound to any interface or is it just floating, so to speak? Is it on VLAN 12?

I guess what I am missing are the details regarding the traffic flow and how each interface is configured.

if you have sample configurations for a router/LB/switch that is using SLB in one armed mode, that would be great....

Thanks

EDIT

OK, I just found a pretty nice document and I see now whats going on. The LB's interface is a L3 interface with an IP address that sits on the same vlan as the VIP and the SVI on the MSFC.

So, here is the data flow: Correct me where I am wrong.

1. Client traffic arrives at the client-facing interface on the routing engine (msfc, separate router, L3 switch...whatever). The SA=client IP - DA =serverfarm VIP, in other words, the application VIP. This is what is advertised to the rest of the application-using world.

2. The routing engine will have a directly connected route for the VIP. What I am thinking is that since the VIP is not bound to an interface, there is no L2 address associated with it to perform L2 forwarding to the VIP address. Therefore, the only way for the routing engine to forward the packet to the VIP is to use a route from the RIB. Lets say a static route to the effect of ip route 192.168.7.100 255.255.255.255 192.168.7.1, where .100 is the VIP and .1 is the L3 interface of the SLB, which is an address that is in the routers ARP table and does have a L2 address binding.

The routing engine's routed interface will be, say, 192.168.7.2...(this is the VLAN 12 you mentioned in your example).

3. The LB will receive the packet through the one interface it has connected and it will first NAT the SOURCE address (client address) to an address that it owns. This would probably be on vlan 12 also. Lets say 192.168.7.3.SA=NAT'ed address - DA = VIP address.

4. The LB will use its loadbalancing policy to determine which real servers it must forward the packet to. This will be the 192.168.6.0 subnet from your example. The LB will forward the packet out its single interface and rely on the routing engine to forward the packet to the real server.SA=NATed address - DA = real server address.

5. The router will use L2 forwarding to forward the packet to the real server since the router will have a directly connected route to the server network

6. The server responds and places its own address as the SA and the NATed address as the DA. This packet is forwarded to the router.

7. The router will use L2 to forward the packet to the LB's NAT address (192.168.7.3).

8. The LB will receive the packet with an SA of the server and a DA of the NAT address. The LB will replace the DA with the client's original address address by doing a DESTINATION NAT this time. AND THEN the source address will be replaced by the VIP address using the typical LB mechanism. So, SA=VIP and DA =client addre
ss.

EDIT

Correct Answer
Jon Marshall Fri, 03/12/2010 - 15:51
 

What do you mean that the SLB will be sitting on another vlan? Lets talk interfaces because describing the whole device lacks necessary detail, right?

I guess you mean that the single interface on the SLB that is used for the hair-pinned traffic is sitting on vlan 12. Correct?

Is it a L2 or L3 interface? Im assuming that since the SVI for vlan 12 is on the MSFC that the interface on the SLB is L2.

What about the serverfarm VIP? Is the VIP bound to any interface or is it just floating, so to speak? Is it on VLAN 12?

I guess what I am missing are the details regarding the traffic flow and how each interface is configured.

if you have sample configurations for a router/LB/switch that is using SLB in one armed mode, that would be great....

Thanks

I don't have a config but lets assume we are dealing with a standalone load-balancer because in effect it doesn't really matter.

In one-armed mode the load-balancer (LB) will be on it's own dedicated vlan eg vlan 12. There would be a L3 SVI on the MSFC for vlan 12 and the LB would have one interface connecting to a port on the switch. This interface would have an IP address from the vlan 12 subnet. So yes in effect the traffic is hairpinned out of this interface.

So traffic comes from a client. It arrives at the 6500 on vlan 10. The packet is destined to a VIP. The MSFC routes the VIPs to the LB interface IP address so the LB is in effect a L3 next-hop from the MSFC.

The packet arrives at the LB. The LB then selects a server and NATs the VIP address to the real server address. It then NATs the client IP address to an address that the MSFC routes to the LB. The packet is then sent back out of the LB interface to the SVI for vlan 12 on the MSFC. The MSFC then routes this onto the server vlan which is vlan 11.

The return flow is from server to vlan 11 SVI, then to LB on vlan 12 which does the necessary NAT on both src/dest, then back to the SVI for vlan 12 on the MSFC and from there the packet is routed back to the client.

Now i have to be honest all this explanation has been done from memory and it was with a CSM-S module in a 6500 switch. But that is virtualised to an extent ie. there are no physical interfaces on the CSM-S so the same should apply to the SLB. Whether or not you actually configure it the same way, i'll need to check so when i get a moment i'll have a read of the SLB docs.

Jon

lamav Fri, 03/12/2010 - 16:10

Jon, please read my post where I lay out the data flow. Please do a sanity check on it.

meanwhile, will check out your example, too, for further understanding and perhaps to fill gaps.

Jon Marshall Fri, 03/12/2010 - 16:17

Victor

Yep, your traffic flow looks pretty much spot on. Only thing i'm not sure is whether there is a route on the MSFC for the VIPs or whether proxy-arp as on firewalls would work in this example ie. the router has a directly coinnected interface in vlan 12 which is the VIP vlan and so it simply arps out for the VIP and the load-balancer responds.

Doesn't make too much difference either way.

Jon

lamav Fri, 03/12/2010 - 16:33

Jon, thanks buddy.

I hate to be a pain and demand such detail from someone. Given my lack of experience in this area and my inability to lab it up - without having an LB at my disposal -- it makes it hard to nail it down, especially when there are so many flavors of archiecture, design and data flow when doing SLB.

Thanks again....

Jon Marshall Fri, 03/12/2010 - 16:41

Victor


I hate to be a pain and demand such detail from someone. Never stopped you before

Seriously though, we've know each other long enough on these forums that you don't need to apologise. Like i said before it helps me get things clear in my mind as well.

Jon

lamav Sun, 03/14/2010 - 16:02

"Like i said before it helps me get things clear in my mind as well."

And God only knows how much fog is up there...

Hope you didn't think I forgot about you...

lamav Sat, 03/27/2010 - 14:19

Jon:

I was reading this document and I thought of you and this thread.

Data Center Blade Server Integration Guide

It has an excellent example of deploying a CSM (with a FWSM, too, by the way) in one-armed mode, including the necessary configurations!

This is exactly what I was looking for...this is better than sex.

I know you dont really need this document for yourself, but in the event that another pain in the a-- asks you the same question, you can refer him/her to it.

Victor

Actions

This Discussion