Add secondary CSS to DR site for failover

Unanswered Question
May 30th, 2007

I have a CSS configured for web server failover.

We have two servers providing different fuctions that failover to backup servers at the DR site if one or the other server fails.

This is all working fine.

We want to add a second CSS at the DR site to provide access to our web servers if the Main site gets wiped out.

The DR site would also provide connectivty to the web servers if there is a loss of Internet connectivity at the main site.

How much configuration would be needed to my existing config to allow for this?

I would also have to configure the secondary CSS to be the failover unit.

Are there any examples on how to do this?

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (18 ratings)
Gilles Dufour Thu, 05/31/2007 - 05:37

you can use a CSS at the DR site, but to select one site or the other, you need to act at DNS level and in this case, what you want is a GSS [global site selector].

The CSS has dns functions but we recommend to use a GSS instead.


Gilles Dufour Thu, 05/31/2007 - 10:09

this method is definitely not supported anymore.

You should go for GSS solution or zone based dns on the CSS - not rule based as shown in the old example mentioned here.


wilson_1234_2 Thu, 05/31/2007 - 14:11

I have services set up on the CSS now, with VIP addresses configured for the two internal web servers.

The externally hosted DNS names resolve to the VIP addresses and then redirected to the Primary server.

If that is now I have sorryServers configured as the backup.

This works great, so my questions are:

Will config for the CSS I have working now, require a drastic change to add the second one at the DR site and implement GSS solution or zone based dns?

Where are some examples of the GSS solution or zone based dns?

acomiskey Thu, 05/31/2007 - 14:18


Why is that not supported? And what exactly does it mean to not be supported anymore?

Gilles Dufour Fri, 06/01/2007 - 00:02

it means that if you one day encounter an issue with dns on the CSS, we might tell you to move out of this solution instead of fixing it.

This is the same for the CSM and the new loadbalancer, ACE, will not offer this feature.

So, if you have to deploy a new setup, you should definitely go for GSS.


wilson_1234_2 Fri, 06/01/2007 - 05:00

I have the CSS devices at the moment and they were recently purchased, so I have to use what I have.

So, are there any configuration examples for what i want to do?

acomiskey Fri, 06/01/2007 - 05:54

Wilson, the link I posted contains the example for what you want to do. I guess it is unsupported but that doesn't mean it won't work. I have it running myself. Let me know if you need help setting it up.

Gilles, I'm not familiar with GSS. Would this work with another CSS or do you need a pair of GSS's? Can Wilson use his existing pair of CSS and use another solution than the one referenced above?

wilson_1234_2 Fri, 06/01/2007 - 06:25


I guess I have these questions:

I have the existing CSS configured and it is working as a stand alone and is redirecting web requests to the primary server using services, if that is down, it redirects to the secondary server across our MPLS to the DR site.

1. Can I just add the neccesary DNS information to my existing congig?

2. How do you have the failover to the second CSS working if your main site Internet connection is down? What redirects the client DNS request to the second CSS.

3. I guess question 2 is also how does CSS1 failover to CSS2?

acomiskey Fri, 06/01/2007 - 06:47

1. not sure what you mean but what good would that do anyway if the css was unavailable? It can only redirect to the secondary site if it is actually reachable.

2. You will have 2 NS records and 2 A records defined at authoritative dns. Something like this...





Then at the CSS level each will contain an A record for the domain

Primary CSS - A

Secondary CSS - A

3. CSS1 and CSS2 are in constant communication with eachother through an app session. This is how each knows which services are available etc. and which address to return. If the request lands at css1 and the primary service is up, it knows to return it's local vip address. If the request lands at css2, it knows through the app session whether or not the primary service is up, if it is not and it's local service is up, it will return it's local vip address, therefore creating a backup. Hope that makes sense.

wilson_1234_2 Fri, 06/01/2007 - 08:08

I was talking about:

Right now, the setup is only one CSS at the main site, if everything (Internet connectivity) is up and the HQ server goes down, the CSS redirects the client web requests to the secondary server.

I want to implement the solution you have in case the Internet or the building get wiped out the web requests are redirected to the DR site.

Now, another question:

1. Is this common practive the Primary secondary entries and MCI (who hosts our DNS) can do this with no problem?

2. How fast is the DNS transition to the secondary record (pretty fast I am thinking).

3. Do the CSS devices need any more configuration added to accomplish this? It looks like with this scenario, you don't.

acomiskey Fri, 06/01/2007 - 09:38

1. I can't imagine having a problem adding/changing your dns records (A, NS, MX etc.)

2. From my experience it is very fast. I can sit next to my server and run an nslookup, and unplug the server, by the time I run another nslookup the address has changed. This of course depends on how your keepalive is setup for the service etc.

3. The only configuration needed is shown in the linked document. You also need the "Enhanced Feature Set" License to run this feature.

wilson_1234_2 Fri, 06/01/2007 - 15:11

I see what you are saying, but I have the CSS configured with two seperate servers.

Outside DNS is resolving names to the VIP address configured. If the service that points to the local server is up, the CSS directs the request to that server.

It works great as with only the primary Internet connection in play, there is nothing configured if the the primary link goes down.

I am not sure if I can modify the existing config to implement this.

My config:

!************************** CIRCUIT **************************

circuit VLAN1

ip address

!************************** SERVICE **************************

service MCI-MCW-backupredirect

type redirect

port 80

keepalive type none

redirect-string ""

ip address


service MCI-MCW-dr

ip address

protocol tcp

keepalive type http

port 80


service MCI-MCW-dr-443

ip address

protocol tcp

port 443


service MCI-MCW

ip address

protocol tcp

keepalive type http

port 80


service MCI-MCW-443

ip address

protocol tcp

port 443



type redirect

port 80

keepalive type none

redirect-string ""

ip address



protocol tcp

port 80

keepalive type http

keepalive uri "/index.asp"

ip address



protocol tcp

port 80

keepalive type http

keepalive uri "/index.asp"

keepalive retryperiod 15

keepalive frequency 15

ip address


!*************************** OWNER ***************************

owner MCI-MCW

content MCI-MCW-http-rule

add service MCI-MCW

primarySorryServer MCI-MCW-dr

balance aca

secondarySorryServer MCI-MCW-backupredirect

vip address

protocol tcp

port 80

url "/*"


owner MCI-MCW-443

content MCI-MCW-https-rule

add service MCI-MCW-lk-443

primarySorryServer MCI-MCW-dr-443

vip address

protocol tcp

port 443




add service

balance aca

protocol tcp

port 80

url "/*"



vip address


!*************************** GROUP ***************************

group MCI-MCW-http-group

add destination service MCI-MCW

add destination service MCI-MCW-dr

vip address

add destination service MCI-MCW-443

add destination service MCI-MCW-dr-443



add destination service

add destination service

vip address


!**************************** ACL ****************************

wilson_1234_2 Tue, 06/05/2007 - 15:14


What happens to the existing DNS records (and Client name resolution) when you make this change to your DNS records?

Do they loose connectivity while the name records converge?

wilson_1234_2 Sat, 06/09/2007 - 07:41

1. Are there any examples that show how this is configured for the app session, and the services.

2. As far as the services go, would the DR config be pretty much the same as the Main site config, which shows the Main site servers as the primary?

3. I am thinking I can leave my services set up as they are already, and add the DNS components to the configs. Is this the case?

4. In the example above, why do I have the A records AND the NS recods in my hosted DNS?

Are the CSS devices having name querys relayed through my hosted DNS?

5. If I make these changes to my hosted DNS, what will happen to my existing client requests while this is propagating?

acomiskey Sun, 06/10/2007 - 07:04

1. There is an example of the app command and services in this doc as well

2. Not really, the DR site config would only contain services at the DR site, the main site config would contain services at the main site. The CSS's know the status of the other site's services through the app session.

3. I wouldn't see why not but I'm no expert.

4. This is because you cannot create an NS record which points to an ip address directly, an NS record must point to a hostname. So therefore you create an NS which points to a hostname which in turn resolves by A name to an ip address. This does nothing more than direct the dns requests to the CSS's.

5. If the CSS is ready to resolve the requests when you make the changes to the hosted DNS, there shouldn't be any downtime. I would test it with a subdomain if you want to try it

wilson_1234_2 Sun, 06/10/2007 - 07:37

My thought for question #2 is that if the main site Internet connectivity dies, I still want the main site servers to be the primary servers.

And allow the failover process to be the same from the DR site as with the Main site.

So, no matter which Internet connection is being used, and if the primary servers are still alive, they will be the first selected.

If the Main site is gone completely, the failover would be seemless to the secondary servers with the same DNS names.

Is this possible?

acomiskey Mon, 06/11/2007 - 06:41

wilson, I'm sure that's possible but we'll need one of the experts here to explain the best way to do it.

I guess you have a direct connection between the two sites?

wilson_1234_2 Mon, 06/11/2007 - 08:06

acomiskey, thanks for the reply.

I think it should work, because I have the standalone CSS doing that right now from the Main site.

If the Primary server is rebooted, it will failover to the secondtary server at the DR site throguh our MPLS connection to DR.

Once the Primary server is back up and the service is active, the Primary takes over again.

The DR site would be doing the same thing really, just reversing which is the primary server in respect to the site.

acomiskey Mon, 06/11/2007 - 08:16

You need a way to tell the backup css that if a request lands on it to prefer the main site if

1. the main server is up across primary connection


2. the main server is up across secondary connection

Take a look at the acl which is written in the backup css in the doc I have been referencing.

wilson_1234_2 Mon, 06/11/2007 - 10:05


I did look at that document and saw the ACL for that.

That brings up another question which is:

What determines which NS that I am going to add to my hosted DNS gets hit first for the web request.

Is it random, and the CSS forwards the request to the prioritized active service?

acomiskey Mon, 06/11/2007 - 10:59

Are you asking which CSS gets hit first? I will have to test and see what happens but the whole idea is that no matter which it lands at, it will resolve to the same ip.

wilson_1234_2 Mon, 06/11/2007 - 11:31


But if I have the DR site as a backup, there is no way for it to have the same IP correct?

I thought that was the idea was to have DNS configured to resolve the name to which IP address is active.

The CSS devices act as a DNS relay,

If the Main site is live then, is

if the main site is dead then, is

If the Main site is down, the name gets resolved to the DR site subnet,


If this is so, than what happens if the main site is dead and the NS1 (CSS1 in hosted DNS) cannot be contacted?

Does hosted DNS automatically direct requests to NS2 (CSS2 in hosted DNS)?

Sorry if I am not making any sense.

acomiskey Mon, 06/11/2007 - 11:47

I'm sorry, should have been more specific.

"But if I have the DR site as a backup, there is no way for it to have the same IP correct?"

-What I was trying to say is no matter where the dns request lands, or, the css's will resolve the requested domain name ( to the same address. If the main site is up, both css's will resolve the domain name to If the main site is down, both css's will resolve to

You asked which css the request lands at, I was just trying to say that it doesn't matter because they will resolve the request to the same address.

"If the Main site is down, the name gets resolved to the DR site subnet, correct?"


"If this is so, than what happens if the main site is dead and the NS1 (CSS1 in hosted DNS) cannot be contacted?"

-Yes, at least that is what I've seen. You are making sense. If you look at your primary zone configuration, you can also assign a priority to the css NS entries. Mine right now are both priority '0'. It appears that most requests land at the primary site, but some do land at the backup. My guess is it is somewhat round robin'd. I'm no dns expert so I can't say for sure. But yes, for this to work, I can completely wipe out my primary site and my requests will then land at CSS2.

wilson_1234_2 Mon, 06/11/2007 - 12:15


I thought I was getting it all mapped out in my coconut, then suddenly I thought I had it wrong.

Sorry for all of the questions, but I like to understand as much as I can.

Where the heck is gdufour? I don't think he likes to get too involved with amatures like me.

wilson_1234_2 Mon, 06/11/2007 - 13:04

Well, I have to go to the DR site and get the CSS.

I am going to leave the main site one alone for now since it is in service on the network/Internet.

Once I have the DR one configured, I am going to test to make sure it will handle DNS requests, then I am thinking I should be able to install the DR CSS, set up the apllication between the two and DNS on the Main site CSS.

Once I am sure the two are communicating properly in service on the Internet, I will check the DNS resolution.

If they both work, I should be able to throw the switch on the Hosted DNS config changes.

acomiskey Tue, 06/12/2007 - 07:52

Wilson, I cannot get the dns resolution on the backup css to prefer the main site using zone based. I can only get it to round robin. Let me know if you get that far and figure it out. Setting up the zone based was a piece of cake, unfortunately it doesn't seem the acl in the backup css has any effect.

bs6825 Tue, 06/12/2007 - 08:09

I have owrked through these same issues for a client. Not easy. One item I didn't see mentioned is that clients will cache DNS records and continue to attempt to access the down site without contacting DNS for a new address. Another otion is to advertise the same IP space in both locations and use BGP to make the primary site preferable. This also has some issues with convergence and advertisements, but it seems to be working.

acomiskey Tue, 06/12/2007 - 08:16

bs6825, the client caching problem should be solved easily by setting the ttl for the records on the css to 0, which is the default. Is this not your experience?

wilson_1234_2 Tue, 06/12/2007 - 09:21


I saw your earlier post.

Thanks for the information.

I wondered if this was a BGP issue with trying to set this up and some of the other guys here said it was not, to use DNS.

I am supposed to go get the DR CSS tomorrow.

Gillis advised not to use this method, maybe why he is not replying to these posts.

When you say round robin, is there any pattern to it (one, then the other, then the first one again)?

or does it seem to be random?

acomiskey Tue, 06/12/2007 - 09:27

No, this scenario does not use bgp. I'm not sure the previous poster read all of this thread.

Anyway, I know gilles has said this is not an advised solution but if you have the equipment and it works, it beats buying more equipment. BTW, cisco just sold us this solution about a year ago.

It is doing 1,2,1,2,1,2,1,2...etc. That is the pattern. If I set the main to be preferlocal I can get the main site to resolve 1,1,1,1,1 but the requests landing at the backup site still go 1,2,1,2,1,2 etc.

wilson_1234_2 Tue, 06/12/2007 - 11:22


I just got this information from MCI, who hosts our DNS records.

I am beginning to see why gilles is saying not to use this method.

I will post my question and their answer:

The idea is that Verizon DNS would act as a relay to the CSS devices.

The css devices would hold the Zone Data.

The follow up question would be this:

If we have two NS records acting as our DNS, what would determine how they would get hit for requests?

In other words, how would you guys forward the requests to the two devices?

Ideally it would be the primary device would get the requests, and if it was not available, the DR device would then get them.

Is this possible for you to do?

Nope, both devices would get hit at equal rates. There isn't any way to assign a weight/preference to an NS record so if a request came in for and host had two NS records, 50% of the queries would hit one and the other 50% would hit the other. If both devices are handing out different records, this is a problem. There really isn't any way to do this via DNS unless your devices are both up and both handing out the same data...and if one network goes down, both devices automatically start handing out new DNS accordingly. If one of these devices is on the affected circuit, however, your plan won't work and 50% of your queries will fail.

acomiskey Tue, 06/12/2007 - 11:36

Wilson, a few questions/comments...

1. Why the concern about which css the dns query lands at? The whole purpose of this is it doesn't matter which it lands at.

2. "In other words, how would you guys forward the requests to the two devices?"

-It's not about forwarding a request, the ISP just gives the client the NS record, which will be one of the CSS's, if it does not reply, the client will request another NS record, which the ISP will automatically give to the client.

3. "If both devices are handing out different records, this is a problem"

-They aren't.

4. "If one of these devices is on the affected circuit, however, your plan won't work and 50% of your queries will fail."

-Not true. If I try the first NS and it is unavailable, I will request another NS (secondary css).

5. "The idea is that Verizon DNS would act as a relay to the CSS devices."

-Don't think of this as a relay or anything special being performed by the isp, this is how everyone's dns works. The only difference is instead of the NS record for being it is

6. "I am beginning to see why gilles is saying not to use this method."

- You have to do the same with zone based as well.

I can POSSIBLY set up a test subdomain and show you it works. You would need to do some nslookups to a domain and at the same time I would have to bring down the service so you could see the failover.

acomiskey Tue, 06/12/2007 - 11:43

Also, if what they said is true there would be no reason to have multiple ns records for a domain, because if 1 ns server was down, you would still timeout part of the time.

Here's an example from on the domain Notice there are 4 NS servers configured for and 4 corresponding A records for the NS servers.

wilson_1234_2 Tue, 06/12/2007 - 12:06

I am having trouble conceptualizing how this is supposed to work I guess.

And I don't really know how DNS works.

But let me make these comments/questions:

1. When we say clients we are talking about end users correct?

2. The end user will request access to, the MCI DNS server (who hosts my DNS) will forward the requst to either one of the two CSS devices correct?

So the end userr will see the CSS devices as a Name Server just as they see the MCI Name server. correct?

3. So when the client requests name resolution, MCI is actually sending request for name resolution to a different Name Server which is the CSS, is that incorrect?

4. I am having trouble seeing it as not a different record from the CSS, if it has a differnet IP address for the same web server

5. If Main Site is alive, then BOTH CSS devices return the ip address of, correct?

6. If Main Site is down, then DR site will resolve to ( the DR Site Internet subnet, which is totally different than the Main site internet subnet of

Is the above correct?

acomiskey Tue, 06/12/2007 - 12:16

1. Yes.

2. Technically he is not forwarding the request, he would no longer be authoritative for He would rather give the client an NS which was responsible for The client then uses that NS record for the request which would be the CSS.

3. Not technically, the client is sending the request to the NS record supplied by the ISP. That NS record is the CSS.

4. What we're saying here is the address given by the CSS's for will be the same, the CSS's will have different addresses if main site is up if main site is down and backup site is up.

5. Yes, if is the public ip of the main server.(didn't go back through this thread to find out.)

6. Yes. Also, if only the main site server was down and the main site css was still alive, both css would resolve to backup server

acomiskey Tue, 06/12/2007 - 12:36

1. Because you define the records on each css and they talk to eachother through the app session.

2. I think I need to draw it out here...

Client -> tries to go to -> request ends up at isp -> isp says hey I've got an NS record for -> isp gives this NS record to the client(either one of the css's) -> the client uses this NS record to request resolution of -> this request ends up at either css1( or css2( whichever the isp gave first -> the requests lands at -> the css then says is the main server up? -> if yes the css says hey I've got an A record for which = -> if the css say the main server is down it will know whether or not the backup server is up -> if it is it will say hey I've got another A record for which is at our backup site

acomiskey Tue, 06/12/2007 - 13:38

Could you make it a jpeg or something I don't have visio here right now.

wilson_1234_2 Tue, 06/12/2007 - 14:11

Here is the drawing.

My thought was the reason to use the CSS devices as DNS servers is that the public IP Address can be different for the same public web server name.

If the HQ site has an Internet subnet of and the DR site has an Internet subnet of,

If when you say both devices are handing out the same "record" and by record you mean server name, and the server name could be from either IP address subnet,

Then I understand, if not then I don't see how this will work

acomiskey Wed, 06/13/2007 - 04:36

"My thought was the reason to use the CSS devices as DNS servers is that the public IP Address can be different for the same public web server name."

-Yes, that is true. But at any one particular moment in time, both CSS would return the same ip address for because 1 server will always be preferred over the other.

"If when you say both devices are handing out the same "record" and by record you mean server name, and the server name could be from either IP address subnet"

-Both devices hand out ip addresses for the names, yes that is correct, and it could be from either subnet, yes.

Your drawing is correct, just get rid of the BGP as it is not needed. I'm also not sure how to go about resolving to the main site server if only the main site internet is down. Mine is not configured that way. If the main site is not reachable via the internet then I use the backup server.

wilson_1234_2 Wed, 06/13/2007 - 06:12

Would it be possible for me to see your configs, minus and sensitive information?

If I saw both devices, I mabe would get a better idea of how this works.

As far as the resolving to the main site server, take a look at the config I posed early in this thread.

The one CSS I have is doing that now just from the Main Site only.

The other CSS would be configured the same way using the ip addresses from that Internet subnet and instead of pointing to the local server firat, it would point to the remote (Main Site) server.

I think it should work, it works with the one CSS now.

acomiskey Wed, 06/13/2007 - 06:15

Do you want to see the config's with rules based dns? That's the only one I have working right now.

prakashj Wed, 06/13/2007 - 06:27


This is regarding GSS,Since I am new to GSS,I am not able to config the same,Can you share the GSS knowledge with some sample config.


saji k.s

acomiskey Wed, 06/13/2007 - 06:30

Wilson, here are my config's for the rules based dns. I have also included towards the bottom the records I have at my hosted dns.

prakashj, I have no experience with GSS sorry.

acomiskey Wed, 06/13/2007 - 07:22

No problem, hope they help and don't confuse you further.

Just to add, the 'A' records I keep referring to, which are the ip addresses returned from the CSS's which equate to an ip address, are the "vip address x.x.134.112" statements.

wilson_1234_2 Wed, 06/13/2007 - 07:37

If the rule based DNS is working, why not use that?

Is there an added benifit to using the Zone based DNS?

And why wont gilles add some input to this?

We too, spent almost $12,000 on just the Advanced Feature Licensing for the two CSS devices.

I don't know how much on the oCSSs themselves, I think it should work the way it is.

Not to mention I be the GSS or whatever it is, is another ton of money.

Not confusing at all, I think I am the one that confuses people all the time.

You have been a great help.


This Discussion