Secure ACS Intermittent

Unanswered Question
Feb 15th, 2007

Hi,

We are using Cisco Secure ACS and for the past week, our switch and router logins are really really intermittent. Most of the time, even if we are into the console already and issue a command, "authorization failed" will appear then just keep pressing up and enter then the command will be accepted. Any idea why is this happening? Thank you very much.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Average Rating: 3 (7 ratings)
darpotter Thu, 02/15/2007 - 03:36

You could try looking in the in failed attempts report. This would be a good place to start.

Vivek Santuka Thu, 02/15/2007 - 04:07

Hi,

One more thing which is worth trying is to increase the tacacs-server timeout value.

Regards,

Vivek

smue_decm Wed, 02/21/2007 - 23:30

Hi,

we have the same problem in our network. Commands entered rapidly first work fine but after a period of time the commands are rejected - "authorization failed". With an increased "tacacs timeout" the message "authorization failed" doesn't appear anymore. as a result the tacacs queue increases and the switch or router has a faint response.

The accepted commands are listed in the "TACACS+ administration" log - as assumed. But the rejected commands don't appear in any log...what's the problem?

ellis_b Thu, 02/22/2007 - 00:11

Have you tried running a constant ping from the device to the ACS server? Is there something that is dropping packets along the path (ie from congestion or something)? What version of ACS are you running?

smue_decm Thu, 02/22/2007 - 01:04

Hi,

thanks for the reply.

We're using ACS 4.0.

The network's fine. We tested connectivity from a switch within our LAN AND over many hops from other devices in the network.

Vivek Santuka Thu, 02/22/2007 - 01:40

Hi,

You can try adding "single-connection" keyword after the "tacacs-server host" command in the device.

The single-connection keyword specifies a single connection (only valid with ACS). Rather than have the router open and close a TCP connection to the server each time it must communicate, the single-connection option maintains a single open connection between the router and the server.

The single connection is more efficient because it allows the server to handle a higher number of TACACS operations.

Regards,

Vivek

smue_decm Thu, 02/22/2007 - 02:25

Hi,

this works better now. But still, after several tries the entered commands end up in a queue...

Is there any possiblity without reconfiguring a switch or router?

Vivek Santuka Thu, 02/22/2007 - 02:31

Hi,

With 3.x there was a way to tweak ACS but with 4.x you will have to open a TAC case.

Regards,

Vivek

darpotter Thu, 02/22/2007 - 03:51

hmmm.. how many concurrent admin sessions might be performing T+ authorisation?

This could be ACS running out of spare connections. You'd see no errors in the ACS CSV, but you might see something in the CSTacacs service log. It really should log if connections/packets are being dropped.

Vivek Santuka Thu, 02/22/2007 - 04:32

Infact not only concurrent authorization session but accounting sessions also matter.

ACS can handle limited number of concurrent tacacs sessions.

John Patrick Lopez Thu, 02/22/2007 - 05:21

The ACS appliance doesn't reply to ping packets. We are using ACS 3.3.

Sometimes we have to type our username and password again and again because the ACS is not responding.

Can the timeout resolve this issue?

Thank you very much guys. =)

at Mon, 02/26/2007 - 09:47

hi,

with acs 3.1 we had the problem that we reached the maximum of 40 single connections ! (Message in the package.cab : "maximum 40 single connection are busy")

we increased the maximum "MaxSessions" in the Registry from 40 (hex 28) to 200 (hex C8 )

Look at

[HKEY_LOCAL_MACHINE\SOFTWARE\Cisco\CiscoAAAv3.1\CSTacacs]

"BaseDir"="\\CSTacacs"

"Version"="3.1(1.27)"

"Port"=dword:00000031

"PacketSize"=dword:00000400

"MaxSessions"=dword:00000028

"LocalSecret"="secret_value"

"SingleConnect"=dword:00000001

"ProxyOn"=dword:00000001

"ProxyRetries"=dword:00000001

"ChPassEnabled"=dword:00000000

"ChPassFastReplicate"=dword:00000000

"CHPassDisabledMessage"="Chpass is currently disabled."

"PackDump"=dword:00000001

regards

alex

akemp@powertel.com Thu, 02/22/2007 - 05:23

Very interesting indeed, do you have that magic number as I'm experiencing that same issue too.

The only figures that I've found documented are from 3.2 and only for 800 AP's and 100K users (how artificially they derived this synthetic info is questionable but Dewan the Product Manager for Cisco Wireless was kinda quiet about it)

Vivek Santuka Thu, 02/22/2007 - 05:53

If I am not wrong Tacacs+ threads hover around a maximum of 50.

Regards,

Vivek

smue_decm Thu, 02/22/2007 - 05:48

Hi,

thanks for your thoughts.

How many tacacs sessions can acs handle?

Vivek Santuka Thu, 02/22/2007 - 06:03

John,

ACS Solution enginer will not reply to ping if CSA Agent is enabled. (System Configuration->Appliance Configuration)

Increasing timeout is not always the answer but is a good first step in identifying the problem.

We could be experiencing a delay from the external db or even from remote logging facility. All these and more would contribute to a delay is authentication

ACS running out of threads is not a common thing and not seen often.

Auth.log would be a very good place to look for problems.

Regards,

Vivek

smue_decm Thu, 02/22/2007 - 06:25

Vivek,

our test-switch is now configured with:

tacas-server host [ip] single-connection

we're testing the following way:

- log in

- send a cmd (for example: show tacacs)

with "single-connection" on, we have to send a command very often to reproduce the queue and the "authorisation failed"-message - without "single-connection" it was worse.

now we're comparing the number of requests in auth.log with the number of commands that the aaa client sent. it seems to us that there are no dropped requests in auth.log - just "Start RQ****" and "Done RQ****".

we'll keep you posted.

Vivek Santuka Thu, 02/22/2007 - 06:37

Hi,

I would suggest setting the Log level to full (system configuration->Service control) while testing.

Regards,

Vivek

smue_decm Fri, 02/23/2007 - 02:32

Hi,

how many single-connections does a acs server handle?

Is there a limit? If so, could this limit be configured?

Can the active single-connections be monitored (how many open connections at a/one time?)

Is it possible to shut down active single-connections?

Vivek Santuka Fri, 02/23/2007 - 03:54

Hi,

Tacacs single connections are taken fromt he total available (which is around 50). The limit can be changed if a TAC case is opened.

Active single connections cannot be "monitored" but auth.log will give an indication of open threads.

Active connections will be closed by Aaa client as per normal operation but abnormally we will have to restart the services.

Regards,

Vivek

smue_decm Sun, 02/25/2007 - 23:42

Hi Vivek,

the generated TAC has being rejected because our current contracts don't include support for ACS.

It needs a while to include this service to our contracts - what's the fastest way to get support for ACS now?

Thanks.

Vivek Santuka Mon, 02/26/2007 - 02:48

Hi,

I believe your AM/SE can help you out there. Since you are using ACS 4.x only TAC can help. If it was 3.x I could have helped here.

Regards,

Vivek

smue_decm Mon, 02/26/2007 - 06:58

Hi Vivek,

we disabled all unnecessary logging. After that we couldn't reproduce the errors. Our colleagues are currently testing and will inform us if the error occurs again.

Our server has never had and still has no performance problems. Any idea which daemon makes so much trouble?

For now we leave the detailed logging off. We'll put it back on only if we have to (in case of any other error or problem).

Vivek Santuka Mon, 02/26/2007 - 09:10

Hi,

Slow authentication response from ACS will be due to CSAuth and CsTacacs/CsRadius.

Regards,

Vivek

John Patrick Lopez Tue, 02/27/2007 - 20:00

Hi Vivek,

We are using ACS 3.3 appliance. Is there a way to increase the maximum session? Because we are still encountering the problem. We already enabled CSAgent. We can now ping the ACS appliance and we don't encounter any dropped packets. Sometimes, if we try to login to the webpage, we are getting protocol error message and we still have to wait until it's accessible.

Please help. Thanks.

at Mon, 02/26/2007 - 10:14

hi,

with acs 3.1 we had the problem that we reached the maximum of 40 single connections ! (Message in the package.cab : "maximum 40 single connection are busy")

we increased the maximum "MaxSessions" in the Registry from 40 (hex 28) to 200 (hex C8 )

Look at

[HKEY_LOCAL_MACHINE\SOFTWARE\Cisco\CiscoAAAv3.1\CSTacacs]

"BaseDir"="\\CSTacacs"

"Version"="3.1(1.27)"

"Port"=dword:00000031

"PacketSize"=dword:00000400

"MaxSessions"=dword:00000028

"LocalSecret"="secret_value"

"SingleConnect"=dword:00000001

"ProxyOn"=dword:00000001

"ProxyRetries"=dword:00000001

"ChPassEnabled"=dword:00000000

"ChPassFastReplicate"=dword:00000000

"CHPassDisabledMessage"="Chpass is currently disabled."

"PackDump"=dword:00000001

regards

alex

Actions

Login or Register to take actions

This Discussion

Posted February 15, 2007 at 12:33 AM
Stats:
Replies:28 Avg. Rating:3
Views:320 Votes:0
Shares:0
Tags: No tags.

Discussions Leaderboard