Solved: using plink with ACE : Too many open files

sebastianvandijk · ‎02-04-2008

Hi,

We have created some scripts for making managing rservers on the ACE easier. We use plink(putty) for the communication with the ACE.

This works great, however after some time the script does not work anymore.

When i try to plink manually i get this error "Could not create socket pairs: Too many open files in system".

I still can connect with putty but not anymore with plink.

Only after a reboot of the ACE the plink commando can be used again.

Does anyone has this experience too ? I am curious if this is a plink or cisco bug but i don't know how and if file parameters can be changed on the ACE.

Regards,

Sebastian

Gilles Dufour · ‎02-05-2008

Sebastian,

I managed to reproduce the issue and found out this is a known ddts

CSCsl74755 - Socket/FD resource leak due to ssh sessions

The fix will be in A1(6.4)

Gilles.

View solution in original post

Gilles Dufour · ‎02-04-2008

If plink can't connect but putty does, there must be a difference in what they do.

Can you sniff a connection of plink and one from putty.

sebastianvandijk · ‎02-04-2008

Hi Gilles,

I've made a capture for the plink part, for some kind of reason a putty connection would not work when capture was enabled.

In the plink capture there are some tcp retransmissions and tcp duplicat acks.

Gilles Dufour · ‎02-04-2008

could you do a 'show resource usage' before and after a failure.

Also, get a 'sho resource alloc'.

Thanks,

Gilles.

sebastianvandijk · ‎02-05-2008

Hi Gilles,

I've done a sh resource usage before and after a reload of the ACE module :

BEFORE RELOAD

Allocation

Resource Current Peak Min Max Denied

-----------------------------------------------------------------------------

Context: FRONTEND

conc-connections 8 226 800000 4800000 0

mgmt-connections 6 44 500 3000 0

proxy-connections 0 10 104858 629144 0

xlates 0 0 104858 629144 0

bandwidth 4435 22996934 50000000 300000000 0

connection rate 3 78 100000 600000 0

ssl-connections rate 0 0 100 600 0

mgmt-traffic rate 2048 76458 12500000 75000000 0

mac-miss rate 0 5 200 1200 0

inspect-conn rate 0 0 600 3600 0

acl-memory 39376 43584 7861044 47166260 0

regexp 0 0 104858 629146 0

syslog buffer 0 0 419430 2516583 0

syslog rate 0 12 300 1800 0

AFTER RELOAD

Allocation

Resource Current Peak Min Max Denied

-----------------------------------------------------------------------------

Context: FRONTEND

conc-connections 14 14 800000 4800000 0

mgmt-connections 6 24 500 3000 0

proxy-connections 0 4 104858 629144 0

xlates 0 0 104858 629144 0

bandwidth 17415 84564 50000000 300000000 0

connection rate 2 10 100000 600000 0

ssl-connections rate 0 0 100 600 0

mgmt-traffic rate 15647 63000 12500000 75000000 0

mac-miss rate 0 0 200 1200 0

inspect-conn rate 0 0 600 3600 0

acl-memory 39376 39440 7861044 47166260 0

regexp 0 0 104858 629146 0

syslog buffer 0 0 419430 2516583 0

syslog rate 0 6 300 1800 0

After a reload the problem disappears. Then after a while it returns.

Regards,

Sebastian

Gilles Dufour · ‎02-05-2008

Can we get a

'show ssh max' and 'show ssh session' when the problem is there.

Also get a 'show ssh key'

Gilles.

sebastianvandijk · ‎02-05-2008

Hi Gilles,

As i have rebooted the ACE this day the problem has dissappeared again. When it returns i'll post the output of the commands here. That might be in a week or 2/3.

regards,

Sebastian

Gilles Dufour · ‎02-05-2008

Sebastian,

I managed to reproduce the issue and found out this is a known ddts

CSCsl74755 - Socket/FD resource leak due to ssh sessions

The fix will be in A1(6.4)

Gilles.

sebastianvandijk · ‎02-06-2008

Hi Gilles,

great you've been able to reproduce it and found the ddts.

I am curious about how you have been able to reproduce this issue as it takes some weeks here to have it appear again ?

Sebastian

Gilles Dufour · ‎02-06-2008

downloaded the plink code, added some functions so that it can repeat connections over and over.

I let it run for a few minutes and then the problem was there.

Gilles.