Cisco 1240AG all go off-line at the same time?

brent.ritchie · ‎07-13-2010

I have an existing site with 5 1240AG Access points and have never had any problem; a few months ago we added 2 more to the existing building and still had no problems.. then a few weeks ago we added an additional building connected via Fiber and added 6 more new 1240AG's (all on the same subnet) and now a couple of times a day all the new AP's in the new building suddenly drop! a few times (or more) it has alos caused the 2 new ones in the original building to also suddenly drop. when i say drop i mean they are no longer accessable, i can no longer ping / snmp / web / telnet to them.. the only way to recover them is to remove power then add it back..

all AP's are on POE (Cisco POE adapters). we have tried seperating power to different UPS's, swapped out the switch (of course the fact that we are loosing AP's in two different buildings at the same time seperates all infrastructure as well)...

there has to be something that these AP's are broadcasting to one another that is causing this, but not affecting the older ones but i cant figure out what?

there are the software versions:

new AP's - 12.4(21a)JA1

old AP's - 12.4(3g)JA

as i was writing this it happened again, first 5 of the 6 new ones dropped at the same time, then the 6th one a minute latter, and now the 2 in the original building just dropped (a few minutes after the first ones)... and of course all 5 of my original units are all up and running fine.. i have chekced the logs on them and there is nothing to explain it

any help is greatly apriciated... thanks

Brent Ritchie

Leo Laohoo · ‎07-13-2010

Define "drop"? Reboot? Crash? Power off and then on? Radios go down and up?

brent.ritchie · ‎07-14-2010

when i say drop i mean they are no longer accessable, i can no longer ping / snmp / web / telnet to them.. the only way to recover them is to remove power then add it back..

I can see the switch port is still lit up, but the AP does not respong

Kayle Miller · ‎07-14-2010

Brent,

This may sound completely crazy, but check spanning tree, I had a similar issue where there was a spanning tree loop that kept killing the AP's... also login to the AP via telnet when it comes back and see the uptime? or are you having to physically power cycle them??

Please rate helpful posts...

Thanks,

brent.ritchie · ‎07-14-2010

yes have to physicaly power them off and back on before we can see them again. i dont think its spanning tree but will check; this problem only affects the new AP's running the newer code, and when they drop sometimes it's all of them, sometimes just a few of them, but never just 1?

Kayle Miller · ‎07-14-2010

Brent,

This is probably impractical but have you tried to console into an AP when it went to this state to see if the AP has locked up??? Other than that I'd say it's a bug in the firmware. Can you downgrade the new AP's to the same code?

Thanks,

Kayle

brent.ritchie · ‎07-14-2010

unfotunatly i'm 300miles from the plant or i woudl try to console in, i have some IT staff on-site to do the reboots; downgrading is the plan if we dont figure this out quick... i have another plant with the same infrastructure (same switches / AP's (new IOS)) and that plant has never had any issue (fingers crossed)... its just this one plant witht he mix of old and new, so i have concerns over some sort of incompatability

Kayle Miller · ‎07-14-2010

also check this bug info, is there any chance somethign like this is happening??

PPTP not connecting through IOS based AP

Symptom:

wireless clients trying to set up a PPTP or GRE connection through the AP will fail and time out.

Moreover, eventually the AP's LAN interface will become wedged, and be unable to
process input traffic. The command "show interface fasthethernet0" will show
that the input queue is full, for example:

Input queue: 81/80/0/0 (size/max/drops/flushes); Total output drops: 0

Conditions:
Seen on all IOS/Autonomous AP platforms running 12.4(21a)JA1; also on
lightweight IOS APs running 12.4(18a)JA (5.2.157.0) through 12.4(21a)JHA (6.0.196.0).

Workaround:

None.

More information:

This is a regression that was introduced in 12.4(18a)JA. It will be fixed
in lightweight IOS via CSCtf27580

brent.ritchie · ‎07-14-2010

thanks but not likely, we've actualy seen the AP's drop when there was no one even in the building and no connections to them; also

these AP's are only for plant floor scan guns, shoudl not be any computers connected to them (the scan guns run a web application from a local server)

Leo Laohoo · ‎07-14-2010

I've used 12.4(21a)JA1 on 1240 and 1250 and I've never seen this happen before. I'm currently using 12.4(21a)JY.

lisakson1 · ‎01-21-2011

Did you ever get this issue resolved? I'm having the same exact problem

at my facility. I'm configured that exact same way to described.

Thanks!!!