Issue after upgrading to

Unanswered Question
Apr 8th, 2010


We tried to upgrade to the latest code from on our 4402 controllers.  We have 3 of them and ~70 1242APs.  The controllers upgraded sucessfully, but many of the AP's don't seem to be coming back after the upgrade.  We checked a few of the APs and we noticed the status light was purple and even tried powering them off and back on.  We are not quite sure what what else we can do, but try rolling back to the old code.  Any ideas would be very helpful.  Thanks!

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Leo Laohoo Thu, 04/08/2010 - 16:59

I'd give them 48 hours.  No joke.  The 6.X code, when it comes to the WLC discovery by the LWAP, is more robust than previous versions.  My experience is that the LWAPs will come back within 24 hours.

Hanzni_2 Thu, 04/08/2010 - 17:04

Thanks for the fast responce and wow.   I read about 30 mins to upgrade, but 24 hours is a lenghty time for the APs to be down.  I don't suppose there is any way to speed up this process or anything I can confirm that is the case?  The controllers shows as downloading and dissapeared after that.  I wouldn't have thought it would take that long..

Leo Laohoo Thu, 04/08/2010 - 17:17

The WLC 4400 upgrades 10 AP at a time.  There's no way to speed things up other than upgrade to the 5500 which upgrades at 100 at a time (if memory serves me correctly).   Normally when I upgrade the APs I start from 7pm and when I come back in the morning they are all there.  But upon closer inspection of the discovery time I would notice a very significant difference with some just discovered withing 2 hours after I came back from work.

Hanzni_2 Thu, 04/08/2010 - 18:33

I consoled into one of the AP that were giving me problems and was seeing the following.

*Apr  9 02:30:33.068: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Apr  9 02:30:33.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_i
p: peer_port: 5246
*Apr  9 02:30:33.001: %CAPWAP-5-CHANGED: CAPWAP changed state to
*Apr  9 02:31:03.000: DTLS_CLIENT_ERROR: ../dtls/dtls_connection_db.c:2015 Max r
etransmission count reached!
*Apr  9 02:31:03.000: %DTLS-3-HANDSHAKE_RETRANSMIT: Max retransmit count for x
.x.x.x is reached.

patoberli Wed, 04/14/2010 - 03:34

Have you verified the changed port requirements on your firewall? As far my memory serves me

right,  version 6 with CAPWAP uses different tcp/udp ports as version 4 with LWAPP. Also verify the management and ap-manager interfaces, they should be now in the same VLAN as the accesspoints.

Also verify the DNS and DHCP settings, I beleive the DHCP requirements have also changed with version 6.



weterry Thu, 04/15/2010 - 05:44

max retransmissions implies that this AP isn't getting its packets to the WLC (or vice versa).   As someone else noted, the UDP ports have changed from 12222/12223 to 5246/5247. Perhaps you are not allowing those ports to the MGMT and AP-MGR interface of the WLAN?

As for the "slow" join time, if it takes 24 hours for your APs to discover and join a WLC, then something is wrong. The AP should discover the WLC as soon as it is back up and the download/join will take place 10 at a time. But you're looking at a couple of minutes for each set of 10,  not an hour or anything like that.

Anyhow, your console output looks to me like WLC Discovery is great, it just can't get its packets to the AP-MGR interface of the WLC (or vice versa).

Hanzni_2 Thu, 04/15/2010 - 14:24

The three controllers are all configured with the same settings and located on the same vlans an same location.  There isn't a firewall inbetween any of the APs and/or controllers.  The APs seem to have no problem connecting to two (A & B) of the three controllers.  The issue is with just one (C) of them.  As soon as they try to connect to "C" the APs will keep showing the same errors posted above, but if I reboot the problem controller they come up just fine on either the A or B controler.

This is where I'm getting stuck.  I checked the config a few times and it is the same across all three controllers.  I've reloaded the controller from scratch just incase there was an error in my config because of the upgrade and that didn't work as well.

weterry Thu, 04/15/2010 - 14:41

Are any APs working on WLC3?

Could there be a duplicate IP address of the AP-MGR address for WLC3?

I hate to say it, but a packet capture at the AP and one at the WLC is really the next step I would take to figure out if any packets are getting lost along the way.

Unless this is something where WLC3 is purposely not responding... Like SSC certs or wrong-time  or something like that?  But I don't recall ever seeing the max retransmissions occur because of the WLC rejecting the JOIN..... you typically see the "no more ap manager ip address remain"  or something like that...


This Discussion