cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2462
Views
5
Helpful
10
Replies

APs randomly will not join the controller

ajc
Level 7
Level 7

Hi all,

In short, APs will join the controller immediately using the recovery image, but once it downloads 7.6.100 and reboots, it either cannot join the controller or will take up to 40 minutes.

 

The AP is in local mode. The AP 1040 eventually joined the controller, 40 minutes after we deleted all the files off the flash which is unacceptable.

 

*Apr  9 15:38:15.842: %CAPWAP-5-SENDJOIN: sending Join Request to 10.10.10.10

*Apr  9 15:38:15.851: %CAPWAP-3-ERRORLOG: Invalid event 10 & state 5 combination.

*Apr  9 15:38:15.851: %CAPWAP-3-ERRORLOG: CAPWAP SM handler: Failed to process message type 10 state 5.

*Apr  9 15:38:15.851: %CAPWAP-3-ERRORLOG: Failed to handle capwap control message from controller

*Apr  9 15:38:15.851: %CAPWAP-3-ERRORLOG: Failed to process encrypted capwap packet from 10.10.10.10

*Apr  9 15:38:16.304: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down

*Apr  9 15:38:16.362: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset

*Apr  9 15:38:16.427: %CAPWAP-5-JOINEDCONTROLLER: AP has joined controller CONTROLLER-DC

 

I also had our technician console into a 2602 and we were seeing exact same errors.

SUMMARY OF THIS CASE:

--AP is 1042 and wlc is on 7.6.100.0

--using dhcp option 43 but the AP has static ip

--checked the private config on the AP and it was not sending the discovery request to the desired  WLC

--cleared the private config

--primed the AP to desired WLC

--AP sending join request but receiving the following errors:

 

Apr  4 14:37:47.000: CAPWAP-3-ERRORLOG Go join a capwap controller

*Apr  4 14:36:47.000: CAPWAP-5-DTLSREQSEND DTLS connection request sent peer_ip: 10.10.10.11 peer_port: 5246

*Apr  4 14:36:47.169: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_handshake.c:90 First fragment for seq 2 is missing

*Apr  4 14:37:17.205: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2176 Max retransmission count reached!

--cleared the flash except rcv

--ap started downloading the image from the wlc but once it reboots the same errors appear again

--research made on the error message and apparently we are hitting a bug:

https://tools.cisco.com/bugsearch/bug/CSCul08933/?reffering_site=dumpcr

 

 

10 Replies 10

Leo Laohoo
Hall of Fame
Hall of Fame

May I ask if you can post the output to the following commands: 

 

  1. WLC:  sh sysinfo; 
  2. WLC:  sh time; 
  3. AP:  sh version; 
  4. AP:  sh ip interface brief; and
  5. AP:  sh inventory

Hi Leo,

 

Next the information, I omitted some serial number and descriptions due to confidentiality matters.

 

(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.

Product Name..................................... Cisco Controller Product Version.................................. 7.6.100.0 Bootloader Version............................... 1.0.1 Field Recovery Image Version..................... 6.0.182.0 Firmware Version................................. FPGA 1.3, Env 1.6, USB console 1.27 Build Type....................................... DATA + WPS

System Name...................................... ABCDEF System Location..................................

System Contact...................................

System ObjectID.................................. 1.3.6.1.4.1.9.1.1069 Redundancy Mode.................................. Disabled IP Address....................................... XXXXXXX Last Reset....................................... Software reset System Up Time................................... 32 days 14 hrs 1 mins 41 secs System Timezone Location......................... (GMT -5:00) Eastern Time (US and Canada) System Stats Realtime Interval................... 5 System Stats Normal Interval..................... 180

 

Configured Country............................... CA  - Canada Operating Environment............................ Commercial (0 to 40 C)

Internal Temp Alarm Limits....................... 0 to 65 C Internal Temperature............................. +40 C External Temperature............................. +21 C Fan Status....................................... OK

State of 802.11b Network......................... Enabled State of 802.11a Network......................... Enabled Number of WLANs.................................. 5 Number of Active Clients......................... 2215

Burned-in MAC Address............................ F0:F7:55:2B:4A:80 Power Supply 1................................... Present, OK Power Supply 2................................... Absent Maximum number of APs supported.................. 500

(Cisco Controller) >show time

Time............................................. Mon Apr 14 08:40:08 2014

Timezone delta................................... 0:0 Timezone location................................ (GMT -5:00) Eastern Time (US and Canada)

NTP Servers

    NTP Polling Interval.........................     36000

     Index     NTP Key Index                  NTP Server                  NTP Msg Auth Status

    -------  ----------------------------------------------------------------------------------

       1              0                                              A.B.C.D                        AUTH DISABLED

 

AP>sh ver

Cisco IOS Software, C3500 Software (AP3G1-K9W8-M), Version 15.2(4)JB3, RELEASE SOFTWARE (fc1) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2013 by Cisco Systems, Inc.

Compiled Wed 18-Dec-13 21:36 by prod_rel_team

 

ROM: Bootstrap program is C3500 boot loader

BOOTLDR: C3500 Boot Loader (AP3G1-BOOT-M), Version 15.2 [chayan-apn-0411 116]

AP uptime is 2 days, 17 hours, 15 minutes System returned to ROM by reload System image file is "flash:/ap3g1-k9w8-mx.152-4.JB3/ap3g1-k9w8-xx.152-4.JB3"

Last reload reason:

 

cisco AIR-CAP3502I-A-K9 (PowerPC460exr) processor (revision A0) with 98294K/32768K bytes of memory.

Processor board ID XXXXXXXX

PowerPC460exr CPU at 666Mhz, revision number 0x18A8 Last reset from reload LWAPP image version 7.6.100.0

1 Gigabit Ethernet interface

2 802.11 Radios

 

32K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address: 70:81:05:9E:F7:36

Part Number                          : 73-12175-05

PCA Assembly Number                  : 800-32268-05

PCA Revision Number                  : A0

PCB Serial Number                    : XXXXXXX

Top Assembly Part Number             : 800-32891-01

Top Assembly Serial Number           : FGL1534S7LP

Top Revision Number                  : A0

Product/Model Number                 : AIR-CAP3502I-A-K9  

 

 

 

Configuration register is 0xF

AP>sh ip int br

Interface                  IP-Address      OK? Method Status                Protocol

BVI1                       X.Y.W.Z                YES TFTP   up                    up     

Dot11Radio0                unassigned      NO  unset  up                    up     

Dot11Radio1                unassigned      NO  unset  up                    up     

GigabitEthernet0           unassigned      NO  unset  up                    up     

GigabitEthernet0.1         unassigned      YES unset  up                    up     

 

AP>sh inventory

NAME: "AP3500", DESCR: "Cisco Aironet 3500 Series (IEEE 802.11n) Access Point"

PID: AIR-CAP3502I-A-K9 , VID: V01, SN: XXXXXXX

I omitted some serial number

You omitted the WRONG serial number.  

Do you need that information?. If it so, what is the serial you want to check.

Let me take a look on the mesh ios you mentioned.

I've got a suspicion your AP is running MESH IOS.

Hi Leo,

 

What is the implication of using MESH IOS with respect to this issue?

 

thanks

If your AP is running mesh, then it means that you'll need to add the AP MAC address into the WLC database so the AP can join the controller.

if Mesh the AP will not join at all , 

As a general information, the problem was a DTLS bug found on version 7.6.100 which was addressed on an special software 7.6.100.25 that we installed recently and tested.

 

regards

Looks like this is not the case. We tried what is indicated in the following link and still the same error/issue:

 

http://www.cisco.com/c/en/us/support/docs/wireless-mobility/wireless-mesh/70531-wlc-mesh-config.html

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: