Can you give us more details

Philip T Kurien · ‎04-03-2014

Hi,

I am facing an issue with my AP not joining the controller.

Following is the error log

Mar 1 00:01:11.923: %CAPWAP-3-ERRORLOG: Could Not resolve CISCO-CAPWAP-CONTROLLER
*Mar 1 00:01:21.923: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Apr 3 08:10:04.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.146.47.150 peer_port: 5246
*Apr 3 08:10:26.775: %CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source
*Apr 3 08:10:33.999: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2051 Max retransmission count reached!
*Apr 3 08:11:03.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.146.47.150:5246
*Apr 3 08:11:03.999: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Apr 3 08:10:04.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.146.47.150 peer_port: 5246

aleopoldie · ‎04-03-2014

Can you give us more details like IP addresses and model of your WLC and AP please ?

cheers,

Alexis

Justin DeVaughn · ‎04-03-2014

Hi Philip,

It looks like the AP has the IP address for the WLC of 10.146.47.150.

Two things that may cause this:

1. The IP address of the WLC is mismatched.

2. The CAPWAP flow cannot be established between the AP and the WLC (which can again be caused by two things).

Verification:

- Please confirm if that is indeed the IP address of the WLC you intend this AP to associate

- Confirm you do not have assymetric routing taking place. That is, make sure the interface on the WLC with Dynamic AP Manager checked has a default gateway IP address that sends traffic directly back the same way traffic is coming from the AP. (This can be tricky because if this is the cause, the symptoms are intermittent association of the AP which makes you think all the routing is okay and can sometimes even result in a successful ping if you have your 2 gateways to reference each other as gateways of last resort. In reality, you're AP and WLC Default Gateway settings are using two different networks to reach each other.)

- The other reason CAPWAP session is not being maintained is because of some device blocking ports UDP 5246 or 5247 on the transit path between AP and WLC. This could be a carrier issue or your own firewall depending on the environment and if it's across a WAN.

If you run the command "show sysinfo" while logged into the WLC and post it, I can determine if the WLC IP is correct.

I apologize if I answer in an inappropriate format. I'm still new to posting on these forums. Let me know how this turns out!

Philip T Kurien · ‎04-05-2014

Hi Guys,

Thanks for your inputs.

I investigated the issue suspecting a problem with the communication between the WLC and the APs. Actually the WLC is connected to 2 6506 chassis with the 1st chassis as the root.

I discovered a minor error on the module in the 1st chassis. When i connected to the 2nd chassis. The AP was able to join the controller. Outputs for the faulty module as below:

Looks like I will need to RMA the module.

Router#sh module
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL1743F20V
2 16 CEF720 16 port 10GE WS-X6716-10GE SAL1746G9RS
3 16 CEF720 16 port 10GE WS-X6716-10GE SAL1746G9QR
5 5 Supervisor Engine 720 10GE (Active) VS-S720-10G SAL1746GC1G

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
1 7c69.f690.8908 to 7c69.f690.8937 4.3 12.2(18r)S1 12.2(33)SXJ6 Ok
2 7c69.f690.badc to 7c69.f690.baeb 2.1 12.2(18r)S1 12.2(33)SXJ6 Ok
3 e4d3.f1a4.39f8 to e4d3.f1a4.3a07 2.1 12.2(18r)S1 12.2(33)SXJ6 Ok
5 5057.a8e3.394a to 5057.a8e3.3951 4.1 8.5(4) 12.2(33)SXJ6 Ok

Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
1 Centralized Forwarding Card WS-F6700-CFC SAL1746GAAX 4.1 Ok
2 Distributed Forwarding Card WS-F6700-DFC3C SAL1746G6T1 1.6 Ok
3 Distributed Forwarding Card WS-F6700-DFC3C SAL1746G844 1.6 Ok
5 Policy Feature Card 3 VS-F6K-PFC3C SAL1746GBJK 1.2 Ok
5 MSFC3 Daughterboard VS-F6K-MSFC3 SAL1746GAGT 5.1 Ok

Mod Online Diag Status
---- -------------------
1 Minor Error
2 Pass
3 Pass
5 Pass

Router#sh diagnostic result module 1

Current bootup diagnostic level: minimal

Module 1: CEF720 48 port 10/100/1000mb Ethernet SerialNo : SAL1743F20V

Overall Diagnostic Result for Module 1 : MINOR ERROR
Diagnostic level at card bootup: minimal

Test results: (. = Pass, F = Fail, U = Untested)

1) TestLoopback:

Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
----------------------------------------------------------------------------
. . . . . . . . . . . . . . . . . . . . . . . .

Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
----------------------------------------------------------------------------
. . . . . . . . . . . . . . . . . . . . . . . .

2) TestTxPathMonitoring ------------> .
3) TestSynchedFabChannel -----------> .
4) TestL3VlanMet -------------------> F
5) TestIngressSpan -----------------> .
6) TestEgressSpan ------------------> .
7) TestAsicMemory ------------------> U
8) TestEobcStressPing --------------> U
9) TestFirmwareDiagStatus ----------> .
10) TestAsicSync --------------------> .
11) TestUnusedPortLoopback:

Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
----------------------------------------------------------------------------
U U U U U U U U U U U . U U U U U U U U U U U .

Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
----------------------------------------------------------------------------
U U U U U U U U U U U . U U U U U U U U U U U .

12) TestOBFL ------------------------> .
13) TestErrorCounterMonitor ---------> .
14) TestPortTxMonitoring:

Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
----------------------------------------------------------------------------
U U U U U U U U U U U U U U U U U U U U U U U U

Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
----------------------------------------------------------------------------
U U U U U U U U U U U U U U U U U U U U U U U U

15) TestLtlFpoeMemoryConsistency ----> .
16) TestRwEngineOverSubscription ----> U

Router#sh diagnostic result module 1 detail

Current bootup diagnostic level: minimal

Module 1: CEF720 48 port 10/100/1000mb Ethernet SerialNo : SAL1743F20V

Overall Diagnostic Result for Module 1 : MINOR ERROR
Diagnostic level at card bootup: minimal

Test results: (. = Pass, F = Fail, U = Untested)

___________________________________________________________________________

1) TestLoopback:

Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
----------------------------------------------------------------------------
. . . . . . . . . . . . . . . . . . . . . . . .

Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
----------------------------------------------------------------------------
. . . . . . . . . . . . . . . . . . . . . . . .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Bootup
Last test execution time ----> Apr 01 2014 10:55:39
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:55:39
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

2) TestTxPathMonitoring ------------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 40
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:30
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:30
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

3) TestSynchedFabChannel -----------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 39
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:30
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:30
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

4) TestL3VlanMet -------------------> F

Error code ------------------> 54 (DIAG_CHECK_UDP_PAK_ERROR)
Total run count -------------> 1
Last test testing type ------> Bootup
Last test execution time ----> Apr 01 2014 10:55:53
First test failure time -----> Apr 01 2014 10:55:53
Last test failure time ------> Apr 01 2014 10:55:53
Last test pass time ---------> n/a
Total failure count ---------> 1
Consecutive failure count ---> 1
___________________________________________________________________________

5) TestIngressSpan -----------------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Bootup
Last test execution time ----> Apr 01 2014 10:56:05
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:56:05
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

6) TestEgressSpan ------------------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Bootup
Last test execution time ----> Apr 01 2014 10:56:05
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:56:05
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

7) TestAsicMemory ------------------> U

Error code ------------------> 3 (DIAG_SKIPPED)
Total run count -------------> 0
Last test testing type ------> n/a
Last test execution time ----> n/a
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> n/a
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

8) TestEobcStressPing --------------> U

Error code ------------------> 3 (DIAG_SKIPPED)
Total run count -------------> 0
Last test testing type ------> n/a
Last test execution time ----> n/a
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> n/a
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

9) TestFirmwareDiagStatus ----------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Bootup
Last test execution time ----> Apr 01 2014 10:56:05
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:56:05
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

10) TestAsicSync --------------------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 5
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:21
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:21
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

11) TestUnusedPortLoopback:

Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
----------------------------------------------------------------------------
U U U U U U U U U U U . U U U U U U U U U U U .

Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
----------------------------------------------------------------------------
U U U U U U U U U U U . U U U U U U U U U U U .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:06
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:06
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

12) TestOBFL ------------------------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Bootup
Last test execution time ----> Apr 01 2014 10:56:05
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:56:05
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

13) TestErrorCounterMonitor ---------> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 2
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:06
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:06
Total failure count ---------> 0
Consecutive failure count ---> 0
Error Records ---------------> n/a
___________________________________________________________________________

14) TestPortTxMonitoring:

Port 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
----------------------------------------------------------------------------
U U U U U U U U U U U U U U U U U U U U U U U U

Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
----------------------------------------------------------------------------
U U U U U U U U U U U U U U U U U U U U U U U U

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 1
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:21
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:21
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

15) TestLtlFpoeMemoryConsistency ----> .

Error code ------------------> 0 (DIAG_SUCCESS)
Total run count -------------> 5
Last test testing type ------> Health Monitoring
Last test execution time ----> Apr 01 2014 10:57:21
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> Apr 01 2014 10:57:21
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

16) TestRwEngineOverSubscription ----> U

Error code ------------------> 3 (DIAG_SKIPPED)
Total run count -------------> 0
Last test testing type ------> n/a
Last test execution time ----> n/a
First test failure time -----> n/a
Last test failure time ------> n/a
Last test pass time ---------> n/a
Total failure count ---------> 0
Consecutive failure count ---> 0
___________________________________________________________________________

Router#sh diagnostic result module 1 failure

Current bootup diagnostic level: minimal

Module 1: CEF720 48 port 10/100/1000mb Ethernet SerialNo : SAL1743F20V

Overall Diagnostic Result for Module 1 : MINOR ERROR
Diagnostic level at card bootup: minimal

Test results: (. = Pass, F = Fail, U = Untested)

4) TestL3VlanMet -------------------> F

Leo Laohoo · ‎04-05-2014

Philip,

Line card 4 is brand new. But I doubt if this is the cause. You can move the WLC link to a different line card and the result should be the same.

From the WLC, post the output to the command "sh sysinfo".
From the AP, post the output to the command "sh version", "sh ip int brief" and "sh inventory".

Philip T Kurien · ‎04-05-2014

Hi Leo,

The controller was connected to Line Card 1. I connected to another line card and it started working fine. The AP registered. When i put the controller back to the old line card, it stopped working again.

That is how i confirmed that it is the line card.

Sandeep Choudhary · ‎04-03-2014

Hi

post the info:

from wlc: sh sysinfo

from AP: sh version

also paste the whole bootup process from AP console.

Regards

Dont forget to rate helpful posts

Wireless AP not joining Controller