Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
New Member

1252 LAP won't join WLC

Hi all

I'm having an issue with a 1252 LAP that is connected to the WLC over a WAN link.

Basically, it won't associate. The following is taken from a console into the LAP:

*Mar 1 00:00:07.799: %LINK-3-UPDOWN: Interface GigabitEthernet0, changed state to up

*Mar 1 00:00:08.799: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0, changed state to up

*Mar 1 00:00:26.851: %LWAPP-5-CHANGED: LWAPP changed state to DISCOVERY

*Mar 1 00:00:27.003: Logging LWAPP message to 255.255.255.255.

%CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source

%CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source

%DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.148.x.x, mask 255.255.255.0, hostname AP002

2.90a3.533a

Translating "CISCO-LWAPP-CONTROLLER.nation.radix"...domain server (10.x.x.x)

%LWAPP-3-CLIENTEVENTLOG: Controller address 10.x.x.x obtained through DHCP

%LWAPP-3-CLIENTEVENTLOG: Did not get log server settings from DHCP.

%SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 255.255.255.255 started - CLI initiated

%LWAPP-3-CLIENTEVENTLOG: Performing DNS resolution for CISCO-LWAPP-CONTROLLER.nation.radix

%LWAPP-3-CLIENTERRORLOG: DNS Name Lookup: could not resolve CISCO-LWAPP-CONTROLLER.nation.radix

%LWAPP-5-CHANGED: LWAPP changed state to JOIN

%LWAPP-3-CLIENTERRORLOG: Join Timer: did not recieve join response (controller - Fxxxxxxx)

%LWAPP-3-CLIENTERRORLOG: Set Transport Address: no more AP manager IP addresses remain

%SYS-5-RELOAD: Reload requested by LWAPP CLIENT. Reload Reason: DID NOT GET JOIN RESPONSE.

%LWAPP-5-CHANGED: LWAPP changed state to DOWN

IOS Bootloader - Starting system.

Xmodem file system is available.

The ap-manager interface is configured correctly and there isn't a duplicate IP address.

The LAP was initially stand alone and was converted to LWAPP.

The MTU over the WAN link is 1500 bytes.

All I'm getting from the WLC debugs is:

Mon Jul 20 11:42:59 2009: 00:22:xx:xx:xx:xx Received LWAPP DISCOVERY REQUEST from AP 00:22:xx:xx:xx:xx to 00:19:xx:xx:xx:xx on port '29'

Mon Jul 20 11:42:59 2009: 00:22:xx:xx:xx:xx LWAPP Discovery Request AP Software Version: 0x3003300

Mon Jul 20 11:42:59 2009: 00:22:xx:xx:xx:xx Successful transmission of LWAPP Discovery Response to AP 00:22:xx:xx:xx:xx on port 29

So basically the join messages don't seem to reach the WLC. In fact they don't even seem to reach the local router on the remote subnet. The discovery packets are seen on the local router but the joins don't seem to appear at all.

I'm not sure if it's a latency issue. Average latency over the WAN link is under 70ms.

I'm assuming the certificate on the WAP is MIC and the MAC details have been entered into the WLC AP Security policies for authentication. I'm not seeing any debugging messages relating to bad authentication at all.

I can't debug from the LAP as it's LWAPP, obviously.

I've been through many Cisco documents trying to troubleshoot the problem, including this http://www.cisco.com/en/US/products/ps6366/products_tech_note09186a00808f8599.shtml, but can't find a solution.

We're running WLC version 4.2.130.0.

Can anyone help?

Thanks

Brodie

37 REPLIES

Re: 1252 LAP won't join WLC

certificate issues can be detected on WLC with "debug pm pki enable". I also don't think you are having certificate issues. 1250 should have a MIC on it.

It looks like your AP is getting WLC's MGMT IP from DHCP option 43.

Can you ping WLC's AP-manager IP from AP's console? Is it possible you reached max limit of APs on your 3750 controller?

Try "show lwapp client config" on AP's console. Is it blank or is there config? Some config would indicate that this AP had already joined some controller before. Look for any issues in there.

New Member

Re: 1252 LAP won't join WLC

Thanks for the response. Yep seems to get DHCP option 43 details ok.

The AP is LWAPP so despite the console I don't know of a way to access command line functionality. Is there a way to get access?

We have three, 100 AP capacity, 4404 WLCs with 90, 88 and 82 LAPs associated respectively (restricted option 43 details for debugging purposes). As far as I know this means there is plenty of spare capacity on each controller. Unless the 100 capacity counts something else, like how many MAC addresses are added to the AP security policy or something?

Cheers

Re: 1252 LAP won't join WLC

You can run those LAP CLI commands through console same way you got that log in your first post.

Was this LAP converted from Autonomous AP or did it come as LAP?

I also just realized that you can't ping AP-Manager IP, WLC doesn't allow it

New Member

Re: 1252 LAP won't join WLC

Unless there is some sort of escape sequence I need to enter in order to access the LAP's command line then I can't access it. That said, the console is plugged into the local router from the LAP and I have accessed the session remotely through the router. I'm not sure if this has an effect on the ability to access the command line.

It was converted from autonomous AP yes.

Re: 1252 LAP won't join WLC

I assume you have connected to router's AUX and doing reverse telnet. You should be getting Password: prompt on your LAP's console. Password and Enable are both Cisco. Below is console output from my lab's 1250 LAP after erasing configuration (which can only be initiated from controller). In my case, the vlan is not configured with Option 43 and no proper DNS, so LAP doesn't join the controller.

By the way, your best bet might be to convert this LAP back to IOS and then back to LAP again. Use this method:

http://www.cisco.com/en/US/docs/wireless/access_point/conversion/lwapp/upgrade/guide/lwapnote.html#wp160918

Do you have "Authorize APs against AAA" checked under Security > AP Policies in any of your WLCs ?

Press RETURN to get started!

*Mar 1 00:00:07.099: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 0

*Mar 1 00:00:07.619: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 1

*Mar 1 00:00:08.595: %LINK-3-UPDOWN: Interface GigabitEthernet0, changed state to up

*May 10 23:17:25.199: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0, changed state to up

*May 10 23:17:26.155: %SYS-5-RESTART: System restarted --

Cisco IOS Software, C1250 Software (C1250-K9W8-M), Version 12.4(10b)JDC, RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2009 by Cisco Systems, Inc.

Compiled Fri 01-May-09 10:49 by prod_rel_team

*May 10 23:17:26.155: %SNMP-5-COLDSTART: SNMP agent on host ap is undergoing a cold start

*May 10 23:17:27.183: %SSH-5-ENABLED: SSH 2.0 has been enabled

*May 10 23:17:27.387: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset

*May 10 23:17:27.387: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset

*May 10 23:17:28.439: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to down

*May 10 23:17:28.439: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down

*May 10 23:17:30.783: %LWAPP-3-CLIENTERRORLOG: ../lwapp/lwapp_l2.c:152 - discarding msg type 12 in state 0

*May 10 23:17:30.783: %CDP_PD-4-POWER_OK: Full power - AC_ADAPTOR inline power source

*May 10 23:17:30.795: %DOT11-6-FREQ_SCAN: Interface Dot11Radio0, Scanning frequencies for 16 seconds

*May 10 23:17:44.571: %LWAPP-5-CHANGED: LWAPP changed state to DISCOVERY

*May 10 23:17:44.731: Logging LWAPP message to 255.255.255.255.

%LINK-3-UPDOWN: Interface Dot11Radio0, changed state to up

%LINK-3-UPDOWN: Interface Dot11Radio1, changed state to up

%LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset

%SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 255.255.255.255 started - CLI initiated

%LINK-3-UPDOWN: Interface Dot11Radio1, changed state to up

%LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset

%LINK-3-UPDOWN: Interface Dot11Radio0, changed state to up

%LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset

%LINK-3-UPDOWN: Interface Dot11Radio1, changed state to up

%LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up

%LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up

%DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 172.16.8.3, mask 255.255.255.0, hostname AP0022.558e.24bc

User Access Verification

Password:

AP0022.558e.24bc>en

Password:

AP0022.558e.24bc#show lwapp ?

client LWAPP Client Information

ids LWAPP IDS Information

ip LWAPP IP configuration

mcast LWAPP Mcast Information

reap LWAPP REAP Information

rm LWAPP RM Information

AP0022.558e.24bc#show lwapp client config

AP0022.558e.24bc#

AP0022.558e.24bc#ping 3.45.47.143

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 3.45.47.143, timeout is 2 seconds:

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

AP0022.558e.24bc#

New Member

Re: 1252 LAP won't join WLC

That's right, reverse telnetting through AUX port.

No the AAA box is not checked.

Thanks I think I'll give that conversion a go. The prompt just doesn't become available as it continuously reboots.

Hall of Fame Super Gold

Re: 1252 LAP won't join WLC

Hi Brodie,

Did you "prime" the LAP before deployment?

Can you ping the WLC Management IP Address from the LAP in question? If you can, in enable mode, can you type in the command lwap ap controller ip address ?

Hope this helps.

New Member

Re: 1252 LAP won't join WLC

No it wasn't primed but that hasn't been an issue with any of the other 1252 LAPs that have been connected over the WAN.

Oh and I can't seem to access the LAPs command prompt at all.

Thanks

New Member

Re: 1252 LAP won't join WLC

This is all the output from the LAP console session:

IOS Bootloader - Starting system.

Xmodem file system is available.

flashfs[0]: 3 files, 2 directories

flashfs[0]: 0 orphaned files, 0 orphaned directories

flashfs[0]: Total bytes: 31868928

flashfs[0]: Bytes used: 2329088

flashfs[0]: Bytes available: 29539840

flashfs[0]: flashfs fsck took 15 seconds.

Reading cookie from flash parameter block...done.

Base Ethernet MAC address: 00:22:90:a3:53:3a

Loading "flash:/c1250-rcvk9w8-mx/c1250-rcvk9w8-mx"...###################################################################

###############

File "flash:/c1250-rcvk9w8-mx/c1250-rcvk9w8-mx" uncompressed and installed, entry point: 0x3000

executing...

Restricted Rights Legend

Use, duplication, or disclosure by the Government is

subject to restrictions as set forth in subparagraph

(c) of the Commercial Computer Software - Restricted

Rights clause at FAR sec. 52.227-19 and subparagraph

(c) (1) (ii) of the Rights in Technical Data and Computer

Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.

170 West Tasman Drive

San Jose, California 95134-1706

Cisco IOS Software, C1250 Software (C1250-RCVK9W8-M), Version 12.4(10b)JA, RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2007 by Cisco Systems, Inc.

Compiled Wed 24-Oct-07 16:09 by prod_rel_team

Image text-base: 0x00003000, data-base: 0x003DC740

Initializing flashfs...

flashfs[1]: 3 files, 2 directories

flashfs[1]: 0 orphaned files, 0 orphaned directories

flashfs[1]: Total bytes: 31868928

flashfs[1]: Bytes used: 2329088

flashfs[1]: Bytes available: 29539840

flashfs[1]: flashfs fsck took 5 seconds.

flashfs[1]: Initialization complete....done Initializing flashfs.

cisco AIR-AP1252AG-N-K9 (PowerPC 8349) processor (revision A0) with 49142K/16384K bytes of memory.

Processor board ID FCW1231Z0HN

PowerPC 8349 CPU at 533Mhz, revision number 0x0031

Last reset from power-on

LWAPP image version 3.0.51.0

1 Gigabit Ethernet interface

32K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address: 00:22:90:A3:53:3A

Part Number : 73-10425-05

PCA Assembly Number : 800-27630-05

PCA Revision Number : A0

PCB Serial Number : FOC12301SG2

Top Assembly Part Number : 800-29039-02

Top Assembly Serial Number : FCW1231Z0HN

Top Revision Number : A0

Product/Model Number : AIR-AP1252AG-N-K9

o

^

% Invalid input detected at '^' marker.

Press RETURN to get started!

*Mar 1 00:00:06.839: %SYS-5-RESTART: System restarted --

Cisco IOS Software, C1250 Software (C1250-RCVK9W8-M), Version 12.4(10b)JA, RELEASE SOFTWARE (fc2)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2007 by Cisco Systems, Inc.

Compiled Wed 24-Oct-07 16:09 by prod_rel_team

*Mar 1 00:00:07.799: %LINK-3-UPDOWN: Interface GigabitEthernet0, changed state to up

*Mar 1 00:00:08.799: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0, changed state to up

*Mar 1 00:00:26.847: %LWAPP-5-CHANGED: LWAPP changed state to DISCOVERY

*Mar 1 00:00:26.999: Logging LWAPP message to 255.255.255.255.

%CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source

New Member

Re: 1252 LAP won't join WLC

continued from last post...

%CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source

Translating "CISCO-LWAPP-CONTROLLER"...domain server (255.255.255.255)

%DHCP-6-ADDRESS_ASSIGN: Interface GigabitEthernet0 assigned DHCP address 10.148.66.5, mask 255.255.255.0, hostname AP002

2.90a3.533a

%LWAPP-3-CLIENTEVENTLOG: Controller address 10.18.11.248 obtained through DHCP

%LWAPP-3-CLIENTEVENTLOG: Did not get log server settings from DHCP.

%LWAPP-3-CLIENTEVENTLOG: Did not get any DNS options from DHCP.

%LWAPP-3-CLIENTEVENTLOG: Performing DNS resolution for CISCO-LWAPP-CONTROLLER

%LWAPP-3-CLIENTERRORLOG: DNS Name Lookup: could not resolve CISCO-LWAPP-CONTROLLER

%SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 255.255.255.255 started - CLI initiated

%LWAPP-5-CHANGED: LWAPP changed state to JOIN

%LWAPP-3-CLIENTERRORLOG: Join Timer: did not recieve join response (controller - FAIRWLC3)

%LWAPP-3-CLIENTERRORLOG: Set Transport Address: no more AP manager IP addresses remain

%SYS-5-RELOAD: Reload requested by LWAPP CLIENT. Reload Reason: DID NOT GET JOIN RESPONSE.

%LWAPP-5-CHANGED: LWAPP changed state to DOWN

IOS Bootloader - Starting system. (repeat as the LAP has rebooted)

Xmodem file system is available. (repeat)

New Member

Re: 1252 LAP won't join WLC

It's interesting that our dhcp server gets the dhcp request and begins a lease for the LAP for an IP in the correct subnet. But the IP address just doesn't stick. I assume that's because the LAP doesn't join the WLC.

Re: 1252 LAP won't join WLC

I would prime it for a static IP address then once it joins the controller reset it back to option 43 after it has found the controller addresses. You would have to go onsite to do this but it is the easiest way to make sure it is up. You can also run LWAPP debugs to see if it ever attempts to join the controllers. If not, I suspect a routing issue exists.

Hall of Fame Super Gold

Re: 1252 LAP won't join WLC

Hi Brodie,

Do you have firewalls anywhere?

New Member

Re: 1252 LAP won't join WLC

Yes. All relevant ports are allowed. We see the LWAPP discovers ok, the LAP gets the IP details of the WLCs, it just seems like the Join LWAPP messages vanish.

Bronze

Re: 1252 LAP won't join WLC

If you are sure you had followed the troubleshooting steps listed at the link you provided(especially there're no mismatchs between WLC and LAP about time/certificate/regulatory, no warning mesg in the output of debug lwapp event and debug pm pki), you have to use some network analyze tools to capture the lwapp join request packets in every hop between WLC and LAP to find out which hop blocked the join request packet.

Hall of Fame Super Gold

Re: 1252 LAP won't join WLC

Hi Brodie,

I see that the LAP loaded the RCV image (very good!). Just wait for the "Press ENTER to continue." or something and then enter the command I posted previously in enable mode.

New Member

Re: 1252 LAP won't join WLC

Unfortunately that doesn't work. As far as I know, "Press RETURN to get started!" is left over from the stand-alone mode as the LAP has been converted to LWAPP. From there the LAP detects the LWAPP software and boots into LWAPP mode, beginning the LWAPP discovery process. At no point, despite the "Press RETURN to get started!" message, can I break into the command line of the LAP. This is, as far as I know, how Lightweight Access Points (LAPs) are supposed to operate, I shouldn't be allowed access to the command line.

Re: 1252 LAP won't join WLC

That's a common misconception. LWAPP image is basically an IOS image with a lot of features modified (local mac -> split mac, control is moved to WLC) and a very limited CLI. You can barely use any commands, but there are some that can be used. You can do a bunch of show commands (including "show lwapp"), and you can do "reload" for example. "Conf t" is not available. You can also do some debug commands.

Like I was saying before, if I were you, I'd convert it back to Autonomous with a very simple procedure (I posted URL before), but you'll need to connect a laptop running TFTP software directly to the LAP (with xover cable, unless your laptop's NIC is auto-mdix). Then convert it back to LWAPP using the conversion utility. You will be back to normal.

Try to telnet to one of your existing LAPs, and you can login with user Cisco, enable Cisco and run show commands. If you can't telnet to it, that's because telnet is by default (I think) disabled. You can enable it from WLC with:

"config ap telnet enable APNAME"

Re: 1252 LAP won't join WLC

I agree. Revert back to autonomous and then upgrade again on a local switch port to the controller. You may be running into the CAPWAP upgrade bug.

New Member

Re: 1252 LAP won't join WLC

Yeah I'll revert back. I think I'll send a fresh LAP to the remote location in the meantime. I'll get it to associate to a WLC first, assign it a static IP address on the remote subnet and then ship it out for installation and see how it goes.

Thanks for everyone's help :)

Re: 1252 LAP won't join WLC

paste here what you see on WLC for this command:

show ap join stats detailed

New Member

Re: 1252 LAP won't join WLC

I didn't know about this command, interesting...

(Cisco Controller) >show ap join stats detailed 00:22:90:A3:53:3A

Discovery phase statistics

- Discovery requests received.............................. 84

- Successful discovery responses sent...................... 84

- Unsuccessful discovery request processing................ 0

- Reason for last unsuccessful discovery attempt........... Not applicable

- Time at last successful discovery attempt................ Jul 22 10:16:24.144

- Time at last unsuccessful discovery attempt.............. Not applicable

Join phase statistics

- Join requests received................................... 0

- Successful join responses sent........................... 0

- Unsuccessful join request processing..................... 0

- Reason for last unsuccessful join attempt................ Not applicable

- Time at last successful join attempt..................... Not applicable

- Time at last unsuccessful join attempt................... Not applicable

Configuration phase statistics

- Configuration requests received.......................... 0

- Successful configuration responses sent.................. 0

- Unsuccessful configuration request processing............ 0

- Reason for last unsuccessful configuration attempt....... Not applicable

--More-- or (q)uit

- Time at last successful configuration attempt............ Not applicable

- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decrytion failure details

- Reason for last message decryption failure............... Not applicable

Last AP disconnect details

- Reason for last AP connection failure.................... Not applicable

Last join error summary

- Type of error that occurred last......................... None

- Reason for error that occurred last...................... Not applicable

- Time at which the last join error occurred............... Not applicable

(Cisco Controller) >?

clear Clear selected configuration elements.

config Configure switch options and settings.

debug Manages system debug options.

help Help

linktest Perform a link test to a specified MAC address.

logout Exit this session. Any unsaved changes are lost.

ping Send ICMP echo packets to a specified IP address.

mping Send Mobility echo packets to a specified mobility peer IP address.

eping Send Ethernet-over-IP echo packets to a specified mobility peer IP address.

reset Reset options.

save Save switch configurations.

show Display switch options and settings.

test Test trigger commands

transfer Transfer a file to or from the switch.

(Cisco Controller) >

Hall of Fame Super Gold

Re: 1252 LAP won't join WLC

Hi Brodie,

If you look at your post (20 July 2009, 12:27am PST), you'll notice that the "Press RETURN to get started!" is available.

New Member

Re: 1252 LAP won't join WLC

Yes but despite this message I can't break in to the command line. I can hit enter until the cows come home but at no stage can I access the command prompt. From there the LAP detects LWAPP mode and continues the discovery process without allowing command line access.

New Member

Re: 1252 LAP won't join WLC

%LWAPP-3-CLIENTERRORLOG: Set Transport Address: no more AP manager IP addresses remain

I know you said that IP isn't the issue but you usually see this message when you have a duplicate IP address. I have seen an AP obtain an IP address, discover the WLC and sometimes even join but not for more than 10 seconds before it reboots and goes through a continual cycle. This was down to a duplicate IP address. How is the DHCP being provided to the LAP? If it is the WLC itself, this isn't the most reliable DHCP server and will not detect duplicates. I would check and double check for conflicts!

Can you place a sniffer on the LAP switchport?

Re: 1252 LAP won't join WLC

I have seen this as well with duplicate IP addresses on the management and AP manager interfaces on the old code. You can't do that on 6.0 but I suspect you might have a duplicate with the gateway or virtual interface. Print out you config and try to ping all the interface addresses and AP addresses when you have the wireless equiptment powered down. If you get a reply from one of the addresses you have found a culprit.

New Member

Re: 1252 LAP won't join WLC

Thanks guys. Yeah I have looked into the dup IP address but only really the ap-manager interface. I'll investigate further the other configured IPs. Unfortunately the LAP is remote so not really possible to set up a sniffer.

New Member

Re: 1252 LAP won't join WLC

Yeah the WLC is the DHCP server for this wireless LAN. TBH this WLAN was set up before I began working here. I am in the process of migrating to a new WLAN that uses a more reliable DHCP server.

I'll reinvestigate IP conflicts, thanks. The thing is I get the no more AP manager IP addresses across three WLCs. My investigations so far have turned up no conflicts. For there to be three IP conflicts seems unlikely.

No sniffer possible unfortunately.

Re: 1252 LAP won't join WLC

Total number of APs and total licensed controllers? Also, are all controllers on the current time and date?

5630
Views
4
Helpful
37
Replies
CreatePlease to create content