Hi All,
I've deployed a pair of WLC4404-100 (in a mobility group) in an Enterprise Data Center, to control 50 lightweight APs located at a remote branch office, using L3 LWAPP transport mode. WLC software version is 4.0.179.8. AP Fallback is enabled. All APs joined to a Primary WLC; the second WLC functions as hot standby.
Users in the branch office use built-in Intel adapter (on Dell notebooks) to associate to the WLAN and use Windows XP WZC. The WLAN is configured to WPA1+WPA2 and authenticate to a backend Cisco ACS server.
Today an issue happened. 50% of the users can't get connected. I started troubleshooting at 11 A.M. Below is my findings:
ACS Server
----------
In Passed Authentications log, last successful authentication at 10:48 A.M. and no more new log after that. ACS service was confirmed running. From WLC, I can ping to the RADIUS server.
In Failed Attempts log, last log is at 9:24 A.M., with Authen-Failure-Code=Authentication session invalidated. No more new log after that.
WLC
---
Configured WLC to send syslog messages to my workstation. Noticed many of the following error messages:
2007-01-22 10:39:55 Local0.Error 10.0.0.1 [SECURITY] 1x_ptsm.c 407: MAX EAP retransmissions reached for mobile 00:13:ce:10:7f:ed
2007-01-22 10:40:01 Local0.Notice 10.0.0.1 [WARNING] apf_80211.c 2408: Received a message with an invalid supported rate from station 00:12:17:83:ec:b8.
2007-01-22 10:40:02 Local0.Error 10.0.0.1 [SECURITY] 1x_auth_pae.c 2417: Reached Max EAP-Identity Request retries (21) for STA 00:13:ce:10:82:b3
2007-01-22 10:40:03 Local0.Error 10.0.0.1 [SECURITY] 1x_bauth_sm.c 423: Backend Authentication SM: abortAuth: Authentication Aborted.
2007-01-22 10:45:05 Local0.Error 10.0.0.1 [SECURITY] 1x_eapkey.c 344: EAPOL Key message with invalid authenticator replay counter (got 00 00 00 00 00 00 00 02, expected 00 00 00 00 00 00 00 03) from mobile 00:16:6f:66:e6:c8
2007-01-22 10:58:41 Local0.Error 10.0.0.1 [SECURITY] dtl_net.c 1176: STA [00:16:6f:4b:ea:7f, 0.0.0.0] ARP (op 1) received with invalid SPA 10.238.4.248/TPA 10.238.4.1
2007-01-22 11:05:16 Local0.Error 10.0.0.1 [SECURITY] 1x_ptsm.c 391: MAX EAPOL-Key M3 retransmissions reached for mobile 00:16:6f:4b:ea:7f
On the Clients page, status=Associated but Auth=No. When I removed those clients, they still unable to get connected.
Resolution
----------
At 11:15 A.M. power off the Primary WLC. All the 50 APs failed over to the Secondary WLC. Clients able to authenticate successfully.
Attached are screenshots of WLC config:
- Client Exclusion Policies
- Timers
- WLAN
Is it a sign of intrusion? Or caveats related to WLC version 4.0.179.8? I've searched cisco.com and WLC Release Notes but can't find much information that helps.
Please advise.
Thank you.
B.Rgds,
Lim TS
