This error is sometimes seen in large scale deployments where large number of client authentication requests fill up the RADIUS queues on the WLC. The WLC should typically be able to recover automatically but in certain cases if there is a lot of latency in the AAA server responses the WLC gets stuck in this state for a long time (have seen recovery times of more than 30-40 mins).
Typical response times from AAA server in normal functioning are in low milliseconds but can spike up to 1-10 second period.
Most customers that I have worked with have typical radius timeout value set to 5 seconds and radius retries set to default of 5 retries.
(Cisco Controller) >show radius summary
Vendor Id Backward Compatibility............ Disabled
Credentials Caching......................... Disabled
Call Station Id Type........................ IP Address
Administrative Authentication via RADIUS.... Enabled
Aggressive Failover......................... Disabled
Keywrap..................................... DisabledAuthentication Servers
!--- This portion of code has been wrapped to several lines due to spatial!
Idx Type Server Address Port State Tout RFC3576
--- ---- ---------------- ------ -------- ---- -------
1 N 10.48.76.50 1812 Enabled 2 Enabled
this is how to configure the RADIUS timeout
This is how to configure:
config radius auth retransmit-timeout 1 5
During the RADIUS Queue full issue on the WLC, the WLC is start rejecting NEW authentication requests from the clients till more space is created in the RADIUS Queue to accommodate new auth requests. In the meanwhile, the WLC will continue to empty the Queue and processing the current authentications requests.
On WLC with version 7.6 you can use the devshell command below to see the accounting and authentication queues
(5508-5) >devshell printRadiusPktIdInQ
Printing Radius IDs in Queue
Queue  Count  ----> Accounting Queue
Queue  Count  -----> Authentication Queue
value = 2 = 0x2
(5508-5) >devshell printRadiusPktIdInQ
Printing Radius IDs in Queue
Queue  Count 
Queue  Count 
value = 2 = 0x2
On WLC running version 8.0
(5508-5) >show radius queue
Auth Alloc : 223642
Acct Alloc : 0
Auth Free : 223642
Acct Free : 0
Auth Alloc Err : 0
Acct Alloc Err : 0
Queue depth in above case is calculated by Auth alloc - Auth Free values.
The Queue depth for both Accounting and Authentication is 256 [0 - 255]
The WLC keeps rolling from 0 - 255 and back to 0 and so on.
Here if you take a WLC port channel capture (capture filter udp port 1812), you can see the RADIUS exchange between the WLC and the AAA server. I have a display filter applied sourced from the WLC's ip address and have applied a Wireshark column to display the 'RADIUS IDs'. if you see closely the ids rotate from 0-255 and back to 0.
You can have a quick view of the auth requests the WLC sends out and the responses by plotting a Wireshark IO graph. Here my graph 1 in black is the auth requests sent by the WLC (filter: radius.code == 1)
and graph2 in red is responses coming back from the AAA server (access challenge, access accept and access-rejects) (filter: radius.code==2 or radius.code==3 or radius.code==4)
typically you would like to see both the graphs follow close with each other. This can also show any spikes in the authentications per second seen/sent by the WLC.
The client debugs will show something like this
Client RADIUS authentication fails. "debug client" shows a message similar to this:
*Dot1x_NW_MsgTask_7: Dec 17 11:43:36.983: 00:11:22:33:44:55 Entering Backend Auth Response state for mobile f0:d1:a9:24:d8:a7
*Dot1x_NW_MsgTask_7: Dec 17 11:43:36.985: 00:11:22:33:44:55 Processing AAA Error 'Out of Memory' (-2) for mobile f0:d1:a9:24:d8:a7
*Dot1x_NW_MsgTask_7: Dec 17 11:43:36.999: 00:11:22:33:44:55 Sent Deauthenticate to mobile on BSSID 20:37:06:00:11:22 slot 0(caller 1x_auth_pae.c:1394)
at the same time, the msglog shows a message similar to this:
*Dot1x_NW_MsgTask_7: Dec 17 12:30:23.296: #DOT1X-3-ABORT_AUTH: 1x_bauth_sm.c:447 Authentication Aborted for client 00:11:22:33:44:55
and the traplog shows a message like this:
297 Mon Dec 17 12:36:29 2012 Client Deauthenticated: MACAddress:00:11:22:33:44:55
Base Radio MAC:20:37:06:00:11:22 Slot: 1 User
Name: unknown Ip Address: unknown Reason:Unspecif
ied ReasonCode: 1
1. try and reduce the auth storm/ reduce number of auths coming in
2. reduce latency between WLC-AAA.
3. can use multiple radius servers and point individual dot1s WLANs to different radius servers.
4. enable client exclusion to help alleviate the high auth rate
5. Can use multiple WLCs to split up the client load
... View more
Block Acknowledgement It is initialized by ADDBA request / response from between originator and recipient. after initiation blocks of QoS data are transmitted from originator to recipient. the originator can start transmitting the blocks of data after a polled TXOP or by winning the EDCA contention. the MPDUs within the block of transmitted frames are acknowledged by a BlockAck frame which is request by the originator in the BlockAckReq (BAR) frame. there are two flavors - Immediate Block Ack and Delayed Block Ack Immediate Block Ack and Delayed Block Ack differ in the way BAR and BA are handled. With Immediate Block Ack, the BA is required after the receipt of BAR whereas with Delayed Block Ack, the BAR itself is acknowledged (by recipient) with a simple Ack frame and the BA is sent later on separately which also gets acknowledged (by originator) separately. the originator or the recipient may tear down the Block Ack agreement by sending the DELBA (delete BA) Request which, if received successfully, is acknowledged with an Ack. Chart showing BlockAck mechanism a) Setup Originator first checks if the recipient STA is capable of Block Ack mechanism by checking Delayed Block Ack and Immediate Block Ack capability bits (as seen in Beacons, Association / Reassociation request, and Response frames). If the recipient is capable of Block Ack mechanism the originator sends a ADDBA Request frame indicating the TID (traffic ID) for which Block Ack is being set up. for Block Ack mechanism between HT-STAs the buffer size field in the ADDBA Request can be changed by the recipient of ADDBA Req frame. the recipient responds with the ADDBA Response frame and can chose to accept or reject the request. if the recipient rejects, the originator may not use the Block Ack mechanism. when recipient accepts, it indicates the number of buffers it allocates for this Block Ack agreement. Buffer size may be different for different Block Ack agreements. the originator changes the size of the transmission window based on the ADDBA response from the recipient. The originator may increase or decrease the size in accordance to the recipient response but is not greater than value of 64. the originator sets the A-MSDU supported field to 1 indicating it might transmit A-MSDU with the TID and recipient can set the same field to 1 to indicate it is capable of receiving an A-MSDU with this TID. The recipient can technically respond with any value of the A-MSDU supported field and if the originator does not like it, it can tear down the Block Ack agreement and send frames using normal ack. Block Ack Timeout Value: duration after which the Block Ack session is terminated when there are no frame exchanges. Start Sequence Number (SSN): the sequence number of the first data frame from the originator for this Block Ack session. b) Data and Block Ack Once the Immediate Block Ack or Delayed Block Ack is setup, the originator may transmit a block of QoS data frames separated by a SIFS (Short Inter-frame Space) duration with total number of data frames not exceeding buffer size as defined by ADDBA Response. the originator may do the following separate the Block and Basic BlockAckReq frames into separate TXOPs split a Block frame across multiple TXOPs split transmission of data MPDUs sent under Block Ack policy across multiple TXOPs Interleave MPDUs with different TIDs within same TXOP sequence or interleave MPDUs for different RAs within a TXOP Originator uses SSN to indicate to the recipient of the sequence number of first frame in the block for which acknowledgment is expected. Recipient maintains a Block Ack record which consists of originator address, TID and acknowledgment state of the data frames received from the originator. In case of Immediate Block Ack policy - recipient responds to basic BlockAckReq frame from originator with a basic BlockAck frame which indicates any missing frames. The originator retries any frames that are not acknowledged in the basic BlockAck frame in another block or individually. The difference with Delayed Block Ack policy is that the recipient responds to the BlockAckReq frame with a normal Ack and then transmits the BlockAck frame in a TXOP obtained later. The originator responds to the basic BlockAck with an Ack and then retries the unacknowledged frames from the BlockAck frame in another block or individually. In BlockAck frame the recipient only acknowledges the frames starting from the starting sequence number (SSN) until the highest sequence number that has been received correctly and sets the bit in the BlockAck bitmap of other frames (frames not received correctly from originator) to 0. The recipient reports the status of old and prior frames (frames before the first frame that the originator sends - SSN) as successfully recieved (bit is the bitmap set to 1). The recipient maintains a field called NextExpectedSequenceNumber which is set to 0 when the Block Ack mechanism is negotiated. if the recipient receives a frame with a sequence number older than the NextExpectedSequenceNumber for that Block Ack agreement than the recipient drops the frame thinking its either old or a duplicate. c) Teardown When originator has not more data to send and the final frame in the Block has been sent the originator signals the end of the Block Ack mechanism by sending the DELBA (Delete BlockAck) frame to the recipient. There is no response needed from the recipient. It just releases all resources allocated for that Block Ack agreement. The Block Ack agreement may be torn down if there are no BlockAck, BlockAckReq or QoS data frames (sent under the Block Ack policy) for the Block Ack's TID received from the peer within duration of the Block Ack timeout value. AP Debugs: debug dot11 d1 monitor address 7cd1.c392.3232 debug dot11 d1 tr pr clients xmt rcv ba *Apr 10 00:15:05.471: 8893F78B r 12 28/17/28/34 77- B000 030 EBBF4F 923232 EBBF4F 1600 auth l 6 *Apr 10 00:15:05.471: 8893F7A2-1 923232 - newauth *Apr 10 00:15:05.471: 8893F7AD-1 923232 - restart B0 300 *Apr 10 00:15:05.471: 8893F7CC-1 923232 - stop: re-assoc aid 1 *Apr 10 00:15:05.471: 8893F990 t 12 0 - B000 001 923232 EBBF4F EBBF4F 0000 auth l 37 *Apr 10 00:15:05.471: 8893FA55 r 12 29/18/29/35 76- 0000 030 EBBF4F 923232 EBBF4F 1610 assreq l 109 *Apr 10 00:15:05.471: 8893FA6E-1 923232 - restart 0 300 *Apr 10 00:15:05.475: 8893FEFF-1 923232 - clrfp *Apr 10 00:15:05.475: 8894003D t 12 0 - 1000 000 923232 EBBF4F EBBF4F 0000 assrsp l 123 *Apr 10 00:15:05.475: 889402A0-1 923232 - add request *Apr 10 00:15:05.475: 88940368-1 923232 - client restart pend - clear and set new key *Apr 10 00:15:05.475: 889405D6-1 923232 - add AID:(status ST_CAN_BF ) (mode SHORT_GI_20MHZ WME AMSDU_LONG AMSDU ) (vrates ) (age ) (blk ) (rate [00FCFFFFFF]) (AID ) (VLAN ) (0 ) (istatus ST_REQ_BF ST_AMSDU_REQUEUE ST_VLAN_ADDED ST_WAS_PSP ) (encr ) *Apr 10 00:15:05.475: 889405EB-1 923232 - uapsd_compliant_client 0 *Apr 10 00:15:05.479: 88941929 r m0-2 28/20/30/32 75- 8809 03C EBBF4F 923232 B92348 0000 q0 l36 ARP1 hdw 1 prot 800 7cd1.c392.3232 10.0.0.107 > 0000.0000.0000 10.0.0.1 0001 0800 0604 0001 7CD1 C392 3232 0A00 006B 0000 0000 0000 0A00 0001 *Apr 10 00:15:05.479: 8894199C r 12 34/24/35/40 70- 4801 030 EBBF4F 923232 EBBF4F 1620 null l0 *Apr 10 00:15:05.479: 88941A88-1 923232 - Request addba 0 *Apr 10 00:15:05.479: 88941DC3-1 923232 - xmt ADDBA req pri 0, seq E0F0, window 64 timeout 0 1 ----> transmit ADDBA req for priority 0, SSN 3599 *Apr 10 00:15:05.479: 88941DD3-1 923232 - fc DOT11_ACTION [D0] mode SHORT_GI_20MHZ WME AMSDU_LONG AMSDU  istatus ST_REQ_BF ST_AMSDU_REQUEUE ST_VLAN_ADDED ST_WAS_PSP  status ST_CAN_BF 10 *Apr 10 00:15:05.479: 88941DDA-1 923232 - fc DOT11_ACTION [D0] mode SHORT_GI_20MHZ WME AMSDU_LONG AMSDU  istatus ST_REQ_BF ST_AMSDU_REQUEUE ST_VLAN_ADDED ST_WAS_PSP  status ST_CAN_BF 10 *Apr 10 00:15:05.479: 88941EE5 t 12 0 - D000 C8EC 923232 EBBF4F EBBF4F 0000 action l 40 ----> same as above. actual frame transmission *Apr 10 00:15:05.479: 88942164 r m0-2 29/25/33/31 70- 8809 03C EBBF4F 923232 mFFFFFF 0010 q0 l336 C392 3232 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 *Apr 10 00:15:05.479: 889421E3 r m23-2 35/25/36/38 69- 8801 030 EBBF4F 923232 m333300 0020 q0 l56 SNAP 86DD 6002 839F 0008 3AFF FE80 0000 0000 0000 7ED1 C3FF FE92 3232 *Apr 10 00:15:05.479: 88942293 r 12 36/25/36/41 70- D000 030 EBBF4F 923232 EBBF4F 1630 action l 9 ---> received ADDBA response for priority 0 from client *Apr 10 00:15:05.483: 8894229C-1 923232 - fc DOT11_ACTION [D0] mode SHORT_GI_20MHZ WME AMSDU_LONG AMSDU  istatus ST_REQ_BF ST_AMSDU_REQUEUE ST_VLAN_ADDED ST_WAS_PSP  status ST_CAN_BF 10 *Apr 10 00:15:05.483: 889422AB-1 923232 - rcv ADDBA rsp pri 0, window 64 timeout 0 ----> same as above. above is actual frame *Apr 10 00:15:05.483: 889427A4 t m2-2 4 - 880A 000 923232 EBBF4F B92348 E0F0 q0 l58 ---> TID 0 data frame transmit to client (last 3 octect 92:32:32) ARP2 hdw 1 prot 800 ecc8.82b9.2348 10.0.0.1 > 7cd1.c392.3232 10.0.0.107 0001 0800 0604 0002 ECC8 82B9 2348 0A00 0001 7CD1 C392 3232 0A00 006B *Apr 10 00:15:05.483: 889428B2-1 923232 - send BAR 0 E100 *Apr 10 00:15:05.483: 889429EA t 12 0 - 8400 1750 923232 EBBF4F 0004 E100 bar ----> sent BAR (BlockAckReq) for priority 0 and SSN 3600 (E10) Note: the response to BAR which is BA is not seen in the debug outputs As you see above the clients indicates the missing frames which the originator may resend using another Block Ack mechanism or send them individually. Note: The aim of this article is to try and summarize the key points involved in the Block Ack mechanism. This article is by no means a comprehensive guide for the Block Ack process. For detailed info please refer to the IEEE 802.11-2012 standard.
... View more
This is a quick way to replay 802.11 frames using Commview for Wifi (am using the demo version). When you first launch the tool, there will be a pop up shown as follows which will show the list of compatible wireless nic cards and prompt you to install the drivers needed to put the wireless card into promiscuous mode. below you can see two adapters listed. I have tried and tested the EnGenius EUB1200AC usb adapter. Below shows the steps to capture and or replay the frames I am just covering the steps to capture and replay packets. first you choose the channel which you want to sniff from the right pane You just need to hit play to start capturing the frames on the designated channel. If using the demo version, a pop up with show up indicating the same and you will have to wait 15 seconds before you can hit start. the packets tab shows you the captured packets the icon (atom like shape) is the tab for the packet generator. you can right click on one of the captured packet and hit send selected or once you open the send packet window you can just drop a frame or multiple frames which can be replayed. In the Send Packet window you can use the hex editor to edit the various fields. You can also select the 802.11 rate at which you want to send the frame, if you want to send select number of frames or continuous.
... View more
The following shows the association, authentication and data traffic of a 11ac 3SS client using a 802.11ac and 802.11n sniffer. The main idea of this exercise is to show that the 802.11n sniffer will not be able to interpret and capture any 802.11ac encoded frames (mainly unicast QoS data frames). The capture on the left is the 802.11n capture and the one on the right is the 802.11ac capture. As you see all management and controller frames can be seen on both captures as they are sent using legacy rates to maintain backward compatibility in a mixed mode environment. Now in the above capture you see that the EAPOL keys 3 and 4 are still visible on both captures but the QoS data frames (priority 6 - TID 6) are only seen on the capture on the right (802.11ac capture). You can see some of the VHT information in the VHT information field of the QoS data like — 80 MHz channel is used, the STA is a 3 spatial stream (3 SS) STA, and MCS 8 (256 QAM 3/4). The 802.11n sniffing device cannot interpret these 11ac data frames.
... View more
The purpose of this tech note is to enable correct and successful converged access deployment for Branch deployment. Important Notes: It is very crucial to have a right design foundation before you deploy converged access 5760 WLC running in centralized mode is NOT converged access. Incorporate all best practices config below to ensure there are no surprises in terms of functional issues Decide on the correct code version depending on the feature set the customer requires. Software versions 3.3.5, 3.6.2aE and 3.7.1 (upcoming) are the codes to consider depending of the feature set requirements. Also, these versions may change when future maintenance releases come out. Below is a overview of branch wireless deployment models Comparison between traditional branch wireless deployment and Converged Access architecture Building the right architectural foundation Note: as you see above it is important to break away from the traditional method of deployment (one wireless vlan across the entire deployment). Recommended option is to have different client vlans per MA/MC (3850 and 3650). For roaming across switches, the default sticky anchoring feature automatically creates anchor / foreign pairs which aids in seamless roaming where client retains if ip address across roams. Even in case where sticky anchoring is disabled, there will be layer 3 roaming established which achieves seamless roams. Latency will not be a concern here as we are talking about branch deployment here where we would typically have 2-3 3650/3850 converged access switches. Typical Branch network deployment (Small and Large branch networks) Above figure shows both small and large branch networks Typically for a branch deployment the average number of APs range from 50-150 APs and client count ranges from 1000-2000 clients Depending on the size of the deployment, build basic blocks and expand as needed. As shown on the figure on the left, you can use 2-4 MA (3650/3850) and a single MC (3650/3850, single sub domain) per building. A single Switch Peer group (SPG) is sufficient per MC If the size if large than above point, you can just replicate the above design into 2 such sets or more (figure on the right) Typically at the branch sites each building has non-contiguous RF hence there is no need to build mobility peering between MCs. Mobility peering is only needed in cases where there is contiguous RF between buildings where seamless roaming is required. Typically for such a deployment you DO NOT need a standalone controller (5760) for AP and client management It is still recommended to use a GUEST Anchor controller like a AireOS 5508 WLC across WAN (at a central location) for guest users (central webauth using ISE) Best Practice Configuration Note: the below configs are tried and tested on multiple deployments and it simply justWORKS. You can simply tweak the configs per customer topology and quickly bring up the network. If for some reason there is a glitch, please leave a comment and we will fine tune the recommendations. Create necessary VLANs (management, clients and guest) DHCP Snooping and ARP inspection: Enable DHCP snooping and ARP inspection on the guest VLAN. Configure L2 trunk connected to the network as trusted for ARP inspection and DHCP snooping Security: Convert relevant authentication commands on the access switches to their Class-Based Policy Language (CPL) equivalents. Note: this command permanently converts the legacy configuration on the switch to identity-based networking services. On entering this command, a message is displayed asking for your permission to continue. Please permit the conversion. Wireless Management Interface Config AAA Configs WLAN Configs Mobility Config (Guest Anchor Controller) Mobility Config (Branch switch) AVC RF Group Enable Fast SSID change wireless client fast-said-change Note: You add build advanced features set on top of these configs as per deployment needs. Use of Cisco Prime Infrastructure workflow (use of templates to deploy multiple branch network within a few minutes) - Branch deployment Automation Cisco PI 2.2.1 (and above running) Wireless Tech Pack 1.0.0 introduces templates for converged access (Small and Large network templates) which enables deploying converged access network with just a few clicks across multiple locations (if needed) at the same time.
... View more
LDPC codes are a class of linear block codes characterized by sparse parity check matrices ‘H’. ‘H’ has low density of 1’s. LDPC codes were originally invented by Robert Gallager in the early 1960s but were largely ignore till they were ‘rediscovered’ in the mid 90s by MacKay. Sparseness of ‘H’ can yield large minimum distance ‘D min’ and reduces coding complexity. can perform within 0.0045 dB of Shannon limit.
Representations for LDPC Codes
Basically there are two different possibilities to represent LDPC codes. Like all linear block codes they can be described via matrices. The second possibility is a graphical representation.
1. Matrix Representation
Lets look at an example for a LDPC matrix first. The matrix defined in equation (1) is a parity check matrix with dimension n x m for a (8,4) code. We can now define 2 numbers describing these matrix. ‘Wt’ for the number of 1’s in each row and ‘Wc’ for the columns. For a matrix to be called low-density the two conditions Wc << n and Wt << m must be satisfied. In order to do this, the parity check matrix should usually be very large, so the example matrix below can’t be really called low-density.
2. Graphical Representation
Tanner introduced an effective graphical representation for LDPC codes. Not only provide these graphs a complete representation of the code, they also help to describe the decoding algorithm. Tanner graphs are bipirtate graphs. That means that the nodes of the graph are separated into two distinctive sets and edges are only connecting nodes of two different types. The two types of nodes in a tanner graph are called variable nodes (v-nodes) and check nodes (c-nodes). Below figure is an example of such a Tanner graph and represents the same code as the matrix in (1). The creation of such a graph is rather straight forward. It consists of m check nodes (the number of parity bits) and n variable nodes (the number of bits in a codeword).
The Equations for a simple LDPC code with n=12
There are actually only 7 independent equations so there are 7 parity digits.
The Parity check matrix for a simple LDPC code
Note above that each symbol is contained in 3 equations and each equation involves 4 code symbols.
Graph for simple LDPC code
Performance and Complexity
The feature of LDPC codes to perform near the Shannon limit of a channel exists only for large block lengths. For example there have been simulations that perform within
0.04 dB of the Shannon limit at a bit error rate of 10^(-6) with a block length of 10^(7). An interesting fact is that those high performance codes are irregular. The large block length results also in large parity-check and generator matrices. The complexity of multiplying a codeword with a matrix depends on the amount of 1’s in the matrix. If we put the sparse matrix H in the form [P^( T )I] via Gaussian elimination the generator matrix G can be calculated as G=[IP]. The sub-matrix P is generally not sparse so that the encoding complexity will be quite high.
Since the complexity grows in O(n^ 2 ) even sparse matrices don’t result in a good performance if the block length gets very high. So iterative decoding (and encoding) algorithms are used. Those algorithms perform local calculations and pass those local results via messages. This step is typically repeated several times. The term "local calculations" already indicates that a divide and conquere strategy, which separates a complex problem into manage able sub-problems, is realized. A sparse parity check matrix now helps this algorithms in several ways. First it helps to keep both the local calculations simple and also reduces the complexity of combining the sub-problems by reducing the number of needed messages to exchange all the information. Furthermore it was observed that iterative decoding algorithms of sparse codes perform very close to the optimal maximum likelihood decoder.
Decoding LDPC Codes
Decoding is quite simple – the most powerful algorithm is to use belief propagation – each check node in generation graph (or generation matrix, which is just a different representation of the same entity) sends to codeword nodes what they belief a valid bit with calculated probability, after error probably that given bit is zero or one becomes less than requested number (according to Shannon’s theorem code, which allows to reduce error probabily infinitely, always exist for given rate and communication capacity), codeword calcualtion (decoding) is completed. There are two mechanisms – hard and soft decision algos, the former is simpler, but the latter frequently is faster. Let me show an example of the hard decision algorithm
Let generation matrix to be this (not very sparse) set:
0 1 0 1 1 0 0 1
1 1 1 0 0 1 0 0
0 0 1 0 0 1 1 1
1 0 0 1 1 0 1 0
And source codeword is:
1 0 0 1 0 1 0 1
Check word, calculated my multiplication of matrix and codeword is:
0 0 0 0
Let’s during transmission of the codeword and check word over the channel codeword was changed to this (secod bit changed):
1 1 0 1 0 1 0 1
Here is a generation graph (originally proposed by Tanner) of the given matrix:
Here starts decoding algorithm, where each codeword node C first sends its bit to each check node F . F0 node will receive 1 1 0 1 bits from C1, C3, C4 and C7 accordingly. F1 node will receive 1 1 0 1 bits F2 node will receive 0 1 0 1 bits F3 node will receive 1 1 0 0 bits
Next step is to calculate the answer for each code node. Received check word is 0 0 0 0 (calculated above), so set of simple equations starts here. Each check node F gets three out of four received bits and XORing (summing modulo 2, since this example works in Galois finite field of power of 1 – GF(1)), and sends to the codeword node a bit it expects to be correct to satisfy received check bit. Here is an example for first check node:
X0 ^ 1 ^ 0 ^ 1 = 0. X0 = 0
1 ^ X1 ^ 0 ^ 1 = 0. X1 = 0
1 ^ 1 ^ X2 ^ 1 = 0. X2 = 1
1 ^ 1 ^ 0 ^ X3 = 0. X3 = 0
Then we send Xi to Ci code bits. After all check nodes are processed, codeword nodes has following set of bits: C0: 0 from F1, 1 from F3, 1 from originally received codeword. C1: 0 from F0, 0 from F1, 1 from originally received codeword. C2: 1 from F1, 0 from F2, 0 from originally received codeword. C3: 0 from F0, 1 from F3, 1 from originally received codeword. C4: 1 from F0, 0 from F3, 0 from originally received codeword. C5: 0 from F1, 1 from F2, 1 from originally received codeword. C6: 0 from F2, 0 from F3, 0 from originally received codeword. C7: 1 from F0, 1 from F2, 1 from originally received codeword.
Then using the voting for each bit (i.e. which bit has more ‘votes’ in above table out of three cases), we get a new codeword:
1 0 0 1 0 1 0 1
The same steps then are repeated until cdeword stopped to change. In our case we get it after the first run.
Soft decision algorithm usses essentially the same logic, but it operates with probabilities of the bit to be 1 or 0, each probabilities are recalculated in each run, and after probability is higher than requested value (or error probability is less than requested value), loop stops.
Real world examples use much bigger codewords (up to several thousands of bits), but logic is always the same.
Note: you need a background in linear codes theory to better understand the content of the article.
Reference: http://www.ioremap.net/projects/ldpc/ http://www.csee.wvu.edu/~mvalenti/documents/TurboLDPCTutorial.pdf http://www.bernh.net/media/download/papers/ldpc.pdf http://sites.ieee.org/scv-pace/files/2013/06/FMS13-LDPC-Module1.pdf
... View more
Transmitting information - There are 3 main steps involved in transmitting a signal over the air: A carrier signal which is generated at the transmitter The carrier is modulated with the information to be transmitted. Any reliable detectable change in signal characteristics can carry information. At the receiver the signal modifications or changes are detected and demodulated. Signal Characteristics that can be modified - amplitude, phase and frequency In AM, the amplitude of a high-frequency carrier signal is varied in proportion to the instantaneous amplitude of the modulating message signal. Frequency Modulation (FM), in FM the amplitude of the carrier is kept constant while its frequency is varied by the modulating message signal. Amplitude and phase can be modulated simultaneously and separately, but this is difficult to generate, and especially difficult to detect. Instead, in practical systems the signal is separated into another set of independent components: I (In-phase) and Q (quadrature). These components are orthogonal and do not interfere with each other. A simple way to view both amplitude and phase is with the polar diagram. The carrier becomes the frequency and phase reference and the signal is interpreted relative to the carrier. The signal can be expressed in polar form as a magnitude and a phase. The phase is relative to a reference signal, the carrier in most communication systems. The magnitude is either an absolute or relative value. I/Q formats This is the rectangular representation of the polar diagram. On a polar diagram, the I axis lies on the zero degree phase reference, and the Q axis is rotated 90 degrees. The signal vector’s projection onto the I axis is it ‘I’ component and the projection of the Q axis is its ‘Q’ component. Digital modulation is easy to accomplish with I/Q modulators. Most digital modulation maps the data to a number of discrete on the I/Q plane. These are known as constellation points. As the signal moves from one point to another, simultaneous phase and amplitude modulation usually results. To accomplish this with an amplitude modulator and a phase modulator is difficult and complex. Alternatively, simultaneous AM and phase modulation is easy with an I/Q modulator. The I and Q control signals are bounded, but infinite phase wrap is possible by properly phasing the I and Q signals. In 16 QAM, there are four I values and four Q values. This results in a total of 16 possible states for the signal. It can transition from any state to any other state at every symbol time. Since 16 = 2^4, four bits per symbol can be sent. This consists of 2 bits for I and 2 bits for Q. Here the transmitted symbol ‘0000’ is represented by a modulated signal phase of 225 degrees and a normalized amplitude of 0.33 (the four outer corners of the constellation have a normalized amplitude of 1). The modulated RF signal is the vector sum of an I channel signal whose amplitude is a normalized value of 0.23 with a relative phase of 180 degrees, and a Q channel signal whose amplitude is normalized value of 0.23 with a relative phase of 270 degrees. Here the transmitted symbol ‘0001’ is represented by a modulated signal phase of 255 degrees and a normalized amplitude of 0.75. The modulated RF signal is a vector sum of an I channel signal whose amplitude is a normalized value of 0.23 with a relative phase of 180 degrees, and a Q channel signal whose amplitude is a normalized value of 0.707 with a relative phase of 270 degrees. Similarly, other values are calculated. QAM modulation with ‘M’ symbols is known as M-QAM, for example 16-QAM, 256-QAM etc. Higher value of ‘M’ are used on channels with low levels of noise and distortion. Constellation sizes that are even powers of 2 (M =2, 4, 16, 64, ..) are typically used to make the constellation the same in both axes and simplify implementation. However, non-square constellations are also used for low values of ‘M’ or where maximum power efficiency is desired. For example, here is an example of a non square constellation for M = 8 (3 bits / symbol) 64-QAM constellation 256-QAM constellation Required receive sensitivity for different modulation and coding rates 802.11ac MCS
... View more
Following is the general MAC header format for a 802.11n frame Frame Control Field (2 Octets) The Type and Subtype fields together define the type of frame Type Value (bits B3,B2) Subtype Value (bits B7,B6,B5,B4) Type Description Subtype Description 00 0000 Management Association Request 00 0001 Management Association Response 00 0010 Management Re-Association Request 00 0011 Management Re-Association Response 00 0100 Management Probe Request 00 0101 Management Probe Response 00 0110 Management Timing Advertisement 00 0111 Management Reserved 00 1000 Management Beacon 00 1001 Management ATIM 00 1010 Management Disassociation 00 1011 Management Authentication 00 1100 Management Deauthentication 00 1101 Management Action 00 1110 Management Action No Ack 00 1111 Management Reserved 01 0000-0110 Control Reserved 01 0111 Control Control Wrapper 01 1000 Control Block Ack Request (BlockAckReq) 01 1001 Control Block Ack (BlockAck) 01 1010 Control PS-Poll 01 1011 Control RTS 01 1100 Control CTS 01 1101 Control ACK 01 1110 Control CF-End 01 1111 Control CF-End + CF-Ack 10 0000 Data Data 10 0001 Data Data + CF-Ack 10 0010 Data Data + CF-Poll 10 0011 Data Data + CF-Ack + CF-Poll 10 0100 Data Null (no data) 10 0101 Data CF-Ack (no data) 10 0110 Data CF-Poll (no data) 10 0111 Data CF-Ack + CF-Poll (no data) 10 1000 Data QoS Data 10 1001 Data QoS Data + CF-Ack 10 1010 Data QoS Data + CF-Poll 10 1011 Data QoS Data + CF-Ack + CF-Poll 10 1100 Data QoS Null (no data) 10 1101 Data Reserved 10 1110 Data QoS CF-Poll (no data) 10 1111 Data QoS CF-Ack + CF-Poll (no data) 11 0000-1111 Reserved Reserved
... View more
Transmit beamforming requires the knowledge of the channel state to compute a steering matrix that is applied to the transmitted signal to optimize reception at one or more receivers. The STA transmitting using the steering matrix is called the VHT beamformer and the STA for which the reception is optimized is called the VHT beamformee. Beamforming is directly enabled by the support of ‘sounding’. Sounding is the term used to denote the process performed by the transmitter to acquire CSI from each of the different users by sending training symbols and waiting for the receivers to provide explicit feedback containing a measure of the channel. This feedback is then used to create a weight or steering matrix that will be used to pre-code the data transmission by creating a set of steered beams to optimize the reception at one or multiple receivers. Any device that shapes its transmitted frames is called a beamformer, and a receiver of such frames is called a beamformee. A single device may act both as a beamformer and a beamformee. The process of beamforming involves measurement of the MIMO channel, and as a result of the channel measurement, a derivation of the steering matrix is done. The steering matrix is a precise mathematical description of how the antenna array should use each individual element to select spatial path for the transmission. Types of feedback mechanism - For an HT beamformer to calculate the appropriate steering matrix for transmit spatial processing when transmitting to a specific HT beamformee, the HT beamformer needs to have an accurate estimate of the channel over which it is transmitting. There are two methods which can be used :- Implicit feedback: in this method the HT beamformer receives long training symbols transmitted by the HT beamformee. This allows the MIMO channel between the HT beamformer and HT beamformee to be estimated. If the channel is reciprocal, the HT beamformer can use the training symbols it receives from the HT beamformee to make a channel estimate suitable for computing the transmit steering matrix. Explicit feedback: When using explicit feedback, the HT beamformee makes a direct estimate of the MIMO channel from the training symbols sent to the HT beamformee by the HT beamformer. The HT beamformee may prepare CSI or steering feedback based on an observation of these training symbols. The HT beamformee quantizes the feedback and sends it to the HT beamformer. The HT beamformer can use the feedback as the basis for determining transmit steering vectors. In 802.11ac, only explicit beamforming is used, hence both the transmitter and receiver must support it. VHT NDP sounding procedure - A VHT beamformer shall initiate a sounding feedback sequence by transmitting a VHT NDP (Null Data Packet) Announcement frame followed by a VHT NDP after a SIFS. The VHT beamformer shall include in the VHT NDP Announcement frame one STA info field for each VHT beamformee that is expected to prepare VHT compressed beamforming feedback and shall identify the VHT beamformee by including the VHT beamformee’s AID in the AID subfield of the STA Info field. The VHT NDP Announcement frame shall include at least one STA info field. Sounding protocol with a single VHT beamformee Sounding protocol with more than one VHT beamformee VHT NDP Announcement frame format (single user) Upon transmission of the VHT NDP Announcement frame, the beamformer next transmits a Null Data Packet frame shown below. Figure shows a PLCP frame with no data field, so there is no 802.11 MAC frame. Channel sounding can be carried out by analyzing the received training symbols in the PLCP header, so no MAC data is needed in a NDP. Within a NDP there is one VHT Long Training Field (VHT-LTF) for each spatial stream used in transmission, and hence in the beamformed data transmission. More specifically, upon reception of the VHT NDP frame each beamformee removes the space-time stream CSD (cyclic shift diversity) applied to the signals transmitted. The CSD consists of a signal shaping technique where different phase shifts are applied to the same signal across different transmit chains. After removing the CSD, the targeted beamformees are required to reply with a VHT compressed beamforming frame. The first intended stations replies immediately whereas the others have to wait to be polled by the beamformer (by using the Beamforming Report Poll). The most relevant information carried by the VHT Compressed Beamforming Frame is as follows - The VHT MIMO Control Field which contains the dimension of the matrix, an indicator of the width of the channel in which the measurements used to create the feedback matrix were taken, and information indicating the size of the codebook entries. The VHT Compressed Beamforming Report containing the compressed beamforming feedback matrix in the form of two angels, as well as SNR of each space-time stream averaged over all subcarriers used. MU Exclusive Beamforming Report carrying explicit information used by a multi-user beamformer in order to create the steering matrices. VHT NDP frame format Calculating the feedback matrix - calculating the feedback matrix can begin only after receiving the NDP from the beamformer. Once the NDP is received, each OFDM subcarrier is processed independently in its own matrix that describes the performance of the subcarrier between each transmitter antenna element and each receiver antenna element. The contents of the matrix are based on received power and phase shifts between each pair of antennas. feedback matrix is transformed by matrix multiplication called Givens rotation, which depends on parameters called ‘angels’. Rather than transmitting the full feedback matrix , the beamformee calculates the angels based on the matrix rotation. 802.11ac protocol specifies the order in which these angels are transmitted so that the beamformer can receive a long string of bits and appropriately delimit each angels. Having calculated the angels, the beamformee assembles them into compressed feedback form and returns them to the beamformer. Only one set of angels is required to summarize the radio link performance for all of the OFDM subcarriers. The set of angels can be quite large with a wider channels. The beamformer receives the feedback matrix and uses it to calculate the steering matrix for transmissions to the beamformee. One feedback matrix is sent by each beamformee. In SU beamforming, there is one feedback matrix from the beamformee and one steering matrix used. In MU beamforming, each beamformee send a feedback matrix and the beamformer needs to maintain a steering matrix for each client. When transmitting the feedback matrix, there are three main factors that determine its size. First, wider channels have more OFDM subcarriers, so the feedback matrix must be larger to accommodate them. Second, higher the number of pairwise combinations of transmitter and receiver antennas is, the larger the matrix will be. Finally, 802.11ac allows two different representations of the angels value to enable devices to use higher resolution when necessary. MU MIMO requires higher resolution because of the need to avoid inter-user interference. 802.11ac compressed V-Matrix feedback report sizing - References: 802.11ac IEEE standard, 802.11ac: A Survival Guide, Aruba Networks whitepaper (WP_80211acInDepth.pdf)
... View more
This document explains about the commands needed for the conversion from Mesh AP mode to Local mode and Vice-versa. This is feature introduced in the CUWN 8.0.
How to configure the Wireless LAN Controller (WLC) and lightweight access point (LAP) for basic operation. Basic Knowledge of Mesh and Local AP modes.
The information in this document is based on all AP hardware models connected to controllers on 8.0 CUWN version.
Behaviour Before 8.0:
An AP in Bridge mode needs to first join a WLC that is configured with the proper AP MAC address in its Auth-list before you can change that AP’s mode.
Configure the AP in CUWN 8.0 and onwards:
capwap ap mode local
capwap ap mode bridge
This command will cause the AP to reload.
Before switching an AP from Local to Bridge mode ensure that the AP has an image with a full radio support (…-k9w8-…) so that it can support Radios and that the AP MAC address has been added to the WLC
Issue the following command on the AP and the WLC to verify if it has changed its mode
On the AP:
show capwap client config
On the WLC
show ap config general
... View more
Below are some useful debugs to collect while working with TAC for CWA issue.
Description: these debugs are mainly for a scenario in which the end device is stuck in a redirect loop. Clients connect to CWA ssid and gets the AUP page with accept button. clicks on accept button and gets the AUP page again.
Here are the debugs/traces/show commands we plan to use on 5760, ISE and client side.
set trace group-wireless-secure filter mac xxxx.xxxx.xxxx
set trace aaa wireless events level debug
set trace aaa wireless events filter mac xxxx.xxxx.xxxx
set trace group-wireless-secure level debug
debug client mac-address
debug aaa wireless all
debug ip http transactions
debug ip http url
debug ip socket error
debug authentication all
debug authentication feature spi al
debug epm all
debug epm plugin acl all
debug epm plugin redirect all
debug epm plugin redirect detail
“log to buffer" “save to ftp" “confirm debug level"
logging buffered 16000000
no logging rate-limit
show wireless client mac-address detail
show authentication session mac detail
show platform acl le | be
Wireshark if possible on laptop during failure and working.
Client mac address, model, ios and browser type on all clients being tested/reported.
Verify it works when opening new tab or new browser and if original browser fails does it work if you go back to original browser.
GUI > Administration > logging > debug log config > click on node > runtime-aaa = debug.
GUI > Administration > logging > debug log config > click on node > > guestportal = debug
GUI > Administration > logging > debug log config > click on node >> guestauth = debug
After issue happen go to Operations > download logs > click on node > click on ‘include debug logs’ and ‘include monitoring and reporting logs.
Add encryption key then create support bundle. After completion, download the bundle.
TCPDump > operations > troubleshoot > tools > tcpdump > select node > filter = udp port 1700.
Operations > reports > Radius Authentications > filter = endpoint ID = mac address > RUN.
... View more
Hi All, I need you feedback about what information do you have about converged access? have you heard about it? played around with 3850/3650/5760? have you deployed it? what reviews do you have about it? We are working on some exciting stuff around Cisco Prime Infrastructure and Converged access solution which will enable quick deployment of several campus and branch locations at the same time. Please share your thoughts about the architecture. If you have deployed the architecture or are considering deploying it, we can discuss more about it. Most people don’t really understand the power of this architecture and how the devices can be leveraged in different network requirements from an architectural standpoint. Looking forward to the comments :)
... View more
In recent times I have come across the question ‘Why do we need m-gig AP backhaul’. I have seen a lot of Wi-Fi experts quickly respond to that question saying ‘You’ll be fine as is and your AP backhaul will not exceed 1Gbps’. I have even heard statements like ‘it is impossible’ to exceed 1Gbps AP backhaul capacity and they back that up with numerous points around 160 MHz channels, 256 QAM etc (some facts, some approximations). I am not challenging anyone’s views but I won’t be so quick to jump to the conclusion that ‘You’ll be ok without m-gig AP backhaul for 802.11ac wave 2’. Here is my personal opinion about the big buzz around Multi-Gigabit for AP backhaul. I am going to try and use less mathematics or any fancy simulator tools that throw out real/theoretical numbers at you. Frankly speaking its not a 'one size fit all’ model. These numbers (explained later) some people talk about may or may not apply to your deployment. Ok enough of my blabbering, lets dive right into it. We already see 4SS SOHO APs out in the market with Enterprise grade 4SS 11ac Wave 2 APs ready to roll out very soon. They will indeed support the mystical 160 MHz channels with a theoretical (as most people like to refer :) ) link speed (data rate) of 3.466 Gbps. This piece of information in itself warrants for m-gig AP backhaul. Lets talk more sense; lets not talk about 160 MHz channel at all from this point on. Now coming to the second impractical data around 802.11ac which is 256 QAM. All those who know what 256 QAM means will know that its rather difficult to sustain. You require very clean RF (will avoid going into SNR and receiver sensitivity etc) to achieve and maintain 256-QAM. Lets say the 11ac client does land up achieving (in a RF isolation room, low density area ;) - just kidding as this is the normal assumption) 256-QAM which corresponds to MCS 8 and MCS 9. Lets look at a more real world scenario i.e. 80 MHz channels. MCS 9 for a 4SS 11ac AP / Client using 80 MHz channel approximates to 1.7 Gbps link speed, at about 65-70% throughput efficiency it is 1.1 Gbps (.65 x 1.7 Gbps). Does this warrant m-gig backhaul? Maybe not just yet right? As most of you will point out mixed-client environment and less than optimal RF conditions which make it difficult to reach MCS 9. Lets rate shift down to MCS 7 at 80 MHz which is 1.3 Gbps. Doing the math again for 65-70% throughput efficiency we arrive at approximately 850 Mbps (hurray!!). We finally arrived to an over the air data rate value which will not over subscribe or stress the 1 Gbps AP backhaul. This is about 85% of your existing AP backhaul capacity. 802.11ac MCS rates table http://wirelessonthego.postach.io/post/802-11ac-mcs-rates When most people explain these 'over the air' data rate / link speed numbers they are only talking about ‘upstream’ traffic from the wireless client to the AP. What we don’t consider are things like unwanted downstream traffic hitting the AP port and potential inside the AP packet processing drops. Speaking out of experience (4 plus years in Wi-Fi support) I’ve seen scenarios where downlink traffic other than traffic destined to the wireless clients hits the AP port. Link local broadcasts are very common these days with so many mobile devices invading the Wi-Fi space. These protocols are very chatty and can quickly add up to few tens or hundreds of Mbps. I have personally been involved in troubleshooting issues where the AP port is stressed in terms of traffic ( > 1Gbps) causing packet drops. I do agree that preventive measures can be taken and networks can be designed properly to avoid such cases but more often than not networks are not designed so accurately. Moving on to MU-MIMO; this is the capability of the 11ac AP to make use of unutilized spatial streams for other clients. What this means is that a 4SS AP can ideally serve Two or three 1SS client (usually mobile devices); with 4th radio used for beamforming Two 2SS clients And more! This will likely make the AP utilize its top end link speed of 1.3 Gbps more often Although MCS 8 and 9 may not be achieved 100% of the time, they will come into play some % of time the client remains associated with the 11ac Wave 2 AP. There are no 4SS 11ac clients in the market at present but it is likely that will change in the near future. We already have 3 SS 11ac clients which can achieve 975 Mbps at MCS 7. This is where MU-MIMO would come in handy to best utilize the AP’s top end link speed. Above data is only the contribution by 5 GHz radio, what about the 2.4 GHz 802.11n radio? That will add an average wireless traffic of about 100-150 Mbps. This m-gig dilemma is somewhat similar to the time when 11n hit prime time and was shipped with GigE ports instead of FastEthernet backhaul. How many deployments do you see out in the field where 11n APs are plugged into FastEthernet ports? Probably only a handful. This is why it maybe a good idea to have m-gig switches for 11ac wave 2 APs which will ship with some sort of m-gig capability. Now lets consider the mixed-client environment. Although older generation (11a/g, 11n etc) Wi-Fi clients are almost always present in the environment but there are a lot of scenarios where you have a mini greenfield deployment if you consider BSSID level coverage. I see deployments like classroom environment which just have latest generation macbook pros or iPads, or office spaces where you only have the latest generation windows / mac laptops which are all 11ac capable. If you are pondering over the question, ‘will applications exceed the 1 Gbps mark?' There are several use cases where large amounts of data needs to be exchanged over Wi-Fi (in excess of 1 Gbps), and the data demands continue to grow at a rapid pace. For most of us cost tends to play an important role in decision making. Highlighted are a few areas of investment to be considered for 11ac wave 2 and m-gig technology: Cabling cost 11ac wave 2 APs M-gig capable access switches A typical refresh cycle for AP is around 4-7 years. When it comes to access switches the refresh cycle is even longer 7-9 years. The point I am trying to make here is if its time for an upgrade or refresh, its a no brainer to be ready for 11 ac wave2 full theoretical capacity (6.9 Gbps) and m-gig capable switches. For folks who are still on 11n and are thinking of making the transition to 11ac, again the choice is simple, skip 11ac wave 1 and directly adopt 11ac wave 2 as thats a more logical thing to do. This would apply to the SMB markets as well. Below is a chart showing the proposed channels to the existing 5GHz spectrum which will make the use of 160MHz channels more feasible. source: FCC Both these technologies are here to stay and things are just starting to get exciting! I’d love to see your comments on my views.
... View more
The converged access lifecycle includes five phases starting from design to deployment phases. In each phase, network administrator should follow the procedure to successfully deploy converged access using the workflow. Following Figure illustrates the five workflow phases where each phase may require work tasks beyond using the Cisco Prime Infrastructure. The converged access workflow lifecycle starts with design phase, and goes through additional phases as shown in Figure above. Design – It is the first phase in the workflow lifecycle, and involves selecting the Converged Access deployment model and determining the guest network service requirements – Plan – The network administrators must plan how they would like to deploy the WLANs and their associated services across their network, such as WLAN, VLAN and wireless policy setings. Prepare – This is the readiness stage of the converged access deployment plan. In this phase, the network administrator must ensure that the network foundation has the required layer 2/3 configuration applied prior to integrating converged access, for example, client VLAN. This step also requires certain wireless specific pre- requisites which are not supported in the workflow today. The network administrator ensures that the network complies with the following key requirements. Automate – This phase automates the configuration of wireless on single or multiple sites based on planning, after the required elements in prepare stage are in place and operational. If the common configuration parameters are exported and saved, future similar deployment automation can be further optimized by importing these configuration values instead of manually entering them again. Validate – This phase validates the deployment. Validation is performed on the Cisco Prime Infrastructure as well as on the converged access devices, and covers verification of APs, clients, and applications.
... View more
Table of Content Overview Topology Base Layer 2/3 Configuration Deploying Converged Access Mobility Security WLAN Guest Solution Advanced IOS Wireless Services Wireless Best Practices Summary Overview The small-size remote branch office or retail store may consist of a single or a stack of Ethernet switches to provide network connectivity to the wired and wireless users. Such small networks can converge the Ethernet switching with next-generation wireless capability on the same Catalyst switch. For such network designs, the switch can integrate WLC Mobility Controller (MC) and Mobility Agent (MA) functions without requiring any additional Converged Access elements, such as Switch-Peer-Group (SPG) in the network. These networks may need Guest wireless services, as well as common security and network access policy enforcement across all branch offices. Below is a typical topology of a single switch branch network and sample configuration which has been tried and tested at various customer deployment. Topology Below figure shows a reference topology for a typical branch network Entire Configuration https://www.evernote.com/l/AfQ8unfLtS1EH7_bJfyQTG3Yf8b8XakPruI
... View more
Mobility Controller Managing Mobility Agent - 3850: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/3e/mobility/configuration_guide/b_mobility_3e_3850_cg/configuring_mobility.html#d25e1720a1635 Mobility Controller Managing Mobility Agent - 3650: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3650/software/release/3e/mobility/configuration_guide/b_mobility_3e_3650_cg/configuring_mobility.html#d25e1399a1635 Mobility Controller Managing Mobility - 4500 Sup 8-E: http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst4500/XE3-7-0E/wireless/configuration-guide/b_37e_4500sup8e_cg/configuring_mobility.html#d219447e1408a1635 These docs are listed on the configuration guide listing pages of all the 3 platforms: Cisco Catalyst 3850 Series Switches -Configuration Guides - 3850 , Cisco Catalyst 3650 Series Switches - Configuration Guides 3650 , and Cisco Catalyst 4500 Series Switches - Configuration Guides - 4500 ———————————— The list of synced commands is published " MC Managing MA - List of Commands Synchronized Between MC and MA" : http://www.cisco.com/c/en/us/td/docs/wireless/controller/mc-ma/mc-ma-sync.html
... View more
Introduction This Document is written by "Viten Patel", is working as Technical Marketing Engineer (Converged Access) at Cisco. He was working as Wireless Escalation Engineer - Cisco TAC and holds many wireless certification like CCIE Wireless, CCNP Wireless, CCNA Wireless, CWNA, CWSP, CWAP, CWNE#146. Below checks are a good starting point for troubleshooting issues with IOS-XE based controllers (5760, 3850, 3650). Its a good practice to keep an eye on some of these stats even during normal operation of the system. It will be handy to have the below info handy before opening TAC case. CPU Utilization Monitor the cpu utilization history and make sure that there are no CPU spikes for longer period (more than couple of minutes). The hourly average CPU utilization of any core for any given minute going higher than ~70% needs further close monitoring. Identify the process that is taking most of the CPU and see the spike can be attributed to any kind of network activity. NOTE: If the controller is added to the Prime Infrastructure(PI), then there could be periodic spikes of some of the processes listed below: snmp_subagent, eicored, eicore_ipc(wcm) and SNMP ENGINE (iosd) The SNMP ENGINE utilization should drop down if there is any other higher priority activity that requires the CPU. Commands show processes cpu sorted detailed | exclude 0.00 # Sorted list of processes to the thread level based on the CPU utilization. show processes cpu history Sample output with PI activity: #sh proc cpu sort det | e 0.00
Core 0: CPU utilization for five seconds: 45%; one minute: 16%; five minutes: 13%
Core 1: CPU utilization for five seconds: 44%; one minute: 14%; five minutes: 10%
Core 2: CPU utilization for five seconds: 52%; one minute: 15%; five minutes: 11%
Core 3: CPU utilization for five seconds: 44%; one minute: 15%; five minutes: 11%
PID T C TID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
(%) (%) (%)
5805 L 2621250 1821404 85 17.69 3.14 1.03 0 snmp_subagent
5805 L 2 5805 1074 1478 0 15.99 1.47 0.39 0 snmp_subagent
5805 L 0 6626 1389355 7685676 0 8.42 7.46 0.41 0 snmp_subagent
5805 L 2 6586 762745 9850229 0 3.94 2.63 0.18 0 CMI default xdm
5805 L 1 13461 874 33226 0 2.56 1.84 1.79 0 SNMP SA CACHE
5805 L 2 6497 276868 724659 0 0.13 0.09 0.09 0 snmp_subagent
8127 L 3936457 4684750 100 16.91 5.53 4.32 0 iosd
8127 L 2 8127 600155 2944832 0 13.54 3.74 2.66 0 iosd
8127 L 3 8742 2624104 1473118 0 2.47 1.69 1.62 0 iosd.fastpath
8127 L 2 8743 707123 2610018 0 0.90 0.11 0.03 0 CMI Thread
281 I 89336 1250203 0 24.44 4.11 1.11 0 SNMP ENGINE
276 I 859231 18049 0 11.66 0.99 0.22 0 SNMP Timers
22 I 1967404 1384683 0 8.55 1.66 0.44 0 CMI IOSd task
59 I 1908371 5190050 0 3.00 2.99 2.88 0 ARP Snoop
312 I 1547565 190253 0 2.77 1.33 1.33 0 QoS stats process
274 I 671863 5192881 0 0.88 0.88 0.88 0 DAI Packet Process
30 I 2510652 2756779 0 0.66 0.55 0.44 0 ARP Input
200 I 911297 1344167 0 0.66 0.22 0.11 0 Tunnel IOSd shim DB
195 I 1717254 4141977 0 0.33 0.22 0.22 0 IP Host Track Proce
232 I 1066616 5559004 0 0.33 0.22 0.22 0 UDLD
223 I 1590722 5162371 0 0.33 0.33 0.33 0 IP Input
219 I 977672 3720297 0 0.11 0.11 0.11 0 DHCP Snooping
8123 L 3603041 7383618 63 4.31 2.04 2.04 0 wcm
8123 L 1 8808 2457789 5526106 0 2.34 4.40 0.40 0 eicore_ipc #show processes cpu history
History information for system:
11111 11111 11111 11111 11111 1111
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
70 * * *
60 * ** #
50 ** * ** ** #*
40 #* *##* ## *##* *#*
30 #* ###* ## *##* *#*
20 #* * #### ## #### *##
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
80 * * *
70 **************** * *
60 ******************************** * * *** * ** * * ** ********
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU% Memory Utilization Monitor the RSS column of the individual processes. If this component is monotonically growing then it is alarming and further investigation needed based on the process that is growing. Monitor the free memory component of the system level memory. Don’t be alarmed if it drops down suddenly as IOSd usage could spike up and the memory freed will be held back in IOSd itself. If the system level free memory goes down below 40% it is alarming and further investigation needed. If the IOSd is process is growing, the run IOSd process specific command and look for Processor 8E2F3008 1610612736 344548600 1266064136 1114693628 1102149236 line in the output. Make sure the Lowest and Largest Columns are not too much apart. Commands show processes memory sorted show memory detailed process iosd allocating-process totals # If the IOSd process RSS is growing. Sample output #show processes memory sorted
System memory : 7995664K total, 2601088K used, 5394576K free, 2227952K kernel reserved
Lowest(b) : 568393300
PID Text Data Stack Heap RSS Total Process
8123 20048 1140084 88 34448 748636 1312796 wcm
8127 63252 1601300 104 932 740208 1924924 iosd
5240 9272 422520 92 115968 486208 783940 fed
5782 4260 157088 88 81200 139288 256792 ffm
5242 612 93880 88 1184 117536 306532 stack-mgr
5805 760 219200 88 22908 102992 283952 snmp_subagent #show memory detailed process iosd allocating-process totals
System memory : 7995664K total, 2601040K used, 5394624K free, 2227952K kernel reserved
Lowest(b) : 568393300
Process iosd, type L, PID = 8127
1924924K total, 63252K text, 1601300K data, 104K stack, 932K heap
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 8E2F3008 1610612736 344548600 1266064136 1114693628 1102149236
IOS Proce 8D2F2008 16777216 7691620 9085596 8954256 8466680
Allocator PC Summary for: Processor
PC Total Count Name
0xF4CFA0B7 43112624 19401 *Packet Data*
0xF4CFA036 23628624 19560 *Packet Header*
0xF5936761 20985680 320 AAA AttrL Sub
0xF5A01BE7 12586816 168267 Init AP and Client stats Monitor the number of APs, clients and make sure there is no sudden drop in the count at any point. Monitor the APs up time to make sure there are no AP drop offs. Monitor the client stats for learn-ip state and clients with self-assigned IP addresses. Monitor the AP load and channel utilization of all the APs. Commands show ap uptime # Look at AP uptime and Association uptime columns and make sure those are not too much apart, otherwise there are AP flappings show wcdb database all | include Num # Client count show wcdb database all | count LEARN|169.254. # Clients with self assigned IP, there may be a problem with the DHCP show ap dot11 24ghz load-info # Make sure the channel utilization is not high show ap dot11 5ghz load-info show wireless exclusion # Make sure the clients are not listed if disabled, Make sure the clients are not stuck if enabled with 0 timer value. show wireless client summary | i Idle # Make sure the clients are not stuck in this state. Sample output #show ap uptime
Number of APs : 498
AP Name Ethernet MAC AP Up Time Association Up Time
tsim-01-1 00aa.0100.02f0 1 day 8 hours 42 minutes 15 seconds 1 day 8 hours 38 minutes 57 seconds
tsim-01-2 00aa.0100.03f0 1 day 8 hours 42 minutes 15 seconds 1 day 8 hours 38 minutes 57 seconds
tsim-01-3 00aa.0100.04f0 1 day 8 hours 42 minutes 14 seconds 1 day 8 hours 38 minutes 56 seconds
#show wcdb database all | include Number
Total Number of Wireless Clients = 3486
#show ap dot11 24ghz load-info
AP Name Radio MAC Slot Channel Utilization (%) Clients
ap-01-1 00aa.0100.0200 0 40 10
ap-01-2 00aa.0100.0300 0 30 5
#show wireless exclusionlist
Manually Excluded Clients
Dynamically Excluded Clients Monitor System traffic / logs Look for % in the logs to get all the syslog messages. Look for the keywords like Trace, CAPWAP, AVL in the system logs Monitor the traffic handled by the CPU and watch for any buffer exhaustion/suspension happening frequently. Commands show logging show platform punt client # Look for failures column for any packets punted to CPU, should be zero. It should not increase fast if it is non-zero show controllers ethernet-controller port-asic statistics exceptions # Look for any drop counter increments Sample output show logging | include Trace|%|AVL|CAPWAP
*May 8 11:50:53.940 PDT: %LINEPROTO-5-UPDOWN: Line protocol on Interface Capwap3, changed state to up
-Traceback= 1#b1a3e3fd1d3138ac635fe72eb77c44c3 :F39AF000+20DA8A3 :F39AF000+1380F83 :F39AF000+1382D17 :F39AF000+138C274
#show platform punt client
tag buffer jumbo fallback packets received failures
alloc free bytes conv buf
27 0/1024/2048 0/5 0/5 0 0 0 0 0
65536 0/1024/1600 0/0 0/512 2687494 2687494 171999616 0 0
65537 0/ 512/1600 0/0 0/512 174604 174604 34383056 0 0
65538 0/ 5/5 0/0 0/5 0 0 0 0 0
65539 0/2048/1600 0/16 0/512 60657819 60657819 2060395227 0 0
65540 0/ 128/1600 0/8 0/0 0 0 0 0 0
65541 0/ 128/1600 0/16 0/32 1490044 1490729 340860488 0 0
65542 0/ 768/1600 0/4 0/0 5623288 5646925 337397580 0 0
65544 0/ 96/1600 0/4 0/0 0 0 0 0 0
65545 0/ 96/1600 0/8 0/32 0 0 0 0 0
65546 0/ 512/1600 0/32 0/512 2482982 2914147 240875710 0 0
65547 0/ 96/1600 0/8 0/32 0 0 0 0 0
65548 0/ 512/1600 0/32 0/256 0 0 0 0 0
65551 0/ 512/1600 0/0 0/256 0 0 0 0 0
65556 0/ 16/1600 0/4 0/0 0 0 0 0 0
65557 0/ 16/1600 0/4 0/0 0 0 0 0 0
65558 0/ 16/1600 0/4 0/0 0 0 0 0 0
65559 0/ 16/1600 0/4 0/0 0 0 0 0 0
65560 0/ 16/1600 0/4 0/0 0 0 0 0 0
65561 0/ 512/1600 0/0 0/256 100170460 129432612 1711903376 0 2
65562 0/ 512/1600 0/0 0/256 3740090 9643238 1266448617 0 0
65563 0/ 512/1600 0/0 0/256 0 0 0 0 0
65565 0/ 512/1600 0/16 0/256 0 0 0 0 0
65566 0/ 512/1600 0/16 0/256 0 0 0 0 0
65567 0/ 512/1600 0/16 0/256 0 0 0 0 0
65568 0/ 512/1600 0/16 0/256 0 0 0 0 0
65583 0/ 1/1 0/0 0/0 0 0 0 0 0
131071 0/ 96/1600 0/4 0/0 0 0 0 0 0
fallback pool: 0/1500/1600
jumbo pool: 0/128/9300 More Information Release Notes for Cisco 5760 Wireless LAN Controller, Cisco IOS XE Release 3.2.xSE Release Notes for Catalyst 3850 Series Switch, Cisco IOS XE Release 3.3.xSE Interface and Hardware Component Command Reference, Cisco IOS XE Release 3SE (Catalyst 3850 Switches) - Interface and Hardware Commands
... View more
Hi Rasika, can you share more about your deployment? I believe you are already in talks my team to discuss this. 3850 / 4500 sup8 both are part of our converged access deployment and there will not be any change in terms of architecture between wave 1 and wave 2 from a converged access deployment standpoint. With Converged access you can apply common (wired and wireless traffic) qos policies, have application visibility etc at the access layer itself which is a benefit. we can discuss more about your deployment if you'd like. - Viten
... View more
Hi Amir, sorry i missed that update. I looked at the debugs and i dont see anything wrong. just that the client is doing a new 'association' on each roam which would explain 2-3 seconds of ping drop as the client would need to do full auth again. what kind of client is this? seems like older client. it is not doing fast roaming. also please turn off WPA-TKIP. just let WPA2-AES on the WLAN setting
... View more
sounds like network issues. if you want to go deep. take a span capture of the AP port and see where the drop is happening or at least first confirm the AP has communication issues with the WLC
... View more