After an upgrade of CS-MARS to the latest version, the device no longer receives logs/incidents

[TCC_2]

Core issue

This issue is due to the presence of Cisco bug ID CSCsh38818.

In this issue, after the Global Controller (GC) and Local Controllers (LC) are upgraded from version 4.2.2 to 4.2.3, the communication states between the GC and LCs can be out of synch. The GC sees the LC as Active, and the LC sees itself as Offline. Toggling Suspend/Resume from the Local Controller Management page of the GC flips both states, so that the GC sees the LC as Suspended but the LC sees itself as Online and pushes information to the GC.

In order to verify that the GC-LC MARS are in this error state, confirm these:

1. The GC and all associated LCs are upgraded from 4.2.2 to 4.2.3.  Refer to these upgrade instructions in the 4.2.3 release notes.
   
   
2. Log into the GC, choose Admin > System Setup > Local Controller Management.
   
   
3. For each LC, choose the LC checkbox and click Details.
   
   
4. Verify that there is a discrepancy between the status on the GC and the status of the LC. Specifically, the status on the GC shows that an LC is Active, while the LC GUI shows that the LC is Offline in the header: CS-MARS Local Controller (Offline). Log into the LC through the GUI interface in order to confirm the LC status.
   
   
5. Make a note of each LC that is in the error state.

Resolution

Once the error is identified, these steps can be completed in order to exit the error state as a workaround:

1. Log into the GC, choose Admin > System Setup > Local Controller Management.
   
   
2. Identify each LC in the error state.  See the Conditions section.
   
   
3. Select each LC in the error state and click Suspend/Resume in order to verify that the GC sees each LC as Suspended but if you click Details for that LC, it shows that the LC is no longer Offline: CS-MARS Local Controller: [hostname]/[zone name]
   
   
4. On the GC Local Controller Management page, choose Refresh Rate "1 minute" from the pulldown menu.
   
   
5. Choose Admin > System Maintenance > License Key and verify that the correct number of LCs (20/50s, and 100/200s) are counted by the GC, under used.
   
   
6. Choose Admin > System Setup > Local Controller Management in the GC browser window.
   
   For each LC that needs to be corrected, complete steps 7-9.
   
   
7. Open a secure shell (SSH) to the LC and run the pnreset -j command, and type Yes in order to confirm.
   
   
8. Within 20 seconds, switch back to the GC browser window and click the browser refresh button every three seconds until the Status message for that LC displays Not responding. This is needed in order to resynchronize communication between the GC and LC.
   
   
9. Wait for the Local Controller Management page of the GC to refresh and verify that the LC status is now Active and the GUI for that LC shows the LC is Active and not Offline. Log into the LC through the GUI interface in order to confirm the LC status.

In order to completely resolve this issue, upgrade CS-MARS to version 4.2(5.2456) through Cisco Downloads.