CUCM 9.1 Replication error

[GlKlondike]

Hello, we have cucm 9.1.2.11900-12 ( 1 pub and 1 sub)

Some time ago replication status on sub turn to 3.

We did stop replication, drop admindb, and reset all on pub.

After this we have

DB and Replication Services: ALL RUNNING

DB CLI Status: No other dbreplication CLI is running...

Cluster Replication State: BROADCAST SYNC Completed on 1 servers at: 2017-06-17-12-05
Last Sync Result: SYNC COMPLETED 603 tables sync'ed out of 603
Sync Errors: 3 Errors or failures were found
Use this command to see the output- file view activelog
/cm/trace/dbl/2017_06_17_12_01_51_dbl_repl_cdr_Broadcast.log

DB Version: ccm9_1_2_11900_12
Repltimeout set to: 300s
PROCESS option set to: 1

Cluster Detailed View from callmanager1 (2 Servers):

PING CDR Server REPL. DBver& REPL. REPLICATION SETUP
SERVER-NAME IP ADDRESS (msec) RPC? (ID) & STATUS QUEUE TABLES LOOP? (RTMT) & details
----------- ------------ ------ ---- -------------- ----- ------- ----- -----------------
callmanager1 10.0.3.11 0.042 Yes (2) Connected 0 match No (2) PUB Setup
Completed
callmanager2 10.0.4.11 5.421 Yes (4) Connected 0 match N/A (4) Setup Completed

In log file find error:

Table scan for
ccmdbtemplate_g_callmanager1_ccm9_1_2_11900_12_1_452_credentialdynamic start --------

Node Rows Extra Missing Mismatch Processed
---------------- --------- --------- --------- --------- ---------
g_callmanager1_ccm9_1_2_11900_12 1123 0 0 0 2
g_callmanager2_ccm9_1_2_11900_12 1123 0 0 2 0

The repair operation completed. Validating the repaired rows ...
Validation failed for the following rows:
data mismatch
key: pkid:eb4f68a2-744a-401b-a41e-5515d68c5733

Table scan for ccmdbtemplate_g_callmanager1_ccm9_1_2_11900_12_1_243_replicationdynamic start --------

Node Rows Extra Missing Mismatch Processed
---------------- --------- --------- --------- --------- ---------
g_callmanager1_ccm9_1_2_11900_12 1 0 0 0 1
g_callmanager2_ccm9_1_2_11900_12 1 0 0 1 0

The repair operation completed. Validating the repaired rows ...
Validation failed for the following rows:
data mismatch
key: pkid:abcdd7a0-4c70-46f6-9e58-d56bf1d409ff

What we can do?

utils dbreplication forcedatasyncsub? clusterreset?

Maybe we can fix this field mismatch in web interface?

Thank.

[Rajan]

Hi,

can you provide the complete output to check. If the status is shown as 4, then the server is not included for replication. You need to first check the following in sub.

utils diagnose test
utils network connectivity

Once you confirm the connectivity is there between pub and sub for replication, you can try repairing all the tables or only the table for which you are seeing errors using "utils dbreplication repair" command.

HTH

Rajan

Pls rate all useful posts

[GlKlondike]

Yeah, all test and network connectivity is good.

After "stop replication, drop admindb, and reset all on pub" subcriber try setup database and show 3 errors, then going to "4" status and inialization "database setup" again.

3 erros:

credentialdynamic table - 2 mistmach

replicationdynamic table - 1 mismatch

You see this in first post.

In this situation then "database setup" not comleted, i can initializaion "utils dbreplication repair" command?

[Rajan]

yes. you can try "utils dbreplication repair" followed by the table name for only those two tables or do a "utils dbreplication repair all"

HTH

Rajan

[GlKlondike]

Thank you for answer

Repair not help, error in same table

Jun 19 2017 17:19:28 ------ Table scan for ccmdbtemplate_g_callmanager1_ccm9_1_2_11900_12_1_452_credentialdynamic start --------

Node Rows Extra Missing Mismatch Processed
---------------- --------- --------- --------- --------- ---------
g_callmanager1_ccm9_1_2_11900_12 1124 0 0 0 1
g_callmanager2_ccm9_1_2_11900_12 1124 0 0 1 0

The repair operation completed. Validating the repaired rows ...
Validation failed for the following rows:
data mismatch

I find antoher error, maybe is connected to main problem

Cisco Presence 9.1 lost axl connection to CUCM publisher and show "axl user" error.

I try to login to "http://pub/axl" user "axluser" - error login/password

Then i login to "http://sub/axl" user "axluser" - is ok, authentication good

Ok, i try to delete "axluser" on pub and see error

"Error occurred. One or more record did not get deleted. Could not delete a row."

After this i add new application users and delete him, all ok, pub and sub see it.

Problem only in "axluser" i suggest this connected with 3 erros in database sync.

And we have corrupted "axluser" on pub and good on sub.

How can fix this? When i look in numbers of error in database is

Node Rows Extra Missing Mismatch Processed
---------------- --------- --------- --------- --------- ---------
g_callmanager1_ccm9_1_2_11900_12 1124 0 0 0 1
g_callmanager2_ccm9_1_2_11900_12 1124 0 0 1 0

It 1 corrupted row of 1124, i think is "axluser" 

What can i do? How fix row in database?