Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Community Member

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Hi,

I've a client with UCCX HA setup (System version: 8.0.2.10000-41).

I saw this alert generated yesterday in the RTMT under Alert Central. It came from Secondary UCCX server (It is running as Master). The client has some weird issues since yesterday, the agents started getting Licensing Error, CSD is not updating Agents statistics, Real time data in RT tables (for wallboard) is not getting updated etc. I bounced the LRM service and agents were able to login fine.

I looked at Bug: CSCtq91894 but not very helpful. Does anyone know what could have caused this alert and any possible fixes? I am attaching the MIVR logs covering this event.

Module Name : HISTORICAL_REPORTING_MANAGER

Module Failure Name : HISTORICAL_DATABASE

UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3

Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat

AppID : Cisco Unified CCX Engine

ClusterID :

NodeID : CELCC6802

TimeStamp : Thu Jan 05 09:50:11 AST 2012.

The alarm is generated on Thu Jan 05 08:50:11 EST 2012.Module Name : HISTORICAL_REPORTING_MANAGER
Module Failure Name : HISTORICAL_DATABASE
UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3
Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat
AppID : Cisco Unified CCX Engine
ClusterID :
NodeID : CELCC6802
TimeStamp : Thu Jan 05 09:50:11 AST 2012.
The alarm is generated on Thu Jan 05 08:50:11 EST 2012.

Thanks,

Kapil

14 REPLIES

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

On UCCX version 8.0.2 if you does not have SU4, its most recommended to have it. It has lots of bug fixes including the one you are likely to hit. http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtl93897

SU4 can be downloaded from here http://www.cisco.com/cisco/software/release.html?mdfid=270569179&flowid=5217&softwareid=280840578&release=8.0(2)_SU4&rellifecycle=&relind=AVAILABLE&reltype=latest

If your setup is already in production, I'd recommend to have Cisco TAC confirm it for you.

Pls rate the post if it helps.


GP.

Community Member

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

We seem to be receiving these same alerts on UCCX 8.5(1)SU2 with COP file on HA.

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Hi,

After searching for the error logs cam across a TAC case reference:617010385

Which is pointing to below 2 defects which have a fix for this issue.

---------------------------------------------------------

CSCtj88620: run time error while generating agent related historical reports

Description: HRC throws run time error while generating some agent related reports. HRC log shows the error "A subquery has returned not exactly one row".

Following is seen in the RTMT for the same time frame.

Module Name : HISTORICAL_REPORTING_MANAGER

Module Failure Name : HISTORICAL_DATABASE

UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3

Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 300 Please check d  AppID : Cisco Unified CCX Engine

ClusterID :   NodeID : uccx1a

TimeStamp : Wed Nov 03 08:02:16 EDT 2010.

The alarm is generated on Wed Nov 03 08:02:16 EDT 2010.

To Be fixed: 8.0(2)SU2 8.5(1)SU3

-------------------------------------------------------------

CSCtl93897: UCCX Engine tries to update missing table, which causes RTMT alerts

Description:

When running a historical report, the customer's system generates RTMT alerts such as:

Subject: [RTMT-ALERT-StandAloneCluster] HistoricalDataWrittenToFiles

Module Name : HISTORICAL_REPORTING_MANAGER Module Failure Name : HISTORICAL_DATABASE UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3

Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat AppID : Cisco Unified CCX Engine ClusterID : 

After further investigations, the cause of this issue was found to be the UCCX engine, which tries to write data to a table from a previous version, called the WorkflowTask table. Given that there is no longer a table within the database by this name, this causes the system to generate RTMT alerts.

While there is no direct impact on the system in a critical way, such as core functionality or critical data loss, this still needs to be addressed, as it causes concerns for the customer who may see these errors.

To Be fixed: 8.5(1)SU3 8.0(2)SU4

--------------------------------------------------------------

As I see you are currently running on UCCX 8.5(1)SU2 with COP file on HA, need to come to 8.5(1)SU3.

As of now I can only see 8.5 SU2 posted in CCO.

http://www.cisco.com/cisco/software/release.html?mdfid=270569179&flowid=5217&softwareid=280840578&release=8.0%282%29_SU4&relind=AVAILABLE&rellifecycle=&reltype=latest

Please contact Cisco TAC for the ES or any other available COP file for this issue.

Hope it helps.

Anand

Please rate helpful posts by clicking on the stars below the right answers !!

Community Member

Re: UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

You saved me from raising a TAC case

Thank you very much Anand.

-Yavuz

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Hi Yavuz,

Glad to know that :-).

Please dont forget to rate the helpful posts by clicking on the stars below the right answer.

Thanks,

Anand

Community Member

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Anand, Do you have an exact date on when UCCX 8.5(1)SU3 be release on CCO downloads page? Thanks, Yavuz

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Hi Yavuz,

The expected release date for UCCX 8.5(1)SU3 is April 18th, unless there are no major issues find during regression testing which might be in progress.

I hope there are no delays in this.

Hope it helps.

Anand

Please rate helpful posts !!

Community Member

Re: UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Fantastic news!

Would you be able to obtain the SU3 release notes from CCBU?

It will be nice to find out how many caveats SU3

UCCX %MIVR-HR_MGR-3-HISTORICAL_DB_WRITE_FAILED

Hi Yavuz,

Releaae notes will come along with the SU, as there will be changes till the last moment in it, and we will not get a copy of it till it is released.
Wait for few more days and you will get it in CCO.

Hope it helps.

Anand

Please rate helpful posts !!

Community Member

Hi Gajanan,I'm getting the

Hi Gajanan,

I'm getting the same alert for UCCX 10.5.1.

Do you know if same bug (CSCtj88620) is affecting 10.5.1 as well or if there is another bug related to that ?

Also getting "CCXToCUICCVDSyncFailed" alert, do you know if this is related to a bug as well ? 

=

HistoricalDataWrittenToFiles
Module Name : HISTORICAL_REPORTING_MANAGER
Module Failure Name : HISTORICAL_DATABASE
UNKNOWN_PARAMTYPE:Module Run-time Failure Cause : 3
Module Failure Message : Historical database queue is full. Historical data saved in files. Queue size= 0 Total number of lost records= 1 Please check dat
AppID : Cisco Unified CCX Engine
ClusterID :

==

 

CCXToCUICCVDSyncFailed
RTR : User with userID:mpoulter,sv025,sv021,sv013,sv024,sv029,sv022,ta031,sv032,sv020,sv011,sv018,sv030,sv003,sv031,ct14,sv036,rn064,sv
AppID : Cisco Unified CCX Cluster View Daemon

 

sharri

Cisco Employee

Hi Saima, In your other below

Hi Saima,

 

In your other below post, wherein I replied few minutes back you did not mention about the CCXToCUICCVDSyncFailed alert hence that response is categorically for "Historical database queue is full" issue. :

https://supportforums.cisco.com/discussion/11454016/uchrmgr-2-uccxhistoricaldatawrittentofiles

Now coming to this CCXToCUICCVDSyncFailed alert in conjunction with Historical database queue is full, you are definitely hitting the below defect wherein the alerts will immediately generate after the schedule purge is done. You can verify the same by going to Cisco Unified CCX Administration >> Tools >> Historical Reporting >> Purge Schedule Configuration and check the time for "Daily Purge At". Then match this time with the time of alerts and you will find it the same confirming the defect:

https://tools.cisco.com/bugsearch/bug/CSCur00452/?reffering_site=dumpcr

 

This defect is fixed in 10.6(1) train and since 10.6(1) SU1 is also out, it will be best to upgrade to that option in order to resolve the issue.

 

Little background on this defect CSCur00452:


In a HA environment, the sync request should only come from publisher UCCX but because of the issue in code, the sync request comes from the secondary node as well and since there is no mechanism in the code to entertain that, false alerts are generated.Starting version 10.6(1)  onwards, there had been a fix implemented in the code itself wherein the sync request does not come from secondary node avoiding these false alerts to generate.

 

Regards

Deepak

Community Member

Thanks Deepak for the

Thanks Deepak for the detailed information.

I will try those and let you know the outcome

 

Sharri 

Community Member

Hi Deepak,So that means if we

Hi Deepak,

So that means if we are getting both alerts 'CCXToCUICCVDSyncFailed' & 'Historical database queue is full'

at the same time as 'Purging Schedule Daily' then we are hitting the defect below:

CSCur00452

 

Sharri

 

 

Cisco Employee

Hi Saima,Yes that is correct.

Hi Saima,

Yes that is correct.

 

Regards

Deepak

3606
Views
5
Helpful
14
Replies
CreatePlease to create content