04-08-2012 11:25 PM - edited 03-16-2019 10:32 AM
Hi Guys,
We had installed a new CUCM8.5 on ESXI 5 UCS210 C series server. we have only one publisher and backup by SRST on single site on two 3945 router.
ISSUE: Suddenly CUCM service goes down. CUCM IP address was ping able but we were not able to login to cisco unified administration , servicablility webpage showing with error " Database communication error". but we were able to login to cisco unified OS administration page.
We tried to restart the server through cisco unified OS administration page it was not happening & Using Vsphere client CLI console was not responding for CLI shutdown/restart.
We tried to power down the CUCM through VMware it was also not responding.
Finally we power down the server physically and started it again. Now the ESXi got UP and we power ON the CUCM VMware machine through Vsphere client.
Now the CUCM service got UP and all the IP phone from SRST mode got registered with CUCM, calls are going fine and we are now able to login to webpages.
But when we tried to login in to CLI console suddenly continues error was displaying
ERROR : " EXT3-fs error (device sdb1) in start_transaction :journal has adorted "
What is the cause of this issue?
how can we rectify this issue?
Please respond immediately it a production setup.
Thanks in advance.
Vigeesh Kalathil
Solved! Go to Solution.
04-13-2012 09:34 AM
If you are seen ERROR : " EXT3-fs error (device sdb1) in start_transaction :journal has adorted "
Your VM has gone into read only, and most likely is corrupted by now. This usually happens if you loose connection to your SAN, your SAN experienced latency, or you might be running into this known condition.
https://supportforums.cisco.com/docs/DOC-23667
Not much to do here, reimage and restore from the last backup.
04-08-2012 11:37 PM
Vigeesh,
I couldnt find anything w.r.t. error message. For now, as the calling is unaffected, I suggest you let it go & do a gracefull restart to the CUCM application & ESXi ( Power Off the VM ) after office hours to see if you still face the CLI error.
Let us know how it goes.
GP.
Pls rate helpful posts by clicking on stars below the post !!
04-08-2012 11:53 PM
Hi Vigeesh,
There is a bug for similar issue but that is for the MCS servers.
So i suspect that there must be something wrong in the virtual center that the UCS is running. May be you want to open a TAC case so that they can take a look into it with logs.
Hope this helps.
-Abdul
Please rate helpful posts!!!
04-09-2012 12:27 AM
Hi,
CSCsv49493 Bug ; this bug normally seen on HP MCS server's
We are using UCS 210c ,ESXI 5
CUCM system version 8.5.1.10000.26
Vmware installation : 2Vcpu intel (R) Xeon(R) CPU E5640 @ 2.67 GHz, Disk1:80 Gb disk2 :80 Gb 6144 Mb RAM.
we have total 10 harddisk, 2 for ESXI with RAID 1, rest 8 harddisk on RAID 5 as cisco recommended.
OVA template create two 80 Gb virtual hard disk from 8 physical harddisk with RAID 5
How we will do the workaround in VMware Setup.
let me try the graceful restart first then we wil go to next step.
Thanks for all replys
Vigeesh kalathil
04-08-2012 11:55 PM
Okay i will try it.
What you think cause for "Database communication error" when trying to login to webpages.
Is it any database error? or any file corrupted?
We had not made any changes on IP address.
As this error message is continues i can't login to CLI. we can restart only through cisco unified OS administration webpage.
what is your opinion?
04-08-2012 11:58 PM
Hi,
The most common issue with DB communiction error is due to tomcat service. If Tomcat service is gone bad, the DB communication will have issues.
Regards
Ronak patel
Rate helpful posts.
04-09-2012 12:07 AM
Vigeesh,
Usually, " Database Comm error " is shown after the CUCM Hostname or IP is changed ( incorrectly ). If you have not done any of them, then let's wait for Restart to see if the issue persists.
GP.
04-09-2012 12:29 AM
Ya you are right let try the restart first.
04-09-2012 03:30 AM
Hi Guys,
As discussed , we tried to restart/shutdown the CUCM through cisco unified OS administration webpage but it's not getting restarted/shutdown.
And we are not able to login to CLI console, error message is continuously displaying.
I hope its a Bug
Please advise what to do next.
Thanks in advance
Vigeesh kalathil
04-09-2012 04:20 AM
Vigeesh,
This is something i feel serious becos if you forcefully shut the server again, it might result into more complications e.g. file corruption etc. As the server is already in production, my sincere suggestion to you would be to have it examined by Cisco TAC. Most likely they might ask you to upgrade the server to more stable version either by issuing an Engineering Special ( ES ) patch or ask you to upgrade to next main release.
Regards
GP.
04-09-2012 04:23 AM
Thanks for your reply
Vigeesh Kalathil
04-09-2012 04:33 AM
Hi ,
We are having DRS backup of the system , If restoring the system with DRS backup; will it cause any file corruption?
Thanks in advance.
Vigeesh Kalathil
04-09-2012 04:48 AM
Hi
DRS back up will backup DB also , so no point of file corruption.
Regards
Ronak Patel
04-09-2012 07:26 AM
Hello
can you try to restart the DB service manullay from CLI
Utils Service restart "A Cisco DB"
04-13-2012 08:37 AM
I am seeing the exact same issue with identical setup. Were you able to find the culprit?
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: