04-28-2008 03:28 AM
Hello,
we are using LMS 2.5, RME 4.0 and the filesystem /var/adm/CSCOpx is running out of space. It is growing when configurstions are successfully collected.
I think the problem might be that the jobs archive update, archive poller, system inventory polling and system inventory collection are failing.
The logfile /var/adm/CSCOpx/log/dmgtd.log shows errors like
Apr 27 21:32:04 [hostname] SQLAnywhere(rmengeng): [ID 702911 local0.notice] Connection terminated abnormally
Apr 27 21:32:04 [hostname] SQLAnywhere(cmfeng): [ID 702911 local0.notice] Connection terminated abnormally
Thank you in advance for your help
04-28-2008 06:18 AM
Believe it or not, "Connection terminated abnormally" is usually benign and probably has nothing to do with the space issue. I bet it's your /var/adm/CSCOpx/files/rme/dcma/difffiles that's filling up the file system. You can verify that with "du -skr /var/adm/CSCOpx/files/rme/dcma/difffiles" and "/var/adm/CSCOpx/files/rme/dcma". If so, this is attributed to CSCse88215, that will hopefully be addressed in RME 4.1 of LMS 3.x. See these two old threads dealing with this subject.
04-28-2008 07:41 AM
you are right there are lots of difffiles.
This will solve the space problem but there is still a problem with the database left.
Thanks a lot
04-28-2008 07:56 AM
What does your /opt/CSCOpx/bin/pdshow output say?
"Connection terminated abnormally" alone doesn't actually indicate you have a database problem. I've got tons of these in my dmgtd.log:
Apr 28 06:25:12 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Connection terminated abnormally
Apr 28 06:25:12 nms SQLAnywhere(rmengeng): [ID 702911 local1.notice] Connection terminated abnormally
04-29-2008 04:21 AM
Please find the pdshow file attached.
What bothers me are the following messages in the /var/adm/CSCOpx/log/dmgtd.log after the start of dmgtd:
Apr 29 13:34:05 hostname dmgt[25600]: [ID 947059 local0.info] #3007:TYPE=INFO:Started application(RmeOrb) "/opt/CSCOpx/lib/vbroker/bin/osa
gent -p 42342" pid=25789.
Apr 29 13:34:07 hostname SQLAnywhere(cmfeng): [ID 702911 local0.notice] Note: Duplicate index "CSI1OgsGroupPropertiesTable" for table "CSO
gsGroupPropertiesTable" in database "cmfDb"
Apr 29 13:34:07 hostname SQLAnywhere(cmfeng): [ID 702911 local0.notice] Note: Duplicate index "mykey" for table "CSUserGroupAssociationTab
le" in database "cmfDb"
Apr 29 13:34:07 hostname SQLAnywhere(cmfeng): [ID 702911 local0.notice] Note: Duplicate index "mykey" for table "CSOgsTagTable" in databas
e "cmfDb"
Apr 29 13:34:08 hostname SQLAnywhere(cmfeng): [ID 702911 local0.notice] Note: Duplicate index "CampusI1OgsGroupPropertiesTable" for table
"CampusOgsGroupPropertiesTable" in database "cmfDb"
Apr 29 13:34:08 hostname SQLAnywhere(cmfeng): [ID 702911 local0.notice] Note: Duplicate index "mykey" for table "CampusUserGroupAssociatio
nTable" in database "cmfDb"
Apr 29 13:34:08 hostname SQLAnywhere(cmfeng): [ID 702911 local0.notice] Note: Duplicate index "mykey" for table "CampusOgsTagTable" in dat
abase "cmfDb"
How can I get additional information about failed jobs (archive poller, archive update)that are scheduled?
04-29-2008 07:19 AM
The pdshow output looks fine to me.
As far as I understand, the "duplicates" notices are harmless too, though I have no idea why they appear at the moments they do:
Apr 11 16:30:34 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "mykey" for table "CampusOgsTagTable" in database "cmfDb"
Apr 11 16:31:33 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "Index1" for table "OgsDcrDeviceTable" in database "cmfDb"
Apr 11 16:31:33 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "mykey" for table "OgsDcrDeviceTable" in database "cmfDb"
Apr 11 16:55:51 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "CampusI1OgsGroupPropertiesTable" for table "CampusOgsGroupPropertiesTable" in database "cmfDb"
Apr 11 16:55:51 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "mykey" for table "CampusUserGroupAssociationTable" in database "cmfDb"
Apr 11 16:55:51 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "mykey" for table "CampusOgsTagTable" in database "cmfDb"
Apr 11 16:56:37 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "Index1" for table "OgsDcrDeviceTable" in database "cmfDb"
Apr 11 16:56:37 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "mykey" for table "OgsDcrDeviceTable" in database "cmfDb"
Apr 27 22:32:52 nms SQLAnywhere(cmfeng): [ID 702911 local1.notice] Note: Duplicate index "mykey" for table "CampusOgsGroupCacheTable" in database "cmfDb"
You can get more info on each failed job by clicking on their respective Job ID in the DCR and/or RME homepage, as well as RME's Job Management tab. If you want to look at them via CLI, I find the convenient approach is to locate the job log using "find /var/adm/CSCOpx/ -name [JobID#].log -print".
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: