Re: Backup process failing on LMS 2.2 (Solaris 8)

jordi.soler · ‎11-14-2006

Hi all,

I am having problems with one of my CW servers.

This box is a Solaris 8, running LMS 2.2 / RWAN 1.3

When I run a manual or automatic database backup from the GUI, it stays there forever. It does create a folder, and subfolders (cmf, dfm, and rme), but only stores the filebackup.tar, nothing else. While old successful backups, have more folders/files within this 3 main folders.

From the log file (/var/adm/CSCOpx/log/dbbackup.log) I get:

************************************************************

ERROR(207): /opt/BACKUP_CW/backup.LOCK file exists. Most probably another backup process is running

Backup to '/opt/BACKUP_CW' started at: 2006/10/11 01:00:03

ERROR(207): /opt/BACKUP_CW/backup.LOCK file exists. Most probably another backup process is running

Backup failed: 2006/10/11 01:00:03

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

avail=23594352

Archiving the contents of the following directories into /opt/BACKUP_CW/628/cmf/filebackup.tar

/opt/CSCOpx/campus/etc/cwsi/ANIServer.properties

/opt/CSCOpx/campus/etc/cwsi/anisnmp.conf

/opt/CSCOpx/campus/etc/users

/opt/CSCOpx/campus/etc/cwsi/archives

/opt/CSCOpx/campus/etc/cwsi/users

/opt/CSCOpx/lib/classpath/com/cisco/nm/cmf/servlet/cwpass

/opt/CSCOpx/lib/eds/filter/namedfilter.str

/opt/CSCOpx/objects/web/conf/ssl.conf

/opt/CSCOpx/www/classpath/com/cisco/nm/cmf/security/jaas/JaasConfig.xml

/opt/CSCOpx/htdocs/XML/Public/maintree

/opt/CSCOpx/htdocs/help/extern

/opt/CSCOpx/htdocs/help/mappingfiles/CRMExtern.hlp

/opt/CSCOpx/conf/logstat.conf

tar: /opt/CSCOpx/lib/eds/filter/namedfilter.str: Permission denied

ERROR(438): Cannot create or back up tar file. Backup cancelled. Verify that you have write-access to this directory.

Backup failed: 2006/10/11 01:00:05

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Removing directory /opt/BACKUP_CW/628

I have checked, and the file namedfilter.str and parent folders have the same owner and permissions as other CW boxes I can check. Also, the backup folder has the same owner / permissions as well.

I need to upgrade to LMS 2.5.1, but can not afford to do so without a proper backup.

Any help much appreciatted.

Joe Clarke · ‎11-14-2006

What are the permissions on /opt/CSCOpx/lib/eds/filter/namedfilter.str? Since you're running the backup through the web, it will execute as casuser. The perms on the above mentioned file should be casuser:casusers 0640. The perms on the parent directory should be casuser:casusers 0750.

Something else you might try is to run a local backup as root from the CLI:

/opt/CSCOpx/bin/perl /opt/CSCOpx/bin/backup.pl BACKUP_DIR LOG GEN

For example:

/opt/CSCOpx/bin/perl /opt/CSCOpx/bin/backup.pl /backup/cw /var/adm/CSCOpx/log/backup.log 0

jordi.soler · ‎11-15-2006

Hi jclarke,

Thanks for your reply.

Permissions, owner and group for the file and parent folder are ok.

The CLI backup stays there doing nothing.

# opt/CSCOpx/bin/perl /opt/CSCOpx/bin/backup.pl /vola /var/log/backupvola.log 0

# tail -f /var/log/backupvola.log

************************************************************

Backup to '/vola' started at: 2006/11/15 10:25:38

avail=22157173

Archiving the contents of the following directories into /vola/0/cmf/filebackup.tar

/opt/CSCOpx/campus/etc/cwsi/ANIServer.properties

/opt/CSCOpx/campus/etc/cwsi/anisnmp.conf

/opt/CSCOpx/campus/etc/users

/opt/CSCOpx/campus/etc/cwsi/archives

/opt/CSCOpx/campus/etc/cwsi/users

/opt/CSCOpx/lib/classpath/com/cisco/nm/cmf/servlet/cwpass

/opt/CSCOpx/lib/eds/filter/namedfilter.str

/opt/CSCOpx/objects/web/conf/ssl.conf

/opt/CSCOpx/www/classpath/com/cisco/nm/cmf/security/jaas/JaasConfig.xml

/opt/CSCOpx/htdocs/XML/Public/maintree

/opt/CSCOpx/htdocs/help/extern

/opt/CSCOpx/htdocs/help/mappingfiles/CRMExtern.hlp

/opt/CSCOpx/conf/logstat.conf

Archiving the contents of the following directories into /vola/0/rme/filebackup.tar

/var/adm/CSCOpx/files/sw_images

/var/adm/CSCOpx/files/schedule

/var/adm/CSCOpx/files/reports

/opt/CSCOpx/objects/swim/data

/var/adm/CSCOpx/files/archive/config

/var/adm/CSCOpx/files/archive/shadow

/opt/CSCOpx/objects/cmf/data

/opt/CSCOpx/www/classpath/com/cisco/nm/config/archive/config.properties

/opt/CSCOpx/cgi-bin/cas/.casExceptionList

/var/adm/CSCOpx/files/jobs/config

/opt/CSCOpx/objects/aclm/data

/var/adm/CSCOpx/files/netconfig

/opt/CSCOpx/htdocs/mapping.properties

Archiving the contents of the following directories into /vola/0/dfm/filebackup.tar

/opt/CSCOpx/objects/smarts/conf

/opt/CSCOpx/objects/smarts/consoles

/opt/CSCOpx/objects/smarts/repos

/opt/CSCOpx/objects/smarts/logs

Output from the web backup:

************************************************************

Backup to '/opt/BACKUP_CW' started at: 2006/11/15 01:00:02

avail=22348878

ERROR(207): /opt/BACKUP_CW/backup.LOCK file exists. Most probably another backup process is running

Backup failed: 2006/11/15 01:00:02

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

ERROR(207): /opt/BACKUP_CW/backup.LOCK file exists. Most probably another backup process is running

Backup failed: 2006/11/15 01:00:02

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Archiving the contents of the following directories into /opt/BACKUP_CW/628/cmf/filebackup.tar

/opt/CSCOpx/campus/etc/cwsi/ANIServer.properties

/opt/CSCOpx/campus/etc/cwsi/anisnmp.conf

/opt/CSCOpx/campus/etc/users

/opt/CSCOpx/campus/etc/cwsi/archives

/opt/CSCOpx/campus/etc/cwsi/users

/opt/CSCOpx/lib/classpath/com/cisco/nm/cmf/servlet/cwpass

/opt/CSCOpx/lib/eds/filter/namedfilter.str

/opt/CSCOpx/objects/web/conf/ssl.conf

/opt/CSCOpx/www/classpath/com/cisco/nm/cmf/security/jaas/JaasConfig.xml

/opt/CSCOpx/htdocs/XML/Public/maintree

/opt/CSCOpx/htdocs/help/extern

/opt/CSCOpx/htdocs/help/mappingfiles/CRMExtern.hlp

/opt/CSCOpx/conf/logstat.conf

Archiving the contents of the following directories into /opt/BACKUP_CW/628/rme/filebackup.tar

/var/adm/CSCOpx/files/sw_images

/var/adm/CSCOpx/files/schedule

/var/adm/CSCOpx/files/reports

/opt/CSCOpx/objects/swim/data

/var/adm/CSCOpx/files/archive/config

/var/adm/CSCOpx/files/archive/shadow

/opt/CSCOpx/objects/cmf/data

/opt/CSCOpx/www/classpath/com/cisco/nm/config/archive/config.properties

/opt/CSCOpx/cgi-bin/cas/.casExceptionList

/var/adm/CSCOpx/files/jobs/config

/opt/CSCOpx/objects/aclm/data

/var/adm/CSCOpx/files/netconfig

/opt/CSCOpx/htdocs/mapping.properties

Archiving the contents of the following directories into /opt/BACKUP_CW/628/dfm/filebackup.tar

/opt/CSCOpx/objects/smarts/conf

/opt/CSCOpx/objects/smarts/consoles

/opt/CSCOpx/objects/smarts/repos

/opt/CSCOpx/objects/smarts/logs

As you can see, both get stuck at the same point.

Thanks.

Joe Clarke · ‎11-15-2006

What process is running most on the CPU? How long have you let this backup go?

jordi.soler · ‎11-17-2006

Hi jclarke,

I have left the CLI-started backup running for about 2 hours. The automatic runs all day long, but ends at the same point.

jordi.soler · ‎11-28-2006

Found the answer.

The crontable for user casuser had three instances of the backup process all at the same time. I deleted them using the CW GUI, and the manual CLI backup process worked again. In the process we also rebooted the server for another reason, so it could be the combination of both.

Hope it helps to someone else.