Next: Replication Up: Internals Previous: Completed Backups Contents
Making a backup with option lateLinks means to split the creation of backups into two parts. The first part is done with storeBackup.pl which identifies and copies all files with new contents, and the second one with storeBackupUpdateBackup.pl, which:
The most important part - regarding storeBackups structure - is the creation of the missing hard links in the backup. This means that the last (unfinished) backup is not a full backup but something like an incremental backup.56
Let's make an example:
$ mkdir /tmp/a $ cd /tmp/a $ mkdir -p s/sub1 b $ cp /bin/ls /bin/pwd s $ cp /bin/ls /bin/pwd s/sub1
If a directory called /tmp/a already exists on you system, select another directory of your choice. We now created a sourceDir s with some contents and a backupDir b.
Now run the first backup with option --lateLinks and -lateCompress and look at the result:
$ storeBackup.pl -s s -b b --lateLinks --lateCompress ..... $ find b -print | sort b b/default b/default/2013.08.10_10.14.00 b/default/2013.08.10_10.14.00/ls b/default/2013.08.10_10.14.00/.md5CheckSums.bz2 b/default/2013.08.10_10.14.00/.md5CheckSums.info b/default/2013.08.10_10.14.00/pwd b/default/2013.08.10_10.14.00/.storeBackupLinks b/default/2013.08.10_10.14.00/.storeBackupLinks/linkFile.bz2
You can see:
This ``link file'' contains, what still has to be done:
$ bzcat b/default/2013.08.10_10.14.00/.storeBackupLinks/linkFile.bz2 # link md5sum # existingFile # newLink # compress md5sum # fileToCompress # dir dirName # symlink file # target # linkSymlink link # existingFile # newLink dir sub1 compress c5f89e40c144b6fb8b61f2ef72e4b556 pwd compress b5607b4dc7d896c0fab5c4a308239161 ls link c5f89e40c144b6fb8b61f2ef72e4b556 ./pwd.bz2 sub1/pwd.bz2 link b5607b4dc7d896c0fab5c4a308239161 ./ls.bz2 sub1/ls.bz2
In the first commented lines the syntax is explained shortly. You can see (all pathes are relative to the actual backup directory):
There are no dependencies to other directories, because this was a first backup.
After running storeBackupUpdateBackup.pl you get:
$ storeBackupUpdateBackup.pl -b b ..... $ find b -ls 232815 0 drwxrwxr-x 3 hjc hjc 60 Aug 10 10:14 b 240109 0 drwx------ 3 hjc hjc 60 Aug 10 12:18 b/default 305119 0 drwxr-xr-x 4 hjc hjc 160 Aug 10 12:15 b/default/2013.08.10_11.52.15 249568 52 -rwxr-xr-x 2 hjc hjc 49915 Aug 10 10:03 b/default/2013.08.10_11.52.15/ls.bz2 249567 16 -rwxr-xr-x 2 hjc hjc 13395 Aug 10 10:03 b/default/2013.08.10_11.52.15/pwd.bz2 323964 0 drwxrwxr-x 2 hjc hjc 80 Aug 10 10:03 b/default/2013.08.10_11.52.15/sub1 249568 52 -rwxr-xr-x 2 hjc hjc 49915 Aug 10 10:03 b/default/2013.08.10_11.52.15/sub1/ls.bz2 249567 16 -rwxr-xr-x 2 hjc hjc 13395 Aug 10 10:03 b/default/2013.08.10_11.52.15/sub1/pwd.bz2 306983 4 -rw-rw-r-- 1 hjc hjc 300 Aug 10 11:52 b/default/2013.08.10_11.52.15/.md5CheckSums.bz2 305122 4 -rw------- 1 hjc hjc 950 Aug 10 11:52 b/default/2013.08.10_11.52.15/.md5CheckSums.info 305120 0 drwx------ 2 hjc hjc 40 Aug 10 12:15 b/default/2013.08.10_11.52.15/.storeBackupLinks
You can see, that ls and pwd have been compressed. There are now also hard links from subdirectory sub1. Finally, the permissions are corrected and .storeBackupLinks/linkFile.bz2 does not exist any more. This is now a completed backup.
Let's run storeBackup.pl a second time:
$ storeBackup.pl -s s -b b --lateLinks --lateCompress ..... $ find b -print | sort b b/default b/default/2013.08.10_10.14.00 b/default/2013.08.10_10.14.00/ls.bz2 b/default/2013.08.10_10.14.00/.md5CheckSums.bz2 b/default/2013.08.10_10.14.00/.md5CheckSums.info b/default/2013.08.10_10.14.00/pwd.bz2 b/default/2013.08.10_10.14.00/.storeBackupLinks b/default/2013.08.10_10.14.00/.storeBackupLinks/linkFrom0 b/default/2013.08.10_10.14.00/sub1 b/default/2013.08.10_10.14.00/sub1/ls.bz2 b/default/2013.08.10_10.14.00/sub1/pwd.bz2 b/default/2013.08.10_11.52.15 b/default/2013.08.10_11.52.15/.md5CheckSums.bz2 b/default/2013.08.10_11.52.15/.md5CheckSums.info b/default/2013.08.10_11.52.15/.storeBackupLinks b/default/2013.08.10_11.52.15/.storeBackupLinks/linkFile.bz2 b/default/2013.08.10_11.52.15/.storeBackupLinks/linkTo
You can see one change in the first backup: There is now a file called .storeBackupLinks/linkFrom0. This file contains:
$ cat b/default/2013.08.10_10.14.00/.storeBackupLinks/linkFrom0 ../2013.08.10_11.52.15
This means that there is at least one unresolved reference from a backup with the relative path from this backup (2013.08.10_11.52.15). If references from multiple other backups would exist, there would be more files (linkFrom1, linkFrom2, ...) of that kind describing those dependencies. Because of one or more of these files, storeBackupDel.pl knows that it must not delete that backup, because otherwise the information where the later backup points to (via yet unresolved links) is lost.
In the new backup (2013.08.10_11.52.15), the file .storeBackupLinks/linkTo lists all other backups where this backup has dependencies to (only one in this case):
$ cat b/default/2013.08.10_11.52.15/.storeBackupLinks/linkTo ../2013.08.10_10.14.00
As you see in the list of all files generated by the second backup, there are no data files in the new backup, only meta data files. That's fine, because no files have changed since the first backup. Let's look into linkFile:
$ bzcat b/default/2013.08.10_11.52.15/.storeBackupLinks/linkFile.bz2 # link md5sum # existingFile # newLink # compress md5sum # fileToCompress # dir dirName # symlink file # target # linkSymlink link # existingFile # newLink link c5f89e40c144b6fb8b61f2ef72e4b556 ../2013.08.10_10.14.00/sub1/pwd.bz2 pwd.bz2 link b5607b4dc7d896c0fab5c4a308239161 ../2013.08.10_10.14.00/sub1/ls.bz2 ls.bz2 dir sub1 link c5f89e40c144b6fb8b61f2ef72e4b556 ../2013.08.10_10.14.00/sub1/pwd.bz2 sub1/pwd.bz2 link b5607b4dc7d896c0fab5c4a308239161 ../2013.08.10_10.14.00/sub1/ls.bz2 sub1/ls.bz2
It is easy to see that all missing files have to be hard linked and the subdirectory has to be created.
Now you see the changes from running storeBackupUpdateBackup.pl:
$ storeBackupUpdateBackup.pl -b b ..... $ find b -ls 232815 0 drwxrwxr-x 3 hjc hjc 60 Aug 10 10:14 b 240109 0 drwx------ 4 hjc hjc 80 Aug 10 11:52 b/default 305119 0 drwxr-xr-x 4 hjc hjc 160 Aug 10 12:15 b/default/2013.08.10_11.52.15 249568 52 -rwxr-xr-x 4 hjc hjc 49915 Aug 10 10:03 b/default/2013.08.10_11.52.15/ls.bz2 249567 16 -rwxr-xr-x 4 hjc hjc 13395 Aug 10 10:03 b/default/2013.08.10_11.52.15/pwd.bz2 323964 0 drwxrwxr-x 2 hjc hjc 80 Aug 10 10:03 b/default/2013.08.10_11.52.15/sub1 249568 52 -rwxr-xr-x 4 hjc hjc 49915 Aug 10 10:03 b/default/2013.08.10_11.52.15/sub1/ls.bz2 249567 16 -rwxr-xr-x 4 hjc hjc 13395 Aug 10 10:03 b/default/2013.08.10_11.52.15/sub1/pwd.bz2 306983 4 -rw-rw-r-- 1 hjc hjc 300 Aug 10 11:52 b/default/2013.08.10_11.52.15/.md5CheckSums.bz2 305122 4 -rw------- 1 hjc hjc 950 Aug 10 11:52 b/default/2013.08.10_11.52.15/.md5CheckSums.info 305120 0 drwx------ 2 hjc hjc 40 Aug 10 12:15 b/default/2013.08.10_11.52.15/.storeBackupLinks 241222 0 drwxr-xr-x 4 hjc hjc 160 Aug 10 10:27 b/default/2013.08.10_10.14.00 249568 52 -rwxr-xr-x 4 hjc hjc 49915 Aug 10 10:03 b/default/2013.08.10_10.14.00/ls.bz2 249567 16 -rwxr-xr-x 4 hjc hjc 13395 Aug 10 10:03 b/default/2013.08.10_10.14.00/pwd.bz2 249566 0 drwxrwxr-x 2 hjc hjc 80 Aug 10 10:03 b/default/2013.08.10_10.14.00/sub1 249568 52 -rwxr-xr-x 4 hjc hjc 49915 Aug 10 10:03 b/default/2013.08.10_10.14.00/sub1/ls.bz2 249567 16 -rwxr-xr-x 4 hjc hjc 13395 Aug 10 10:03 b/default/2013.08.10_10.14.00/sub1/pwd.bz2 238098 4 -rw-rw-r-- 1 hjc hjc 300 Aug 10 10:14 b/default/2013.08.10_10.14.00/.md5CheckSums.bz2 241225 4 -rw------- 1 hjc hjc 950 Aug 10 10:14 b/default/2013.08.10_10.14.00/.md5CheckSums.info 241223 0 drwx------ 2 hjc hjc 40 Aug 10 12:15 b/default/2013.08.10_10.14.00/.storeBackupLinks
As you see, the result is a complete backup.
This all was normal behavior. Nothing went wrong. I create a new backupDir (b1) and let things go wrong:
$ storeBackup.pl -s s -b b1 ..... $ storeBackup.pl -s s -b b1 --lateLinks --lateCompress ..... $ find b1 -print | sort b1 b1/default b1/default/2013.08.10_13.47.41 b1/default/2013.08.10_13.47.41/ls.bz2 b1/default/2013.08.10_13.47.41/.md5CheckSums.bz2 b1/default/2013.08.10_13.47.41/.md5CheckSums.info b1/default/2013.08.10_13.47.41/pwd.bz2 b1/default/2013.08.10_13.47.41/.storeBackupLinks b1/default/2013.08.10_13.47.41/.storeBackupLinks/linkFrom0 b1/default/2013.08.10_13.47.41/sub1 b1/default/2013.08.10_13.47.41/sub1/ls.bz2 b1/default/2013.08.10_13.47.41/sub1/pwd.bz2 b1/default/2013.08.10_13.49.21 b1/default/2013.08.10_13.49.21/.md5CheckSums.bz2 b1/default/2013.08.10_13.49.21/.md5CheckSums.info b1/default/2013.08.10_13.49.21/.storeBackupLinks b1/default/2013.08.10_13.49.21/.storeBackupLinks/linkFile.bz2 b1/default/2013.08.10_13.49.21/.storeBackupLinks/linkTo
You now see one complete backup (the first one) and the second backup which is not completed.
For whatever stupid reason, the second backup is deleted. Maybe you recognized during the (let's assume long) backup that some option where wrong and pressed control-c and deleted the backup. In this example, I now delete the second backup:
$ rm -r b1/default/2013.08.10_13.49.21
But the file
b1/default/2013.08.10_13.47.41/.storeBackupLinks/linkFrom still
exists, so the backup series is not consistent.
Running storeBackupUpdateBackup.pl shows an error message:
$ storeBackupUpdateBackup.pl -b b1
BEGIN 2013.08.10 13:57:46 475 checking references and backup copying in <b1>
VERSION 2013.08.10 13:57:46 475 storeBackupUpdateBackup.pl, 3.4 +
INFO 2013.08.10 13:57:46 475 creating lock file </tmp/storeBackup.lock>
INFO 2013.08.10 13:57:46 475 scanning directory <b1> for existing backups
INFO 2013.08.10 13:57:46 475 scanning directory <b1/default> for existing backups
STATISTIC 2013.08.10 13:57:46 475 found 1 backup series, 1 backups, 0 renamed backups
ERROR 2013.08.10 13:57:46 475 link <../2013.08.10_13.49.21> to non existing dir in
</tmp/a/b1/default/2013.08.10_13.47.41/.storeBackupLinks/linkFrom0>
ERROR 2013.08.10 13:57:46 475 found 1 inconsistencies, please repair and check again
Deleting this file may not be a good idea, because in a real world situation, there might be other references to this backup also which means things may be not that simple and it is easy to make something wrong (or worse).
There are two possibilities available with storeBackupUpdateBackup.pl:
$ storeBackupUpdateBackup.pl -b b1 --autorepair
BEGIN 2013.08.10 17:06:59 4569 checking references and backup copying in <b1>
VERSION 2013.08.10 17:06:59 4569 storeBackupUpdateBackup.pl, 3.4 +
INFO 2013.08.10 17:06:59 4569 removing old lock file of process <475>
INFO 2013.08.10 17:06:59 4569 creating lock file </tmp/storeBackup.lock>
INFO 2013.08.10 17:06:59 4569 scanning directory <b1> for existing backups
INFO 2013.08.10 17:06:59 4569 scanning directory <b1/default> for existing backups
STATISTIC 2013.08.10 17:06:59 4569 found 1 backup series, 1 backups, 0 renamed backups
ERROR 2013.08.10 17:06:59 4569 link <../2013.08.10_13.49.21> to non existing dir in
</tmp/a/b1/default/2013.08.10_13.47.41/.storeBackupLinks/linkFrom0>
INFO 2013.08.10 17:06:59 4569 autorepair: deleted
</tmp/a/b1/default/2013.08.10_13.47.41/.storeBackupLinks/linkFrom0>
ERROR 2013.08.10 17:06:59 4569 ----- repeating consistency check -----
INFO 2013.08.10 17:06:59 4569 scanning directory <b1> for existing backups
INFO 2013.08.10 17:06:59 4569 scanning directory <b1/default> for existing backups
STATISTIC 2013.08.10 17:06:59 4569 found 1 backup series, 1 backups, 0 renamed backups
INFO 2013.08.10 17:06:59 4569 consistency check finished successfully
INFO 2013.08.10 17:06:59 4569 found no references to backups from lateLinks that need
storeBackupUpdateBackup run
INFO 2013.08.10 17:06:59 4569 everything is updated, nothing to do
STATISTIC 2013.08.10 17:06:59 4569 duration = 1s
STATISTIC 2013.08.10 17:06:59 4569 [sec] | user| system
STATISTIC 2013.08.10 17:06:59 4569 -------+----------+----------
STATISTIC 2013.08.10 17:06:59 4569 process| 0.08| 0.00
STATISTIC 2013.08.10 17:06:59 4569 childs | 0.05| 0.00
STATISTIC 2013.08.10 17:06:59 4569 -------+----------+----------
STATISTIC 2013.08.10 17:06:59 4569 sum | 0.13| 0.00 => 0.13
INFO 2013.08.10 17:06:59 4569 syncing ...
END 2013.08.10 17:06:59 4569 checking references and copying in <b1>
$ find b1 -print | sort
b1
b1/default
b1/default/2013.08.10_13.47.41
b1/default/2013.08.10_13.47.41/ls.bz2
b1/default/2013.08.10_13.47.41/.md5CheckSums.bz2
b1/default/2013.08.10_13.47.41/.md5CheckSums.info
b1/default/2013.08.10_13.47.41/pwd.bz2
b1/default/2013.08.10_13.47.41/.storeBackupLinks
b1/default/2013.08.10_13.47.41/sub1
b1/default/2013.08.10_13.47.41/sub1/ls.bz2
b1/default/2013.08.10_13.47.41/sub1/pwd.bz2
The output of the find command above shows, that everything is
fine now - there's one completed backup.
Now let's make a second backup with lateLinks again:
$ storeBackup.pl -s s -b b1 --lateLinks --lateCompress ----- $ find b1 -print | sort b1 b1/default b1/default/2013.08.10_13.47.41 b1/default/2013.08.10_13.47.41/ls.bz2 b1/default/2013.08.10_13.47.41/.md5CheckSums.bz2 b1/default/2013.08.10_13.47.41/.md5CheckSums.info b1/default/2013.08.10_13.47.41/pwd.bz2 b1/default/2013.08.10_13.47.41/.storeBackupLinks b1/default/2013.08.10_13.47.41/.storeBackupLinks/linkFrom0 b1/default/2013.08.10_13.47.41/sub1 b1/default/2013.08.10_13.47.41/sub1/ls.bz2 b1/default/2013.08.10_13.47.41/sub1/pwd.bz2 b1/default/2013.08.10_17.17.53 b1/default/2013.08.10_17.17.53/.md5CheckSums.bz2 b1/default/2013.08.10_17.17.53/.md5CheckSums.info b1/default/2013.08.10_17.17.53/.storeBackupLinks b1/default/2013.08.10_17.17.53/.storeBackupLinks/linkFile.bz2 b1/default/2013.08.10_17.17.53/.storeBackupLinks/linkTo
But this time, I delete the first backup:
$ rm -r b1/default/2013.08.10_13.47.41
This means, I now have an uncomplete backup (created with lateLinks) and pointing to nowhere. I try to use --storeBackupUpdateBackup.pl with option --autorepair:
$ storeBackupUpdateBackup.pl -b b1 --autorepair
BEGIN 2013.08.10 17:21:08 5019 checking references and backup copying in <b1>
VERSION 2013.08.10 17:21:08 5019 storeBackupUpdateBackup.pl, 3.4 +
INFO 2013.08.10 17:21:08 5019 creating lock file </tmp/storeBackup.lock>
INFO 2013.08.10 17:21:08 5019 scanning directory <b1> for existing backups
INFO 2013.08.10 17:21:08 5019 scanning directory <b1/default> for existing backups
STATISTIC 2013.08.10 17:21:08 5019 found 1 backup series, 1 backups, 0 renamed backups
ERROR 2013.08.10 17:21:08 5019 FATAL ERROR: link <../2013.08.10_13.47.41> to non existing dir in
</tmp/a/b1/default/2013.08.10_17.17.53/.storeBackupLinks/linkTo>
ERROR 2013.08.10 17:21:08 5019 found 1 inconsistencies, please repair and check again
Now I get a FATAL ERROR: No chance for
storeBackupUpdateBackup.pl to repair this. The only chance
would be to copy57 the missing (first) backup from somewhere else (e.g. a
copy from the backup or from replication) to the right location and run
storeBackupUpdateBackup.pl with --autorepair again. If
there is no other copy of the missing backup, all backups depending on
that backup (and backups depending on that one etc.) will never be
complete. You should delete them or move those partly backups outside
the backupDir, so they do not harm the workflow of
storeBackup's tools.
Heinz-Josef Claes 2014-04-20