Next: Components / Programs to Up: How storeBackup works Previous: Performance Contents
Here you can see the statistical output of a big backup I ran on my laptop and saved to an NFS server. (I'm running this backup including OS once or twice a week and a smaller one every day, similar to the description of example 3, section 9.4.) I had to backup more than 500,000 entries:
STATISTIC 2008.09.08 23:40:17 3961 [sec] | user| system STATISTIC 2008.09.08 23:40:17 3961 -------+----------+---------- STATISTIC 2008.09.08 23:40:17 3961 process| 386.30| 166.27 STATISTIC 2008.09.08 23:40:17 3961 childs | 209.02| 116.96 STATISTIC 2008.09.08 23:40:17 3961 -------+----------+---------- STATISTIC 2008.09.08 23:40:17 3961 sum | 595.32| 283.23 => 878.55 (14m39s) STATISTIC 2008.09.08 23:40:17 3961 directories = 43498 STATISTIC 2008.09.08 23:40:17 3961 files = 482516 STATISTIC 2008.09.08 23:40:17 3961 symbolic links = 12024 STATISTIC 2008.09.08 23:40:17 3961 late links = 462267 STATISTIC 2008.09.08 23:40:17 3961 named pipes = 3 STATISTIC 2008.09.08 23:40:17 3961 sockets = 48 STATISTIC 2008.09.08 23:40:17 3961 block devices = 0 STATISTIC 2008.09.08 23:40:17 3961 character devices = 0 STATISTIC 2008.09.08 23:40:17 3961 new internal linked files = 178 STATISTIC 2008.09.08 23:40:17 3961 old linked files = 462089 STATISTIC 2008.09.08 23:40:17 3961 unchanged files = 0 STATISTIC 2008.09.08 23:40:17 3961 copied files = 2896 STATISTIC 2008.09.08 23:40:17 3961 compressed files = 5204 STATISTIC 2008.09.08 23:40:17 3961 excluded files because rule = 78 STATISTIC 2008.09.08 23:40:17 3961 included files because rule = 0 STATISTIC 2008.09.08 23:40:17 3961 max size of copy queue = 22 STATISTIC 2008.09.08 23:40:17 3961 max size of compression queue = 361 STATISTIC 2008.09.08 23:40:17 3961 calculaed md5 sums = 50606 STATISTIC 2008.09.08 23:40:17 3961 forks total = 9176 STATISTIC 2008.09.08 23:40:17 3961 forks md5 = 3957 STATISTIC 2008.09.08 23:40:17 3961 forks copy = 12 STATISTIC 2008.09.08 23:40:17 3961 forks bzip2 = 5204 STATISTIC 2008.09.08 23:40:17 3961 sum of source = 10G (10965625851) STATISTIC 2008.09.08 23:40:17 3961 sum of target all = 10.0G (10731903808) STATISTIC 2008.09.08 23:40:17 3961 sum of target all = 97.87% STATISTIC 2008.09.08 23:40:17 3961 sum of target new = 109M (114598007) STATISTIC 2008.09.08 23:40:17 3961 sum of target new = 1.05% STATISTIC 2008.09.08 23:40:17 3961 sum of md5ed files = 744M (779727492) STATISTIC 2008.09.08 23:40:17 3961 sum of md5ed files = 7.11% STATISTIC 2008.09.08 23:40:17 3961 sum internal linked (copy) = 32k (32472) STATISTIC 2008.09.08 23:40:17 3961 sum internal linked (compr) = 6.2M (6543998) STATISTIC 2008.09.08 23:40:17 3961 sum old linked (copy) = 3.3G (3515951642) STATISTIC 2008.09.08 23:40:17 3961 sum old linked (compr) = 6.6G (7094777689) STATISTIC 2008.09.08 23:40:17 3961 sum unchanged (copy) = 0.0 (0) STATISTIC 2008.09.08 23:40:17 3961 sum unchanged (compr) = 0.0 (0) STATISTIC 2008.09.08 23:40:17 3961 sum new (copy) = 11M (11090534) STATISTIC 2008.09.08 23:40:17 3961 sum new (compr) = 99M (103507473) STATISTIC 2008.09.08 23:40:17 3961 sum new (compr), orig size = 321M (336637589) STATISTIC 2008.09.08 23:40:17 3961 sum new / orig = 32.96% STATISTIC 2008.09.08 23:40:17 3961 size of md5CheckSum file = 16M (16271962) STATISTIC 2008.09.08 23:40:17 3961 size of temporary db files = 0.0 (0) STATISTIC 2008.09.08 23:40:17 3961 precommand duration = 1s STATISTIC 2008.09.08 23:40:17 3961 deleted old backups = 0 STATISTIC 2008.09.08 23:40:17 3961 deleted directories = 0 STATISTIC 2008.09.08 23:40:17 3961 deleted files = 0 STATISTIC 2008.09.08 23:40:17 3961 (only) removed links = 0 STATISTIC 2008.09.08 23:40:17 3961 freed space in old directories = 0.0 (0) STATISTIC 2008.09.08 23:40:17 3961 add. used space in files = 125M (130869969) STATISTIC 2008.09.08 23:40:17 3961 backup duration = 27m3s STATISTIC 2008.09.08 23:40:17 3961 over all files/sec (real time) = 297.30 STATISTIC 2008.09.08 23:40:17 3961 over all files/sec (CPU time) = 549.22 STATISTIC 2008.09.08 23:40:17 3961 CPU usage = 54.13%
It took about 27 minutes to run the backup.
But look at the number of calculated md5 sums: 50,606. This is the number
of files, a ``normal'' backup (which does not examine the contents)
would have saved because a time stamp has changed or they have moved
(I didn't move files around, the changes were mainly from OS
updates.). StoreBackup calculates the md5 sums and recognises that
only 8,100 files (copied + compressd files) have
changed.
So only 16% of the files which normally whould have been saved had to
be stored. Over the time, this makes a big difference in the space you
need for your backups. And naturally, the files in the backup are
compressed (if reasonable).
Because the backup ran with the option lateLinks, I later had to run (via cron) storeBackupUpdateBackup.pl to set all the links etc.:
INFO 2008.09.09 02:17:52 13323 updating </disk1/store-backup/fschjc-gentoo-all/2008.09.08_23.13.14> INFO 2008.09.09 02:17:52 13323 phase 1: mkdir, symlink and compressing files STATISTIC 2008.09.09 02:18:18 13323 created 43498 directories STATISTIC 2008.09.09 02:18:18 13323 created 12024 symbolic links STATISTIC 2008.09.09 02:18:18 13323 compressed 0 files STATISTIC 2008.09.09 02:18:18 13323 used 0.0 instead of 0.0 (0 <- 0) INFO 2008.09.09 02:18:18 13323 phase 2: setting hard links STATISTIC 2008.09.09 02:27:55 13323 linked 462267 files INFO 2008.09.09 02:27:55 13323 phase 3: setting file permissions STATISTIC 2008.09.09 02:31:05 13323 set permissions for 482442 files INFO 2008.09.09 02:31:05 13323 phase 4: setting directory permissions STATISTIC 2008.09.09 02:31:47 13323 set permissions for 43498 directories
It took about 14 minutes to ``complete'' the backup for 500,000 entries.
Heinz-Josef Claes 2014-04-20