Tarlz is a massively parallel (multi-threaded) combined implementation of the tar archiver and the lzip compressor. Tarlz uses the compression library lzlib.
Tarlz creates tar archives using a simplified and safer variant of the POSIX pax format compressed in lzip format, keeping the alignment between tar members and lzip members. The resulting multimember tar.lz archive is backward compatible with standard tar tools like GNU tar, which treat it like any other tar.lz archive. Tarlz can append files to the end of such compressed archives.
Keeping the alignment between tar members and lzip members has two advantages. It adds an indexed lzip layer on top of the tar archive, making it possible to decode the archive safely in parallel. It also minimizes the amount of data lost in case of corruption. Compressing a tar archive with plzip may even double the amount of files lost for each lzip member damaged because it does not keep the members aligned.
Tarlz can create tar archives with five levels of compression granularity; per file (--no-solid), per block (--bsolid, default), per directory (--dsolid), appendable solid (--asolid), and solid (--solid). It can also create uncompressed tar archives.
Of course, compressing each file (or each directory) individually can't achieve a compression ratio as high as compressing solidly the whole tar archive, but it has the following advantages:
Note that the POSIX pax format has a serious flaw. The metadata stored in pax extended records are not protected by any kind of check sequence. Corruption in a long file name may cause the extraction of the file in the wrong place without warning. Corruption in a large file size may cause the truncation of the file or the appending of garbage to the file, both followed by a spurious warning about a corrupt header far from the place of the undetected corruption.
Metadata like file name and file size must be always protected in an archive format because of the adverse effects of undetected corruption in them, potentially much worse that undetected corruption in the data. Even more so in the case of pax because the amount of metadata it stores is potentially large, making undetected corruption and archiver misbehavior more probable.
Headers and metadata must be protected separately from data because the integrity checking of lzip may not be able to detect the corruption before the metadata have been used, for example, to create a new file in the wrong place.
Because of the above, tarlz protects the extended records with a Cyclic Redundancy Check (CRC) in a way compatible with standard tar tools.
Tarlz does not understand other tar formats like gnu, oldgnu, star, or v7. The command 'tarlz -tf archive.tar.lz > /dev/null' can be used to check that the format of the archive is compatible with tarlz.
The diagram below shows the correspondence between each tar member (formed by one or two headers plus optional data) in the tar archive and each lzip member in the resulting multimember tar.lz archive, when per file compression is used:
tar +========+======+=================+===============+========+======+========+ | header | data | extended header | extended data | header | data | EOA | +========+======+=================+===============+========+======+========+ tar.lz +===============+=================================================+========+ | member | member | member | +===============+=================================================+========+
The manual is available in the info system of the GNU Operating System. Use info to access the top level info page. Use info tarlz to access the tarlz section directly.
An online manual for tarlz can be found at manual/tarlz_manual.html.
The latest released version of tarlz can be found at http://download.savannah.gnu.org/releases/lzip/tarlz/. Tarlz requires lzlib to build. Lzlib is available at http://www.nongnu.org/lzip/lzlib.html. You may also subscribe to lzip-bug and receive an email every time a new version is released.
Of course, tarlz can extract its own source tarballs. But if you do not have tarlz installed in your system, you may need lzip from http://www.nongnu.org/lzip/lzip.html to extract the files.
Once lzip is installed, you may compile and optionally install tarlz by running the following commands:
tar -xf tarlz[version].tar.lz cd tarlz[version] && ./configure && make check
then (as root) type:
make install
For general discussion of errors (bugs) in tarlz, the mailing list lzip-bug@nongnu.org is the most appropriate forum. Please send messages as plain text. Do not send HTML, base64 MIME, nor multiple formats. Use a descriptive subject line with the word "tarlz" in it.
An archive of the bug report mailing list is available at http://lists.gnu.org/mailman/listinfo/lzip-bug.
To contact the author, either to report an error (bug) or to contribute fixes or improvements, send mail to lzip-bug@nongnu.org. Please send messages as plain text. If posting patches they should be in unified diff format against the latest version. They should include a text description.
If you know someone who is using gzip, bzip2, or xz, please tell him/her about the advantages of switching to lzip. See this comparison of the formats and this comparison of the tools.
See also the lzip project page at Savannah.
Tarlz is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.
Copyright © 2024 Antonio Diaz Diaz
Lzip logo Copyright © 2013 Sonia Diaz Pacheco
You are free to copy, modify, and distribute all or part of this article without limitation.
Updated: 2024-12-11
This page does not use javascript.