What does it mean that lbzip2 is a block compressor?
It means that if file being compressed is big enough lbzip2 will internally divide it into blocks of data and compresses them independently. This allows number of blocks to be processed in parallel, as well as makes it possible to recover data from partially damaged .bz2 files. It does not mean that lbzip2 can compress only block devices, nor that input file size must be multiply of block size.
Why progress estimation doesn’t work with pipes?
Knowledge of input file length is necessary for lbzip2 to be able to estimate current progress and predict ETA. lbzip2 uses fstat(2) system call to obtain this information. When processing files which size is not available, such as pipes, named FIFO queues, or some special devices) lbzip2 has no way of measuring progress.
Does lbzip2 support non-regular files?
Yes, lbzip2 should work correctly with almost any kind of files. This includes regular disk files, pipes, block devices, character devices, FIFOs, network sockets, UNIX-domain sockets, streams and any other special files, as long as they support read(2) and/or write(2) system calls. Files don’t need to support random access system calls, like lseek(2) or pread(2).
Note that when you execute lbzip2 from shell you probably will only be able to open named files and pipes. For example, lbzip2 won’t open any network connections on its own, but it should work with network sockets if you pass it as one of three standard file descriptors.
How can I enable progress indicator by default?
By default lbzip2 doesn’t display any information about files it processes. This behavior follows what bzip2 does. Progress measurement can be useful, especially when processing large files.
You can pass any arguments to lbzip2 via it’s environmental variable LBZIP2. Simply add a line such as this to your shell configuration file:
For more information about environmental variables and options accepted by lbzip2, see the manual page.
What’s the recommended way of using lbzip2 with GNU tar?
GNU tar has built-in support for lbzip2. This means it will use lbzip2 for compression and decompression of bz2 tarballs provided that bzip2 is not installed.
To use lbzip2 no matter if bzip2 is installed or not, you need to configure it at build time or use --use-compress-program option. For more information see GNU tar documentation about using lbzip2 with GNU tar.
How can I concatenate compressed files?
To concatenate files while recompressing them you can run:
Files in bz2 format can be simply concatenated without decompression, so it is also possible to concatenate them directly – just use the cat utility:
The second solution may be faster, but be aware that it is also less portable. While lbzip2 and recent versions of bzip2 will be able to decompress the resulting file, other software may have problems. In particular the following software is known not to support concatenated streams: libbzip2 (all versions), bzip2 older than version 0.9.0, Apache Commons Compress.