zpaq

ZPAQ is a format and journaling archiver by Matt Mahoney. Where other tools overwrite or rewrite archives, zpaq only ever appends: each run adds a new version containing the changes since the last one, with identical data deduplicated across files and versions. Every earlier state remains restorable. That makes it a good fit for incremental backups of a directory that changes slowly, such as projects, VM images or databases, where it often beats everything else on stored size.

The ZPAQ format is self-describing: each archive embeds the decompression program for its own data, so any compliant reader can open archives written with any compression model, including ones that did not exist at the time. The reference tool is zpaq 7.15 from 2016, and the format has been stable since.

Common commands

zpaq a backup.zpaq dir/                    # add (first run: full; later runs: only changes)
zpaq a backup.zpaq dir/ -m5                # slower, better compression
zpaq l backup.zpaq                         # list the current contents
zpaq l backup.zpaq -all                    # list every version
zpaq x backup.zpaq dir/ -to restore/       # restore the latest version into restore/
zpaq x backup.zpaq dir/ -until 3 -to old/  # restore as of version 3
zpaq x backup.zpaq -test                   # verify without writing

Put the options after the file names. Options that matter:

Option	Effect
-m0 … -m5	Method: 0 stores (deduplication only), 1 is default and fast, 5 is the slowest and best
-t N	Number of threads (default: all cores)
-key password	Encrypt the archive
-until N	List or extract as of version N (or -N counting from the end)
-to path	Extract into a different location
-f	Add files whose contents changed even if dates did not

By default zpaq a decides what changed by file modification dates, so a file touched without changes is re-checked cheaply through deduplication.

What it is good and bad at

• Good: incremental backup, deduplication, high ratios on text, source trees and disk images, restoring any past version, and appending without rewriting.
• Bad: speed at high -m levels (compression and decompression speed are about the same, and slow), and wide availability: it is not installed by default anywhere and there is no native OS support or HTTP coding. Extraction needs the zpaq tool or a compatible library.
• No in-place deletion. Removed files are tracked as deletions; the data remains until you rewrite the archive (-repack, or extract and re-add).
• Single file by design. An archive name with ???? (quote it in the shell), such as 'backup_????.zpaq', writes numbered parts, and you must keep every part.

For archives you write once and read widely, use 7z, xz or zstd.