{
  "schemaVersion": 1,
  "contentVersion": "ca42e61ede0a24f789d7287a0001244ad22f4abf",
  "id": "docs/reference/data-compression-tools",
  "collection": "docs",
  "title": "Data compression and archive tools",
  "description": "Which general-purpose compressor or archive format to use for backups, downloads, web assets and logs, with typical commands for each.",
  "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/",
  "apiUrl": "https://encode.wiki/api/v1/documents/docs/reference/data-compression-tools.json",
  "markdownUrl": "https://encode.wiki/api/v1/markdown/docs/reference/data-compression-tools.md",
  "status": "current",
  "tags": [],
  "license": "CC BY-SA 4.0",
  "dates": {
    "lastVerified": "2026-10-08T00:00:00.000Z"
  },
  "authors": [],
  "sources": [
    {
      "title": "Zstandard command-line manual (v1.5.7)",
      "url": "https://github.com/facebook/zstd/blob/v1.5.7/programs/zstd.1.md",
      "accessedAt": "2026-10-08T00:00:00.000Z"
    },
    {
      "title": "RFC 9110 — HTTP Semantics (Accept-Encoding)",
      "url": "https://www.rfc-editor.org/rfc/rfc9110#name-accept-encoding",
      "accessedAt": "2026-10-08T00:00:00.000Z"
    },
    {
      "title": "RFC 9659 — Window Sizing for Zstandard Content Encoding",
      "url": "https://www.rfc-editor.org/rfc/rfc9659",
      "accessedAt": "2026-10-08T00:00:00.000Z"
    },
    {
      "title": "IANA HTTP content coding registry",
      "url": "https://www.iana.org/assignments/http-parameters/http-parameters.xhtml#content-coding",
      "accessedAt": "2026-10-08T00:00:00.000Z"
    },
    {
      "title": "Can I use — zstd content-encoding",
      "url": "https://caniuse.com/zstd",
      "accessedAt": "2026-10-08T00:00:00.000Z"
    },
    {
      "title": "XZ Utils NEWS",
      "url": "https://github.com/tukaani-project/xz/blob/v5.8.4/NEWS",
      "accessedAt": "2026-10-08T00:00:00.000Z"
    }
  ],
  "headings": [
    {
      "depth": 2,
      "text": "Which one should I use?",
      "anchor": "which-one-should-i-use"
    },
    {
      "depth": 2,
      "text": "The formats at a glance",
      "anchor": "the-formats-at-a-glance"
    },
    {
      "depth": 2,
      "text": "Speed against size",
      "anchor": "speed-against-size"
    },
    {
      "depth": 2,
      "text": "HTTP content coding",
      "anchor": "http-content-coding"
    },
    {
      "depth": 2,
      "text": "Checking an archive",
      "anchor": "checking-an-archive"
    }
  ],
  "sections": [
    {
      "id": "_preamble",
      "title": "Page introduction",
      "depth": 0,
      "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/",
      "apiUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/_preamble.json",
      "markdownUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/_preamble.md",
      "checksum": "b578db930726ef8a509ccef9f45e2a82415614e799d2d721764814d7c8bbda40",
      "bytes": 295
    },
    {
      "id": "which-one-should-i-use",
      "title": "Which one should I use?",
      "depth": 2,
      "anchor": "which-one-should-i-use",
      "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/#which-one-should-i-use",
      "apiUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/which-one-should-i-use.json",
      "markdownUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/which-one-should-i-use.md",
      "checksum": "e06fef3979248631eac09f3a0a9f39b9c154c11e1084efe47bf571ba267c0a59",
      "bytes": 1700
    },
    {
      "id": "the-formats-at-a-glance",
      "title": "The formats at a glance",
      "depth": 2,
      "anchor": "the-formats-at-a-glance",
      "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/#the-formats-at-a-glance",
      "apiUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/the-formats-at-a-glance.json",
      "markdownUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/the-formats-at-a-glance.md",
      "checksum": "74a5b9275fb2983b8f5074b124cefe89dd7c9a609729dee7e3a1e2b25e034c52",
      "bytes": 1927
    },
    {
      "id": "speed-against-size",
      "title": "Speed against size",
      "depth": 2,
      "anchor": "speed-against-size",
      "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/#speed-against-size",
      "apiUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/speed-against-size.json",
      "markdownUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/speed-against-size.md",
      "checksum": "71f1ffbf2265700f7323ec6aa075bdc8a370b81ef5d7d3de5adaba2c2269c56b",
      "bytes": 473
    },
    {
      "id": "http-content-coding",
      "title": "HTTP content coding",
      "depth": 2,
      "anchor": "http-content-coding",
      "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/#http-content-coding",
      "apiUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/http-content-coding.json",
      "markdownUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/http-content-coding.md",
      "checksum": "7bcebcd08fe622ead8db30c8f8e7c82585539f89f8fec9cac7e17d3d018d3017",
      "bytes": 602
    },
    {
      "id": "checking-an-archive",
      "title": "Checking an archive",
      "depth": 2,
      "anchor": "checking-an-archive",
      "canonicalUrl": "https://encode.wiki/reference/data-compression-tools/#checking-an-archive",
      "apiUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/checking-an-archive.json",
      "markdownUrl": "https://encode.wiki/api/v1/sections/docs/reference/data-compression-tools/checking-an-archive.md",
      "checksum": "a451cbd0ee53f51b698e75c76045eae35968f14fded0b4dbcdba658bab4657e8",
      "bytes": 262
    }
  ],
  "outboundReferences": [
    "/codecs/data/7z/",
    "/codecs/data/brotli/",
    "/codecs/data/bzip2/",
    "/codecs/data/gzip/",
    "/codecs/data/tar/",
    "/codecs/data/xz/",
    "/codecs/data/zip/",
    "/codecs/data/zpaq/",
    "/codecs/data/zstd/",
    "https://www.rfc-editor.org/rfc/rfc9659"
  ],
  "checksum": "3f44e869d5bdeafe07b90cbc868ee3b6d61ee0494195440e77cc8f1e976445af",
  "contentFormat": "markdown",
  "markdown": "This wiki is about media encoding, but you will still compress logs, project folders, caches, subtitles and source tarballs. These pages cover the general-purpose tools. None of them shrink video, audio or image files: those are already compressed, and re-compressing them saves almost nothing.\n\n## Which one should I use?\n\n| You want…                                          | Use                                                                    | Typical command                      |\n| -------------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------ |\n| A good default for anything on Linux/macOS         | [zstd](/codecs/data/zstd/)                                             | `zstd -T0 file`                      |\n| Backups of a directory with Unix metadata          | [tar](/codecs/data/tar/) + zstd or xz                                  | `tar --zstd -cf backup.tar.zst dir/` |\n| Smallest file you will compress once               | [xz](/codecs/data/xz/) or [7z](/codecs/data/7z/)                       | `xz -9e file`                        |\n| A file anyone can open without installing software | [ZIP](/codecs/data/zip/)                                               | `zip -r archive.zip dir/`            |\n| Static web assets                                  | [brotli](/codecs/data/brotli/) (+ [gzip](/codecs/data/gzip/) fallback) | `brotli -q 11 file`                  |\n| Maximum compatibility, scripts, old systems        | [gzip](/codecs/data/gzip/)                                             | `gzip -9 -k file`                    |\n| Incremental, deduplicated, versioned backups       | [zpaq](/codecs/data/zpaq/)                                             | `zpaq a backup.zpaq dir/`            |\n| Opening old `.tar.bz2` files                       | [bzip2](/codecs/data/bzip2/)                                           | `bzip2 -d file.bz2`                  |\n\n## The formats at a glance\n\n| Format | Archive?                   | Strength                                                    | Weakness                                              |\n| ------ | -------------------------- | ----------------------------------------------------------- | ----------------------------------------------------- |\n| gzip   | No (single stream)         | Universal, fast, tiny memory                                | Weakest ratio                                         |\n| bzip2  | No                         | Better than gzip on text                                    | Slow, superseded                                      |\n| xz     | No                         | Very small output, cheap to decompress                      | Slow and memory-hungry to compress                    |\n| zstd   | No                         | Fast at every level, close to xz at the top, multi-threaded | Less universal than gzip; HTTP window limit           |\n| brotli | No                         | Best for web text (built-in dictionary)                     | Slow at high quality; web-oriented                    |\n| tar    | Yes (no compression)       | Preserves Unix metadata; pairs with any compressor          | No index, no random access                            |\n| ZIP    | Yes (per-file compression) | Opens everywhere, random access                             | Weaker ratio; mixed support for newer methods and AES |\n| 7z     | Yes (solid)                | Strong ratio, AES-256, big dictionaries                     | Slow to extract single files; drops Unix owner/ACLs   |\n| ZPAQ   | Yes (journaling)           | Deduplication, every version restorable                     | Slow, rare                                            |\n\nA \"single stream\" format compresses one byte sequence. For directories, put them in tar or zip first, which is why you see `.tar.gz`, `.tar.xz` and `.tar.zst`.\n\n## Speed against size\n\nLevels trade time for size, with sharply diminishing returns. zstd `-3` and gzip `-6` are the fast defaults; zstd `-19` and xz `-6` are slow but noticeably smaller; xz `-9e`, 7z `-mx=9`, zstd `--ultra -22` and brotli `-q 11` are for files compressed once and read many times. If you compress frequently (logs, intermediate files, network transfers), stay at zstd `-1`…`-9`. zstd needs `-T0` to use all cores; xz has done so by default since 5.6.0.\n\n## HTTP content coding\n\nServers and browsers negotiate a coding with `Accept-Encoding` and `Content-Encoding`. gzip, `br` (Brotli) and `zstd` are the three registered codings that matter. gzip works everywhere; `br` is accepted by browsers over HTTPS; zstd is advertised by Chrome and Edge 123+, Firefox 126+ and Safari 26.3+. Always keep a fallback and serve with `Vary: Accept-Encoding`. zstd for HTTP is limited to an 8 MiB window ([RFC 9659](https://www.rfc-editor.org/rfc/rfc9659)), so files made with `--long` or `--ultra` levels cannot be served that way. xz, 7z and bzip2 are not HTTP codings.\n\n## Checking an archive\n\n`zstd -t`, `xz -t`, `gzip -t`, `bzip2 -t`, `brotli -t`, `unzip -t`, `7z t` and `zpaq x -test` verify integrity without extracting. The built-in checks catch accidental corruption, but not tampering: publish a SHA-256 alongside downloads.\n"
}