# Encode Wiki full corpus Content version: d505714a5a686d22c08eede79a428587acaa46b8 # AV1 Encoding for Dummies Canonical URL: https://encode.wiki/articles/av1-encoding-for-dummies/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 17a1eafe2fa5bc78e692d6e62111eb65349194a12c864bdea4eb0cb3c87c7ea1 This guide will show you how to encode in AV1 the *right* and *optimal* way. Yes, you using standalone ``libaom``, ``libsvtav1``, and ``librav1e`` from FFmpeg or even piping ``yuv4mpeg`` into **mainline** aomenc are all unoptimal. > **Danger — Outdated Information.** > > While a lot of the information presented in this guide is still relevant and correct, the AV1 ecosystem has changed dramatically since this guide's inception which has demanded a rewrite. Please see the [AV1 for Dummies](/articles/av1-for-dummies/) blog post for more information. ![Compare](/img/compare-guide.webp) In this guide, we'll be installing Av1an for chunked encoding and infinite threading, because the current state of AV1 encoders, except for [SVT-AV1](/tools/software-encoders/svt-av1/), unfortunately lacks threading and will only use very low amount of cores, which hampers speeds. The only caveat to this approach is **RAM consumption**, encoding 2160p (4K) with [aomenc](/tools/software-encoders/aomenc/) with 4 workers could take upwards of **16GB** of RAM! So do keep this in mind. ## Installing the Tools Given all of the different operating systems that people use on a day to day basis and the various different encoding workflows that exist, there are a number of ways to do this. **Jump to**: [Windows](#microsoft-windows) | [macOS](#macos) | [Linux](#linux) ## Microsoft Windows ### The GUI Way 1. Install [NMKODER](https://github.com/n00mkrad/nmkoder) which is a GUI front-end to av1an with all dependencies installed. 2. You're done, you can skip to the encoding part > **Danger — Almost abandonware.** > > Since Nmkoder already ships everything by default and its last release was 29th March 2022. You need to manually update all encoders and tools to get better encoding speeds. Missing out on updates will result in your encodes being sub-optimal. ### The WSL2 Way *(Recommended)* If you're not already familiar with WSL2, the The Windows Subsystem for Linux (WSL) is a feature of the Windows operating system that allows you to run a Linux file system, along with Linux command-line tools and GUI apps, directly on Windows. This lets Linux distributions run on bare metal without managing any virtual machines, so encoding performance is very good. The easiest way to encode with WSL2 is to use [rAV1ator CLI](/tools/utilities/rav1ator-cli/), an interactive TUI for [Av1an](/tools/utilities/av1an/). An ArchWSL2 installation tutorial is provided [here](/tools/utilities/rav1ator-cli/#windows). ### The Automated Way There is now a batch script for automating the install process, which can be found [here](https://github.com/Hishiro64/av1an-win-script). The instructions are in the README file. > **Caution — The script will download outdated version encoders and tools such as `aom-av1-psy` and MKVToolNix v76.0, if you are fine with these you can proceed..** > > ### The Manual Way 1. Install **Python 3.10.x, this will change so consult from the** [Vapoursynth website](http://www.vapoursynth.com/doc/installation.html) **if you're reading this from the future** from [here](https://www.python.org/downloads/windows/) and select "Windows Installer 64-bit". Upon installation check the tick for adding Python to PATH like so ![Python PATH](/img/python-path.webp)) 2. Download and install Vapoursynth from [here](https://github.com/vapoursynth/vapoursynth/releases) and select "VapourSynth64-RXX.exe" 3. Open the terminal and type ``vsrepo.py install lsmas ffms2`` to install some plugins for Av1an to work. 4. Download MKVToolNix from [here](https://mkvtoolnix.download/downloads.html#windows), select "mkvtoolnix-64bit-XX.X.X-setup.exe", and install **(Also available on winget!)** 5. Download Av1an from [here](https://github.com/master-of-zen/Av1an/releases) (SELECT LATEST AND CLICK THE "ASSETS" DROPDOWN) 6. Download **shared libraries** FFmpeg from [gyan.dev](https://www.gyan.dev/ffmpeg/builds) 7. Download or build the Aomenc fork [aom-av1-lavish](https://github.com/Clybius/aom-av1-lavish/tree/Endless_Merging), which has features such as different defaults, tunes, and optimizations. The Windows binary linked by the original September 2023 guide is no longer available; check the project or AV1 community channels for a current build. > **Info — If you opt to compile aomenc yourself, you can view the instructions on how to do that [here](/tools/software-encoders/aomenc/#installation)..** > > 8. Move Av1an, FFmpeg **(Including the FFmpeg DLLs)**, and aomenc to somewhere preferable, eg ``C:\Encoding``. 9. Add the folder **AND MKVTOOLNIX INSTALLATION FOLDER** to the [Windows PATH environment](https://www.maketecheasier.com/what-is-the-windows-path/). ## macOS macOS is very similar to Linux, although there aren't any GUI tools for AV1 encoding that I can comfortably recommend. **Homebrew + Macports for Av1an + rav1e:** *Note that some commands may have to be run with `sudo`, which I won't explicitly include for security reasons.* Installing the Homebrew package manager is a well documented process at this point: ```bash /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" ``` As is installing MacPorts. Install the relevent `.pkg` for your macOS version from the MacPorts Project website: [www.macports.org/install.php](https://www.macports.org/install.php) Now, you can run the following commands: ```bash brew update && brew upgrade brew install rav1e aom mkvtoolnix ffmpeg # Usually you must run MacPorts commands for package installations as root port upgrade outdated port install av1an ``` This is the easiest way to get everything set up & working to produce AV1 video with `rav1e` or mainline `aomenc` & Av1an. You can check that things are installed by running the following commands & parsing their output: ```bash % av1an --version av1an 0.4.1-unstable (rev e10880d) (Release) * Compiler rustc 1.70.0 (LLVM 16.0) * Target Triple aarch64-apple-darwin * Date Info Commit Date: 2023-06-25 * VapourSynth Plugins systems.innocent.lsmas : Not found com.vapoursynth.ffms2 : Not found ``` ```bash % rav1e --version | grep "release" -C 1 rav1e 0.6.6 () (release) rustc 1.69.0 (84c898d65 2023-04-16) (built from a source tarball) aarch64-apple-darwin ``` ```bash % aomenc --help | grep "AOMedia" -C 3 Included encoders: av1 - AOMedia Project AV1 Encoder 3.6.1 (default) Use --codec to switch to a non-default encoder. ``` Notice `systems.innocent.lsmas : Not found` in the Av1an output. This means you won't be able to use the lsmash chunking method through vapoursynth & may instead have to rely on hybrid chunking, through `-m hybrid`. This is slower & takes up disk space while encoding, but still works. A sample Av1an command with this basic installation may look like this: ```bash av1an -i "input" -y --resume --verbose --split-method av-scenechange -m hybrid -c mkvmerge -e rav1e --force -v " --tiles 8 -s 4 --quantizer 80 --no-scene-detection" --photon-noise 7 --chroma-noise --pix-format yuv420p10le -w 8 -o "output.mkv" ``` **Building From Source** If you want lsmash support, aom-av1-lavish instead of mainline, or anything else that isn't covered by the more basic installation, you'll have to compile from source. Things are very similar to Linux, with a few oddities: - macOS sometimes doesn't have a `/usr/local/bin` by default. You can fix this by doing `mkdir /usr/local/bin`. - Homebrew installs *everything* in its own directory structure. If you're building things from source that rely on libraries from vapoursynth, zimg, lsmash, etc, make sure to copy them from `/opt/homebrew/lib` to `/usr/local/lib`. Finding them is a matter of `ls | grep "keyword"` & copying what looks reasonable to be associated with the tool you're using. - Building most things from source will have instructions for \*nix which work for both macOS & Linux. Even if it says Linux, there's a good chance it'll work on macOS as well, & it is always worth trying Linux build instructions on Mac. I won't be going through building every encoding tool & dependency from source, as it is generally much more intuitive than Windows, but building Av1an is worth detailing here just as an example. ```bash brew install git rust nasm git clone https://github.com/master-of-zen/Av1an cd Av1an RUSTFLAGS="-C target-cpu=native" cargo build --release cd .. && cd target/release cp av1an /usr/local/bin ``` **More Difficult: Building aom-av1-lavish from Source** If you want to make the most out of your hardware & eke out every last drop of quality, it may be worth building aom-av1-lavish from source. The first step is to clone it from the Endless Merging branch: ```bash git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging cd aom-av1-lavish ``` Now, you need to make some manual changes to the source code until Clybius merges [this commit](https://github.com/Clybius/aom-av1-lavish/pull/1/files). - Add the line `#include "aq_variance.h"` at line 19 in `av1/encoder/encodeframe_utils.c` - Comment out line 2546 in `av1/encoder/speed_features.c`. This line is `const int qindex_thresh_cdef_sf_s1_s3_l2[2] = { 92, 48 };` & becomes `// const int qindex_thresh_cdef_sf_s1_s3_l2[2] = { 92, 48 };`. Now you can continue to build according to the Linux instructions below. Obviously you'll need cmake, which you can install with homebrew along with any other tools you may need. While still in the `aom-av1-lavish` directory: ```bash mkdir -p aom_build && cd aom_build cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native" make -j$(nproc) # This may need to be run as root: make install ``` Now you can run `aomenc --help | grep "AOMedia" -C 3` to see if lavish installed. If you're getting the same output as above, you may need to copy the `aomenc` executable to `/opt/local/bin`, `/usr/local/bin`, & `/opt/homebrew/bin` if you already installed mainline aomenc. Running the version info command again, the correct output should look something like this: ```bash % aomenc --help | grep AOMedia -C 3 Included encoders: av1 - AOMedia Project AV1 Encoder Psy v3.6.0 (default) Use --codec to switch to a non-default encoder. ``` Notice how it says `AOMedia Project AV1 Encoder Psy` instead of `AOMedia Project AV1 Encoder`. You should be all set after this to start using aom-av1-lavish & following the current parameter meta as outlined below. ## Linux > **Info — Yet again, try using Arch. It's way easier..** > > ### The GUI Way - Install [Aviator](https://github.com/gianni-rosato/aviator) ([SVT-AV1](/tools/software-encoders/svt-av1/) + [FFmpeg](/tools/utilities/ffmpeg/)) or [rAV1ator](https://giannirosato.com/blog/post/aviator-1/) basically same thing but [Av1an](/tools/utilities/av1an/) + [rav1e](/tools/software-encoders/rav1e/). Both are only available as [Flatpaks](https://beta.flathub.org/apps/net.natesales.Aviator). Keep in mind Aviator ships with **SVT-AV1** and rAV1ator with **rav1e** instead of aomenc/AOM-AV1, which I will not be covering here. ### The TUI Way *(Recommended)* - Install [rav1ator-cli](/tools/utilities/rav1ator-cli/), a TUI for using Av1an meant to be easy to use. Much more flexible than the GUI options & can work with a number of encoders. See [this page](/tools/utilities/rav1ator-cli/#installation) for more info. Can be easily used on any distro. ### The Compiling Route #### Ubuntu The guide below is targeted towards 22.04, packages and other things may be different on other versions. First Install Rust via `rustup` first, as apt version of Rust is severely outdated, then you can continue. Install dependencies: ```bash sudo apt install wget python unzip unrar build-essential meson autoconf automake libtool git nasm yasm python3-dev python3-pip cython3 libass-dev libqt5websockets5-dev libfftw3-dev libtesseract-dev ffmpeg libavcodec-dev libavformat-dev libswscale-dev libavutil-dev libswresample-dev libmediainfo-dev mkvtoolnix mediainfo perl nasm yasm git cmake libavutil-dev libavcodec-dev libavformat-dev libavdevice-dev libavfilter-dev libswscale-dev libswresample-dev libpostproc-dev llvm libclang-dev libssl-dev ``` Install l-smash: ```bash git clone https://github.com/l-smash/l-smash.git cd l-smash ./configure --enable-shared --extra-cflags="-march=native" make -j$(nproc) sudo make install ``` Install zimg: ```bash git clone --recursive https://github.com/sekrit-twc/zimg.git cd zimg ./autogen.sh ./configure make -j$(nproc) sudo make install ``` Install ImageMagick: ```bash git clone https://github.com/ImageMagick/ImageMagick cd ImageMagick ./configure make -j$(nproc) sudo make install ``` Install Vapoursynth R63: ```bash wget https://github.com/vapoursynth/vapoursynth/archive/refs/tags/R63.zip unzip R63.zip cd vapoursynth-R63 ./autogen.sh ./configure CFLAGS="-march=native" CXXFLAGS="-march=native" --libdir=/usr/lib make -j$(nproc) sudo make install sudo mkdir /usr/lib/vapoursynth sudo ldconfig ``` The plugin directory will be located in `/usr/lib/vapoursynth`. Install L-SMASH-Works Vapoursynth Plugin: ```bash git clone https://github.com/AkarinVS/L-SMASH-Works -b ffmpeg-4.5 cd L-SMASH-Works/VapourSynth && mkdir build && cd build meson .. --optimization=3 --default-library=static -Db_lto=true -Dc_args="-march=native" -Dcpp_args="-march=native" ninja -j$(nproc) sudo cp libvslsmashsource.so /usr/lib/vapoursynth/ ``` > **Danger — L-SMASH-Works doesn't work on **aarch64**, it is recommended to use other plugins instead..** > > Install FFMS2 Vapoursynth Plugin: ```bash git clone https://github.com/FFMS/ffms2 cd ffms2 ./autogen.sh ./configure CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native" make -j$(nproc) sudo cp src/core/.libs/libffms2.so src/core/.libs/libffms2.so.5 src/core/.libs/libffms2.so.5.0.0 /usr/lib/vapoursynth ``` Install Av1an: ```bash git clone https://github.com/master-of-zen/Av1an cd Av1an RUSTFLAGS="-C target-cpu=native" cargo build --release sudo cp target/release/av1an /usr/local/bin ``` When there's no errors, proceed to compiling `aom-av1-lavish`. ### Arch Install dependencies: ```bash sudo pacman -S vapoursynth ffmpeg av1an mkvtoolnix-gui git perl cmake ninja meson nasm vapoursynth-plugin-lsmashsource ffms2 ``` you're done, proceed. #### Compiling aom-av1-lavish ``` bash git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging cd aom-av1-lavish && mkdir -p aom_build && cd aom_build cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" make -j$(nproc) sudo make install ``` ## Encoding The moment you've all been waiting for, let's just get into it. Here's an example *recommended* parameter as of now (09/03/23) [MM/DD/YY]: ```bash av1an -x 300 -i input.mkv -w 4 -e aom -c mkvmerge --resume -m lsmash --photon-noise=10 --set-thread-affinity=2 --verbose -a " -an " -f " -an " -v " --bit-depth=10 --cpu-used=4 --end-usage=q --cq-level=24 --threads=2 --tile-columns=0 --tile-rows=0 --lag-in-frames=64 --tune-content=psy --tune=ssim --enable-keyframe-filtering=1 --disable-kf --kf-max-dist=9999 --enable-qm=1 --deltaq-mode=0 --aq-mode=0 --quant-b-adapt=1 --enable-fwd-kf=0 --arnr-strength=1 --sb-size=dynamic --enable-dnl-denoising=0 " -o "output.mkv" ``` > **Info — Parameter Meta.** > > It is strongly recommended to join the [AV1 Discord server](https://discord.gg/vpREHAvYvh) to get the latest updates on what to use and which to set, as it's the only easily reachable place for everything AV1 & encoding tips in general. Now let's dissect it one-by-one **Av1an parameters:** - ``-i`` Input. - ``-x 300`` Sets scene split length to 300 frames, you can increase it for more quality at the tradeoff of video seekability. - ``-w 4`` Specifies the amount of "workers" or amount of encoders working on the video. - ``--verbose`` Sets logging to verbose. - ``--resume`` Resumes the encode even when you haven't encoded yet. I strongly recommend leaving this if you resume a lot since you can accidentally delete your whole progress (There's no delete confirmation feature.. yet) if you "resumed" without the parameter in place. - ``-e aom`` Specifies we're using aomenc encoder which should be the default option. - ``-c mkvmerge`` Specifies we're using mkvmerge (MKVToolNix) to concatenate the parts when done, you can specify with ffmpeg if you want to but this is the best method. - ``-m lsmash`` Specifies we're using l-smash (Vapoursynth plugin) to split the videos, this is also the best method because ffms2 causes video lag (Tested a year ago, might change now) and other methods just suck (Slow and not worth it, learned the hard way). You can attempt to use ffms2 when inputting VC-1 videos as it is not possible with l-smash (Or convert it to lossless with x264 qp 0). - ``-f " -an "`` ``-f`` Stands for ffmpeg parameters, ``-an`` is to remove all audio since its better to encode and merge it separately. To crop use ``-f " -an -vf crop=1920:800 "`` for example to crop the video to 1920x800. - ``-v " "`` Is where you put the encoder's parameters in. - ``-a " -an "`` FFmpeg audio encoding options, we're removing it cause we can always add it later. But if you want to, you can also encode directly. Here's an example for encoding to Opus using libopus assuming stereo: `-a " -c:a libopus -b:a 128k "`. - ``--photon-noise=10`` AV1 grain synthesis, which is a technique where the encoder puts fake grain in so it looks more natural and potentially hiding video artifacts (cause grain is hard to encode and explodes bitrate usage because of their randomness), 5-8 for almost none to little grain, 10-14 for medium, 15+ heavy, 20+ extremely heavy, 30+ for extremely grainy 90s live action films. - ``--set-thread-affinity=2`` Pins the thread to the encoder, aligns with ``--threads=2`` in the encoder parameter so set them accordingly. **aomenc parameters:** - ``--bit-depth=10`` We're using 10bit because it makes the video smaller and reduces [banding](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Video_codecs#contouring). - ``--cpu-used=4`` This is the preset which ranges from 0-9, you can go to 3 if you want more efficiency, 2 if you have a lot of time, 4 is the sweet spot, and 6 if you want speed. Don't go above 6 (Worst efficiency) or even 0 (It would take WEEKS to finish). - ``--end-usage=q --cq-level=24`` This specifies that we are going to use a knockoff version of CRF level similar to x264/x265 encoders, in this case CRF 24. - `--threads=2` Sets the amount of threads the encoder can use, aligns with `--set-thread-affinity` in Av1an. - ``--tile-columns=0 --tile-rows=0`` This is the tiles options, where the encoder splits the videos into tiles to encode faster, see the image below (Yellow lines): Tiling > **Note — Tile usage.** > > Do NOT use tiles for 1080p and below, use 1 ``tile-columns`` at 1440p (2K), 2 ``tile-columns`` and 1 ``tile-rows`` for 2160p (4K) - ``--lag-in-frames=64`` Similar to x264/x265 `rc-lookahead`. Sets a number of frames to look ahead for frametype and ratecontrol, allowing for better compression decision making. Setting to a value greater than 64 is generally not considered useful. - ``--aq-mode`` adaptive quantization mode, 0 is better most of the time - ``--tune-content=psy --tune=ssim`` As the name suggests they are tunes that affect the video output, for the better, and for the worst > **Info — Tunes to use.** > > Set ``tune-content`` to ``animation`` if you're encoding above ``cq-level=30`` A.K.A lower quality, despite it's name > Set ``tune-content`` to ``psy`` for everything else, **do not use if you encode above ``cq-level=30``** > For ``tune``, this is a bit tricky. For now, the meta seems to be ``ssim``, but back then it was ``lavish`` which is considered THE best tune because it's based on [butteraugli](https://github.com/google/butteraugli). Now it's fallen behind because its more blurry than ``ssim``, and before that it was ``butteraugli``, and then ``ipq_vmaf_psy``, and finally just ``ipq``. > If you use any of the VMAF tunes, **you need to specify ``--vmaf-model-path=`` to where you put it**. - ``--enable-keyframe-filtering=1`` We're setting it to 1 because of compatibility reasons, 2 is more efficient but there are seeking issues and FFmpeg for some reason can't input it. - ``--sb-size=dynamic`` Allows the encoder to use 128x128 block partitioning besides 64x64 which gives an efficiency boost, ignore it. - ``--deltaq-mode`` set to 0 because its just better. - ``--arnr-strength=1`` Controls how strong the filtering will be, 1 is good for 3D Pixar CGI-like and 2D animation, use 4 if you're doing live action content. Using maximum at higher bitrates would just result in a blurry mess. - ``--disable-kf --enable-fwd-kf=0`` We're disabling keyframes cause **Av1an already did scene detection, so we wont have to.**. And it speeds things up. - ``--kf-max-dist=9999`` Maximum keyframe interval, we're setting it at the highest possible value since av1an's scene detection keyframe interval is already 240 by default - ``--enable-chroma-deltaq=1 --enable-qm=1 --quant-b-adapt=1`` Parameters that give you free efficiency boost. - ``--enable-dnl-denoising=0`` Disables the encoder's built-in denoising technique when grain synthesis is enabled, you can optionally set it to 1 when you have a pretty noisy video since it works quite well. > **Info — Concatenation Error on Linux.** > > Run ``ulimit -n 200000``, resume, and it should concatenate just fine. If it still errors, head to the encode directory > encode, and run ``mkvmerge @../options.json`` ## Merging Everything Once you're done just encode your audio using ffmpeg (or just passthrough it), subtitles should be carried along with your video output, and merge them in MKVToolNix! Don't want Matroska files? That's fine, you can use FFmpeg or MP4Box to output into `mp4`, just keep in mind that PGS/SUP/VOBSUB subtitles are not supported and Opus audio support is still experimental. ## Tips & Tricks - `--denoise-noise-level=10` Alternative to `photon-noise`, slower than photon-noise and is the OG grain synthesis method, performs okay and just serves as an alternative. Don't attempt to use it at high values (>12) since it creates noticeable grain patterns. - `--arnr-maxframes` to set max reference frames that will be used to filter the encode, higher values would make the video blurrier at high fidelity but look better at lower bitrates. - `--butteraugli-resize-factor=2` if you use any of the butteraugli-based tunes (lavish, butteraugli) to speed it up without much losses and `--butteraugli-intensity-target=250` to match the content light level. ## Final Thoughts Encoding has always been about experimentation for the best, there is really no "One size fits all" for encoding content, as they differ from scene complexity, how it's captured (2D/Real life), film grain, dark scenes, etc. So experiment away for your specific type of content! > **Guide originally hosted on https://rentry.co/AV1, rewrite and migration by Simulping.** --- # AV1 for Dummies Canonical URL: https://encode.wiki/articles/av1-for-dummies/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 752f6568d5b19b15805725b778e577aa0ec1de824332974b1cc4d4f7a2345893 AV1 for Dummies is a comprehensive, legible guide on how to get started with AV1 at any experience level. Whether you're on Windows using your first video encoding program, or a seasoned Linux user looking to optimize your encoding pipeline, this guide has you covered. ![Compare](/img/av1_for_dummies_guide.avif) ## Introduction [AV1](/codecs/video/av1/) is a royalty-free video codec developed by the Alliance for Open Media. It is designed to replace [VP9](/codecs/video/vp9/) and presently competes with [H.266](/codecs/video/vvc/). AV1 is known for its high compression efficiency, which the marketing will have you believe reduces file sizes by up to 50% compared to [H.264](/codecs/video/avc/) and up to 30% compared to [H.265](/codecs/video/hevc/) across the board. It is supported by several major browsers and is widely used across many streaming services and video platforms. ### Why AV1? Before we dive in, it is important to understand *why* you may want to use AV1 instead of other codecs. The reality is that AV1 is *not* better than H.264/5 in every single scenario; video encoding is a complicated field, and the best codec for you will depend on your specific needs. AV1 excels in: - Low to medium-high fidelity encoding - Higher resolution encoding - Encoding content with very little grain or noise - Slow, non-realtime contexts (e.g. offline encoding) The enumeration above still consists of broad strokes, but the point is to understand that AV1 is not a silver bullet. It will not automatically make your videos smaller while preserving your desired quality. To make things more difficult, the [x264](/tools/software-encoders/x264/) & [x265](/tools/software-encoders/x265/) encoders are very mature, while AV1 encoding efforts designed to meet the extremely complicated needs of the human eye are still in their infancy. This guide focuses almost entirely on tools great for offline, non-realtime encoding for various kinds of content. Streaming is an altogether different application of AV1, and is only covered sparsely here. ### Do's & Don'ts Due to a lot of misunderstandings about codecs and compression, there are a lot of common misconceptions that are held regarding video encoding. We'll start by outlining some bad practices: - **Don't** encode the same video multiple times. This is a common mistake made by people new to video encoding. Every time you encode a video, you lose additional quality due to *generation loss*. This is because video codecs are lossy, and every time you encode a video, you lose more information. This is why it is important to keep the original video file if you frequently re-encode it. - **Don't** blindly copy settings from others without understanding them. What works for one person's content and workflow may not work for yours. Even the default settings on many encoders are not optimal for most content. - **Don't** assume that higher bitrate equates to better quality. Inefficient encoding can waste bits without improving visual quality, and efficient encoding can make lower bitrate video look drastically better than higher bitrate video using the same codec. - **Don't** assume all encoders/presets/settings/implementations are created equal. Even given two encoding frameworks that use the same underlying encoder, you may achieve different results given encoder version mismatches or subtly different settings used under the hood. - **Don't** use unnecessarily slow presets/speeds unless you have a specific need and ample time. While slower presets improve encoding efficiency most of the time, the quality gains reach a point of diminishing returns beyond a certain point. Use the slowest preset you can tolerate, not the slowest preset available. - **Don't** blindly trust metric scores. It is unfortunate how trusted [VMAF](/reference/metrics/vmaf/) is considering how infrequently it correlates with visual fidelity in practice now that it has become so popular. Even the beloved [SSIMULACRA2](/reference/metrics/ssimulacra2/) is not a perfect one-to-one with the human eye. Now, let's move on to some **good** practices: - Experiment with different settings and compare the results. - Consider your content type when choosing encoding settings. Film, animation, and sports all have different characteristics that benefit from distinct approaches. - Try to use CRF for offline encoding, as opposed to CBR or VBR. While the latter two are effective for precisely targeting a particular bitrate, CRF is more effective at targeting a specific quality level efficiently. - Always use 10-bit color, even with an 8-bit source. AV1's internal workings are much more suited to 10-bit color, and you are almost always guaranteed quality improvements with zero compatibility penalty as 10-bit color is part of AV1's baseline profile. - Consider using grain synthesis for grainy content, as AV1 can struggle with preserving film grain efficiently. - Keep your encoding software up-to-date; the encoding world moves quickly. ## Tools Since writing the last guide, there are now several versatile tools available for AV1 encoding. Below, we've assembled some tables of popular encoding tools and their features. ### GUI Here's the key: - **Encoder(s)**: The AV1 encoder(s) the software uses or is able to use. This is between [SVT-AV1](/tools/software-encoders/svt-av1/), [aomenc](/tools/software-encoders/aomenc/), [rav1e](/tools/software-encoders/rav1e/), [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/), and various aomenc forks. We'll get into this more later. - **Ease of Use**: How intuitive the software is, especially for beginners. - **Complexity**: How flexible the software can be for advanced users. - **Efficiency**: How "good" the underlying encoder(s) are. This is more subjective, but tools with bad defaults, misleading options, or slow release cycles won't score well here. | Name | Platform(s) | Encoder(s) | Ease of Use | Complexity | Efficiency | |---------|-------------|:----------:|:-----------:|:-----------:|------------| | [Aviator](/tools/utilities/aviator/) | Linux | SVT-AV1-PSY| ●●●●● | ●○○○○ | ●●●●● | | [NMKODER](/tools/utilities/nmkoder/) | Windows | Any | ●●●○○ | ●●●●○ | ●●○○○ | | [Av1ation Station](https://github.com/Av1ation-Association/Av1ation-Creations/releases/) | Any | Any | ●●●○○ | ●●●●● | ●●●●○ | | [StaxRip](https://github.com/staxrip/staxrip) | Windows | Any | ●●○○○ | ●●●●● | ●●●●○ | | [NEAV1E](https://github.com/Alkl58/NotEnoughAV1Encodes) | Windows | Any | ●●●○○ | ●●●○○ | ●●○○○ | | [Handbrake](https://handbrake.fr/) | Any | SVT-AV1 | ●●●●○ | ●●○○○ | ●●○○○ | | [FastFlix](https://github.com/cdgriffith/FastFlix) | Any | Any | ●●●●○ | ●●●○○ | ●●○○○ | | [rAV1ator](/tools/utilities/rav1ator/) | Linux | rav1e | ●●●●○ | ●●●○○ | ●○○○○ | | [Autocompressor](https://autocompressor.net/) | Web | SVT-AV1 | ●●●●● | ●○○○○ | ●●●○○ | Our top pics are: - For beginners: **Aviator**. If you are on Linux and you want zero hassle whatsoever, Aviator is simple but highly effective. As a Flatpak application, it ships a bundled SVT-AV1-PSY binary that is up-to-date, compiled efficiently, and handled with extensively tested defaults. It is hard to go wrong with Aviator even as an advanced user, despite its simplicity. - For advanced users: **Av1ation Station**. Av1ator Station is thoughtfully designed to be your one-stop shop for video encoding. Even though it is brand new, it has proven itself to be a capable and reliable solution for advanced video encoders that have a lot of settings to tweak and encodes to keep track of. - For Windows users: **StaxRip**. StaxRip is a long-standing, well-maintained, and highly flexible video encoding tool that supports a plethora of encoders and formats. It comes bundled with SVT-AV1-PSY and is a great choice for Windows users who want a powerful and versatile tool for video encoding. ### CLI In the terminal, advanced encoders can explore a much greater degree of complexity than many GUIs allow. That being said, ease of use is still a consideration when managing complex command-line encoding workflows. Let's begin with the key: - We're omitting the "Platform(s)" column here, as this section targets Linux users. As a Windows user, WSL has you covered - as a Mac user, you're likely to experience parity with Linux (most of the time). | Name | Framework(s) | Ease of Use | Complexity | Features | Scene Detection | |------|--------------|:-----------:|:----------:|:--------:|:---------------:| | SvtAv1EncApp | None | ●○○○○ | ●●○○○ | ●○○○○ | No | | [FFmpeg](/tools/utilities/ffmpeg/) | Itself | ●●○○○ | ●●●●● | ●●●●○ | No | | [Av1an](/tools/utilities/av1an/) | FFmpeg | ●○○○○ | ●●●●● | ●●●○○ | Yes | | [rAV1ator CLI](/tools/utilities/rav1ator-cli/) | Av1an | ●●●●○ | ●●●●○ | ●●●●○ | Yes | | [alabamaEncoder](https://github.com/kingstefan26/alabamaEncoder) | FFmpeg | ●●○○○ | ●●●●● | ●●●●● | Yes | Our top picks are: - For beginners: **rAV1ator CLI**. rAV1ator CLI essentially walks you through the process of writing Av1an commands. It can install binaries for you, save your previous commands, and detect scenes via Av1an. This makes it a great choice for beginners who want to learn the ropes of AV1 encoding without diving into the deep end headfirst. For advanced users, it makes rapidly writing, testing, & cataloging Av1an encoding commands much simpler. - For advanced users: **alabamaEncoder**. alabamaEncoder is a powerful and flexible tool that allows you to encode video with FFmpeg and AV1. It is highly configurable and supports a mind-bending array of powerful features that would be hard to find elsewhere. While the tool is still new, it is already a great choice for advanced users who want to push the boundaries of what is possible to incorporate into an encoding workflow. ### Conclusion In conclusion, the best AV1 tool for you will depend on your needs and experience level. If you are a beginner, Aviator is a great choice for Linux users, while StaxRip is a good option for Windows users. For advanced users, Av1ation Station is a powerful and versatile tool that can handle numerous encoding tasks. If you prefer the command line, rAV1ator CLI is a great choice for beginners, while alabamaEncoder is a powerful tool for advanced users. No matter which tool you choose, it is hard to go wrong in the modern AV1 encoding landscape when it comes to your utility of choice. ## Encoders The world of AV1 encoding is diverse and complex, with several open-source encoders available, each bringing its own set of strengths, weaknesses, and unique features to the table. In this section, we'll dive deep into the characteristics of four major AV1 encoders: 1. SVT-AV1, 2. rav1e, 3. aomenc (libaom), and 4. SVT-AV1-PSY Understanding these encoders is crucial for making informed decisions about what best suits your specific encoding needs. ### SVT-AV1 [SVT-AV1](/tools/software-encoders/svt-av1/) (Scalable Video Technology for AV1) is an AV1 encoder library and application developed by Intel, Netflix, and others. It has gained significant popularity in the encoding community due to its impressive balance of speed, quality, and scalability. Links: - Wiki page: [SVT-AV1](/tools/software-encoders/svt-av1/) - Git repository: https://gitlab.com/AOMediaCodec/SVT-AV1 - Documentation: https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/README.md 1. **Performance & Scalability** - SVT-AV1 is renowned for its encoding speed, particularly at higher speed presets. - It leverages parallel processing, making it exceptionally efficient on multi-core systems. Fun fact: SVT-AV1's parallel processing is lossless, so it doesn't compromise quality for speed. 2. **Quality-to-Speed Ratio** - SVT-AV1 strikes an impressive balance between encoding speed and output quality. - At faster presets, it usually outperforms other encoders in quality per unit of encoding time. - While it may not achieve the absolute highest *quality per bit* possible, its quality is generally considered impressive for its speed. 3. **Flexibility** - SVT-AV1 offers a wide range of encoding options and presets, allowing fine-tuned control over the encoding process. - It provides 14 presets (0-13), with 0 being the slowest and highest quality, and 13 being the fastest but lowest quality. - Advanced options allow users to adjust parameters like hierarchical levels, intra-refresh type, and tuning modes. 4. **Continuous Development** - SVT-AV1 receives frequent updates and optimizations, with new releases often coming alongside big changes. - The open-source nature of the project encourages community contributions and rapid feature development. SVT-AV1 is an excellent choice for a wide range of encoding scenarios. It's particularly well-suited for: - High-volume encoding operations where speed is crucial - Live or near-live encoding of high-resolution content - Scenarios where a balance between quality and encoding speed is required - Users with multi-core systems who want to leverage their hardware efficiently Some downsides include: - Higher memory usage compared to other encoders - The developers assess quality via its performance on traditional legacy metrics, which harms its perceptual fidelity ceiling. ### rav1e [rav1e](/tools/software-encoders/rav1e/) is an AV1 encoder written in Rust & Assembly. Developed by the open-source community alongside Xiph, it brings a unique approach to AV1 encoding with its focus on safety and correctness. Links: - Wiki page: [rav1e](/tools/software-encoders/rav1e/) - Git repository: https://github.com/xiph/rav1e - Documentation: https://github.com/xiph/rav1e/tree/master/doc#readme 1. **Safety & Reliability** - Being written in Rust, rav1e emphasizes memory safety and thread safety. - This focus on safety translates to a more stable and reliable encoding process, with reduced risks of crashes or undefined behavior. 2. **High Fidelity** - At high fidelity targets - an area where AV1 usually lacks - rav1e is a strong contender compared to other encoders. - It excels in preserving fine details and textures, making it a good choice for high-fidelity encoding. 3. **Quality** - While not typically matching aomenc or SVT-AV1 in pure compression efficiency, rav1e can produce high-quality output videos. - It often achieves a good balance between quality and encoding time, especially at medium-speed settings. 4. **Perceptually Driven** - rav1e's development is driven by visual fidelity, without relying heavily on metrics. - This focus on perceptual quality leads to a stronger foundation for future potential improvements in visual quality, as well as making the encoder very easy to use as it does not require excessive tweaking. rav1e is well-suited for: - Projects where stability is paramount - Users who prioritize a community-driven, open-source development approach - Encoding tasks where a balance between quality and speed is needed, but the absolute fastest speeds are not required Some limitations of rav1e include: - Lagging development compared to other encoders - Slower encoding speeds compared to SVT-AV1 at similar quality & size - Fewer advanced options compared to other encoders ### aomenc (libaom) [aomenc](/tools/software-encoders/aomenc/), based on the libaom library, is the reference encoder for AV1. Developed by the Alliance for Open Media (AOM), it is the benchmark for AV1 encoding quality and compliance. Links: - Wiki page: [aomenc](/tools/software-encoders/aomenc/) - Git repository: https://aomedia.googlesource.com/aom/ 1. **Encoding Quality** - aomenc is widely regarded as the gold standard for AV1 encoding quality. - It often achieves high compression efficiency among AV1 encoders, especially at slower speed settings. - The encoder squeezes out nearly every last bit of efficiency from the AV1 codec, making it ideal for archival purposes or when quality per bit is critical. 2. **Encoding Speed** - aomenc is generally the slowest among major AV1 encoders. - It offers 13 CPU speed levels (0-12), but even at its fastest settings, it's typically slower than other encoders at their slower settings. - The slow speed is often considered a trade-off for its high compression efficiency. 3. **Extensive Options** - As the reference implementation, aomenc offers the most comprehensive encoding options. - It provides fine-grained control over nearly every aspect of the AV1 encoding process. - Advanced users can tweak many parameters to optimize for specific content types or encoding scenarios. 4. **Flexibility** - Being the reference encoder, aomenc produces highly standards-compliant AV1 bitstreams that take advantage of the full arsenal of AV1 features. - It supports 4:2:0 and 4:4:4 chroma subsampling, 8- to 12-bit color depth, and various other advanced features that more specialized encoders like SVT-AV1 do not support. aomenc is ideal for: - Scenarios where achieving the highest possible quality is the primary goal - Archival encoding where compression efficiency is crucial - Research and development in video compression - Encoding projects where encoding time is not a significant constraint Some drawbacks of aomenc include: - Unresponsive development driven by legacy metrics, leading to slower adoption of new techniques and ignoring improvements communicated by people outside the Google development team - Cripplingly difficult to use for beginners, with a culture of cargo-culting settings - Slow encoding speeds compared to other AV1 encoders, which has less of an impact on the quality of the output than it used to compared to maturing encoders like SVT-AV1 ### SVT-AV1-PSY [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/) is a community fork of the SVT-AV1 encoder focused on psychovisual optimizations to enhance perceived visual quality. It aims at closing the distance between SVT-AV1's high speeds and the perceptual quality of aomenc's slow brute force approach. Links: - Wiki page: [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/) - Git repository: https://github.com/gianni-rosato/svt-av1-psy - Documentation: https://github.com/gianni-rosato/svt-av1-psy/blob/master/Docs/PSY-Development.md 1. **Perceptual Optimizations** - SVT-AV1-PSY introduces various psychovisual enhancements to improve the perceived quality of encoded video. - These optimizations often result in output that looks better to the human eye, even if it might not always score as well in objective metrics. 2. **Additional Features** - Introduces new options like variance boost, which can help maintain detail in high-contrast scenes. - Offers alternative curve options for more nuanced control over the encoding process. - Extends the CRF (Constant Rate Factor) range to 70 (from 63 in mainline SVT-AV1), allowing for extremely low-bitrate encodes. - Introduces additional tuning options, including a new "SSIM with Subjective Quality Tuning" mode that can improve perceived quality. 3. **Visual Fidelity Focus** - Aims to produce more visually pleasing results, sometimes at the expense of metric performance. - Includes options like sharpness adjustment and adaptive film grain synthesis which can significantly impact the visual characteristics of the output. - Features modified defaults driven by perceptual quality considerations. 4. **Extended HDR Support** - Includes built-in support for Dolby Vision & HDR10+ encoding. - This makes it particularly useful for encoding HDR content without requiring additional post-processing steps or external tools. 5. **Performance** - Based on SVT-AV1, it retains the performance characteristics of its parent encoder. - Adds super slow presets (-2 and -3) for research purposes and extremely high-quality encoding. These additional presets can be useful for creating reference encodes or applications where encoding time is not a concern. SVT-AV1-PSY is particularly well-suited for: - Encoding scenarios where subjective visual quality is prioritized over pure metric performance - HDR content encoding in Dolby Vision or HDR10+ - Users who want fine-grained control over psychovisual aspects of encoding - Projects that require a balance between the speed of SVT-AV1 and enhanced visual quality - Encoding challenging content with complex textures or high-contrast scenes Some drawbacks are: - Everything that applies to SVT-AV1, including the lack of support for 4:4:4 chroma subsampling and 12-bit color depth that are useful in specific scenarios ### Conclusion While SVT-AV1 is known for being fast, aomenc is renowned for its high-quality output, and rav1e is recognized for its safety and reliability, each encoder has strengths and weaknesses. The best encoder for you will depend on your specific needs and priorities. As this guide is focused on offline encoding, **SVT-AV1-PSY** combines aomenc's traditional perceptual strength with SVT-AV1's speed. Like rav1e, it is easy to use due to strong default settings that prevent cargo culting, and overall it can be considered the best of all three worlds. It is actively developed by a team of responsive community members, and it is a great choice for most users who want a balance between quality per bit, speed, and ease of use. The "best" encoder often depends on your use case, content type, and target audience. It's always worth experimenting with different encoders and settings to find the optimal balance for your needs. Many advanced users even employ multiple encoders in their workflows, choosing the most appropriate tool for each specific task or content type. ## Final Conclusion The AV1 ecosystem has grown to the point where a single comprehensive guide cannot effectively cover the entire extent of the available tools and techniques for AV1 encoding across *every* use case. If you want more detail about a particular tool, where to acquire it, or how to compile an encoder, you can find that information throughout the various wiki entries linked on this page. Reading can get you far, but it is natural to have questions. Please don't hesitate to connect with the team behind the Codec Wiki and many of these tools via our [AV1 for Dummies Discord server](https://discord.gg/bbQD5MjDr3). We're happy to help you with anything you need, and your questions and feedback help the wiki grow and improve. We hope you enjoy your journey into AV1 encoding, and we wish you the best of luck in your encoding endeavors! --- # Codec Wiki: One Year Later Canonical URL: https://encode.wiki/articles/codec-wiki-one-year-later/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 73b6e83b88c544a1824edd1a4ea984b536d744779bd6d1af82c97b4b9086d1d9 Around a month ago, Codec Wiki celebrates it's very first anniversary as it officially hits one year old (Wow, that's a very long time!). First and foremost, I apologize for this delayed celebration post, I have been very busy for the past month, y'know, real life things. Second of all, I would like to say **THANK YOU** for everyone who helped build this wiki from the ground up and writing down the entries, you know who you are! ## Stats Thanks to everyone's efforts, over the course of 12 months Codec Wiki has grown into the (almost) go-to documentation and guide for newcomers just starting out in the quirky and wild world of multimedia encoding. So here are some pretty graphs to look at our progress so far! ### Plausible As you may know, we ran our own Plausible Analytics instance: self-hosted, open source, and privacy-respecting analytics. It was hosted in Singapore and its former public dashboard is no longer available. Plausible Analytics 7.7K unique total visitors for a whole year may not be much but it's a great start! Let's not forget those who use uBlock Origin or similar to block the analytics script. So the visitor count could be two or three times more! ### Google Search Console For those who are unfamiliar, Google Search Console is used for viewing well.. Google search data for your website. This data is unfortunately private and only the owner and those invited can see the data, no setting available for public view. Probably for the better. Google Search Console



And for our top 10 search queries in the last 28 days: Search Queries page 1 Search Queries page 2



With pretty much zero competition in this niche of a market, it skyrocketed Codec Wiki to the top of search results for everything multimedia related 😅. ## A Brief History Around a year ago, I started work on an unofficial "wiki" for all things encoding, it initially started as a half-serious joke because I was annoyed there were no good documentation available anywhere on the internet on the real application and usage of encoders such as "Should I use tune SSIM in x encoder?". With the search results you'll be presented with being one or more of these: - Doesn't exist. - Mentioned in the FFmpeg mailing list but nothing else. - PDF research papers on IEEE and similar. - Outdated information on Doom9 or Multimedia Wiki. - "The documentation are in the code blocks". - Link rot. - Abandoned, or mostly focused on other topics. - Focused on a specific type of storytelling medium. - Actual good explanation in an old forgotten site (rare). Some or maybe even most of these results still appear up to this day. Frustrated with how scarce information is available on the open internet and combined with the fact that most, if not all of them only exist in non-search-engine-reachable private chat communities such as Discord where you'll have to join and scroll around older conversations. Prone to data erasure as Discord can nuke these servers for whatever reason they please and years of "documentation" is lost in an instant. It is simply not a good platform for a plethora of valuable information. **That's why I took initiative.** ## Bumps in the Road There were also generally a lot of distrust and pushbacks when this project first began, particularly from those who are more experienced with encoding. Which is understandable because everyone who attempted before burned out and never finished it. But Codec Wiki stood the test of time with constant updates through active collaboration of everyone involved, nowadays those voices have mostly subsided or even joined the project themselves. And I am grateful for that. ## Closing Statement Once again, **THANK YOU VERY MUCH** for those who are involved in this project and making the once pipe dream of a one-stop-shop user-friendly documentation for encoding into something that's possible. Let's continue documenting the most crucial yet invisible driving force of internet traffic, pages by pages. Together we are strong, divided we burn out.

> "There are times when a critic truly risks something, and that is the discovery and defense of the *new*. **The world is often unkind to new talent, new creations. The new needs friends**." > > — Anton Ego --- # Embedding the Un-Embeddable Canonical URL: https://encode.wiki/articles/embedding-the-un-embeddable/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 586932761bd4c04cba6aa9d8ded5624f1ce488f76b849e1b6b8ee17c4e54d113 ![Feature image](/img/discord-embed-blog-image.webp) **A 567.14 MB, 12 min 11 s, 2K (2,048 x 858), VP9 + Opus, 6.51 Mbps average, Blender short film "Cosmos Laundromat"** ## A Scenario While chatting in your favorite Discord servers & group chats, you may see a friend send a weird link. You might even consider it suspicious on first glance. It is a video featuring an image of a movie poster with a play button that is almost begging to be clicked. Naturally, you click it. It loads for a second, and to your surprise it is a full-length, 90-minute (sometimes even two hour)-long unauthorized copy of a movie. If you don't know exactly what is going on, you probably sit there dumbfounded as a pixel perfect HD movie plays back. You may have expected a stereotypically muddy, blocky, laggy shitpost, but this has defied your expectations. ![stolen.shoes](/img/stolenshoes-puss.webp) The truth is, there are *multiple* site that do this. Currently, there are five at the time of writing. Below is a list the ones I am currently familiar with: - https://stolen.shoes - https://discord.nfp.is - https://embeds.video - https://x266.mov/discord-embed - https://autocompressor.net/av1 The big question is, **how do they work?** Let's get to dissecting. ## But First, a Quick Disclosure The Codec Wiki unequivocally condemns any form of piracy, including the unauthorized distribution of copyrighted content. This blog post is intended to educate & inform. You may not use the tools discussed to infringe upon the intellectual property rights of content creators without serious legal risk. We encourage our readers to respect copyright laws & use the tools we discuss here appropriately. ## How it Works The entire scheme is actually very simple, as it is all just HTML meta tags (If you are familiar with web development, this is all a walk in the park). The technology's inner working can be divided into two distinct parts. First, let's see how it works on the website's end. ### The Website's End If you view each website's source, you will find this specific line in each one but they may have a different order: ```html ``` These are the `head` parts of HTML, which dictate metadata for the document itself such as what the website title/name is, cosmetic embed, defining the site's icon, etc. They are usually found in between the `` and `` tags. Here's an example of a static HTML site serving one specific video: ```html some embed site

Hi

Just your friendly neighborhood video embed site

``` < br /> These interactive sites usually deploy a live script, like a Javascript framework. Examples are NodeJS, ExpressJS, Svelte, etc. These are used to parse video and thumbnails realtime so they can be embedded on Discord (or potentially other platforms). ### Discord's End Traditionally, Discord's media embedder will impose it's own video embed size limit (50 MiB) when a user sends a direct video link as usual. But in this case Discord will embed the thumbnail first, not the video. You could say the link "tricks" Discord by showing a "false face" first. ## Strengths & Limitations After a combination of countless hours of observation, rigorous testing throughout the period of a year, and conversations with the sites' creators, the current strengths & limitations of this exploit are enumerated below. #### Strengths - You can embed non-web compatible codecs such as [HEVC](/codecs/video/hevc/) in [MP4/MOV](/start-here/terminology/#mp4--m4v), but the user must be using a compatible browser. [Thorium](https://thorium.rocks) or Safari version 13 or greater will work for HEVC playback. - There is no maximum size. You could embed a video the size of a raw Bluray, although I do not condone this unless you have the necessary legal permissions to do so or you're uploading a Creative Commons licensed movie like Big Buck Bunny while adhering to the restrictions of the applicable Creative Commons license. This also means you can send high bitrate gaming clips to your friends without any restrictions, assuming you already have a place to upload them. #### Limitations - You can only use [hotlinks](https://simple.wikipedia.org/wiki/Hotlinking), which means direct linking to the video itself ending in the appropriate file extension such as `.mp4`. Cloud services like Google Drive or OneDrive will not work for storage. - You cannot use Discord's CDN (cdn.discordapp.com) as the video source. I assume this is because of Discord's proxy blocking embeds over 50 MiB, but **only discord.nfp.is can do this**, as it **proxies cdn.discordapp.com** itself. - You cannot embed videos in any resolutions higher than 3840 x 2160, Discord imposes a hard limit for this on all video after it was discovered that some videos could play normally but then be maliciously scaled to ridiculous resolutions during playback to crash Discord. ## Differences between Sites As mentioned before, there are five known sites at the time of writing. They all serve the same function, but one may interest you more than another due to slight differences in features & functionality. Here are the sites, each with one noteworthy special benefit: - https://stolen.shoes - Recognition, as it is the OG. - https://discord.nfp.is - You can use Discord CDN as video source. - https://embeds.video - Immediately input video source into the URL (`https://embeds.video/https://example.com/v/video.mp4`) - https://x266.mov/discord-embed - Attractive domain, simple layout. - https://autocompressor.net/av1 - Lots of info dump, pretty advanced features. That concludes the technical overview! Next, let's cover the history of this exploit. ## The Lore ### Dwayne In around April of 2022, a Reddit user going by the name of u/CreativeGamer03 [posted a video on r/discordapp](https://www.reddit.com/r/discordapp/comments/u96kky/someone_sent_this_in_the_memes_channel_and_bruh) of a link where a GIF of Dwayne "The Rock" Johnson plays caption with "Is this a GIF or is it a video?" When played, a low-quality music video of Rick Astley's "Never Gonna Give You Up" plays. The link used, `archuser.de/the-rock`, has since been removed. ### Discovery On 23rd June 2022, a Discord user *Clybius* on the AV1 Community server asked people for [VP9](/codecs/video/vp9/) or [H.264](/codecs/video/avc/) videos that were over 100 MB in size. At the time the current 500 MB nitro tier did not exist. They then decided to use a 59 minute 1080p sample video of nature scenery from around the world with a thumbnail featuring a GIF of a waterfall to test the exploit. It worked. He tried shortly afterward with [AV1](/codecs/video/av1/). Eureka, it also worked: ![AV1](/img/clybius-av1.webp) Clybius confirmed that this could be patched if discovered. He cites having had the idea from the Dwayne Johnson example above, but forgetting about it for a couple of months. So, it seems this entire concept stemmed from a silly rickroll. ![Dwayne](/img/clybius-dwayne.webp) ### The Experiments & Interactive Site After the discovery of AV1 embedding, experimentation brought about the discovery that *any* video codec will work as long as the user can decode/play the codec and the container/extension is an MP4, MOV, or WebM. These are all traditionally web-compatible containers. If you're interested in learning about containers, please see the [Containers](/start-here/terminology/#container) section on the [Terminology](/start-here/terminology/) page. This applies to HEVC, ProRes, [xHE-AAC](/codecs/audio/aac/#xhe-aac), and other bizarre codecs that are rarely seen on the Web. While experimentating, Clybius converted one their idle domains `stolen.shoes` into an interactive embedder that provided a textbox for a video URL, a thumbnail URL, a width value, & a height value for the desired video. This would be the first website for Discord embedding. ### Virality It's not long before people outside of the AV1 Community discovered `stolen.shoes`, and its popularity increased rapidly. Its use usually involved the illicit distribution of full-length, unauthorized copies of movies; this sometimes happened very shortly after some movies were released. There were a couple notable instances of this happenening that caused quite the stir online each time. - The first instance featured the DreamWorks sequel of "Puss in Boots (2011)", "Puss in Boots: The Last Wish (2022)". A 1080p video sourced from a streaming site was the first wake up call that attracted attention to the existence of these embed sites. This example used `stolen.shoes`. ![puss](/img/stolenshoes-puss.webp) - The second instance was when highly-anticipated animated film "The Super Mario Bros. Movie (2023)" produced by Illumination, Universal Studios, and Nintendo was spread around Discord. It was first spotted as a Cam (A camera recording by someone in theaters), then as it went out on streaming services a different link appeared but spread faster and with upgraded 1080p quality. Both used `stolen.shoes` as the embed site. ![mario](/img/stolenshoes-mario.webp) - The third instance is very recent as of the day this was posted. A streaming-service sourced "Five Nights at Freddy's (2023)" was spread around since the movie released both in theaters and streaming service (Peacock) day one, and it gained steam extremely fast as most people had not seen it yet. Currently, this illegal novelty is gaining [hundreds of upvotes within the r/discordapp subreddit](https://www.reddit.com/r/discordapp/comments/17hx45y/is_discordnfp_an_ip_grabber/). The copy seems to be a compressed 720p encode. This example used `discord.nfp.is`. ![fnaf](/img/discordnfpis-fnaf.webp) Note the ones listed here are the ones that I saw become extremely popular. There may be lesser known links that have been spread around privately or just did not cause enough noise for me to notice. Some less popular examples I've noticed, featuring more illicit copyrighted content distribution: - Top Gun Maverick (2022) - The SpongeBob trilogy (2005/2015/2020) - Spider-Man: Across the Spider-Verse (2023) ## Closing The ability to embed unusually large videos on Discord has enabled both positive and negative use cases. On the one hand, it allows high-quality content to be shared easily among friends. However, it has also facilitated mass copyright infringement by empowering virtually anyone with a Discord accound to freely spread pirated movies. While this is fascinating from a technical perspective, embedding techniques like these tread a fine ethical line. As with anything, it is important to be mindful of how our actions affect others, and I should remind everyone that content creators deserve to be compensated for their work. As users, we should support them by accessing their content via legitimate platforms. It is hard to say how long this exploit will continue to be usable. Instead of enabling piracy, which may cause Discord to be more likely to patch this exploit if they see it as a serious threat, let's instead use these capabilities responsibly to share our own creations, gaming highlights, and other media which we can share legally. Given some thoughtfulness, perhaps we can find a fair balance between respecting copyright law and appeasing Discord's sensibilities while allowing some creative flexibility on the platform. Thank you for reading this blog post, I hope you learned something! --- # Reducing Image Load Online Canonical URL: https://encode.wiki/articles/site-optimization/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: b673a1da8857b20dd1b2fafa9a1ab9083fa63b82a62d123597c2ce8f40d5f482 A big part of understanding any multimedia codec technology is knowing the application for such technology. For images, a big use case is web delivery. Compared to other multimedia, images are incredibly popular on the Web & knowing how to serve them properly can be a massive boon to your website's traffic as well as less of a headache for users on slower connections or who are under bandwidth constraints. The most disappointing part is that images are often poorly done on the web; all too frequently will you run into a site serving massive photographic PNGs for no reason, or photography sites serving photographs fresh out of the editing software with no thought put into their final delivery. A little effort, patience, & knowledge will go a long way toward improving the user experience for individuals using your site, & this article will illustrate some of the basics. > **Caution — These instructions are for *photographic* images; other kinds of images, like non-photographic, artwork, pixel art, etc. should likely be handled differently..** > > > **Danger — Many images won't load properly unless your browser supports JXL, AVIF, & proper ICCv2 color management. This is for demonstration purposes only & shouldn't represent an actual common website experience. If you're curious anyway, the following browsers can display the contents of this page perfectly:.** > > - [Thorium](https://thorium.rocks/) | *Linux, [macOS](https://github.com/Alex313031/Thorium-Special/releases), [Windows](https://github.com/Alex313031/thorium/releases/), [Android](https://github.com/Alex313031/Thorium-Special/releases)* > - [Waterfox](https://www.waterfox.net/) | *[Linux](https://flathub.org/apps/net.waterfox.waterfox), [macOS](https://www.waterfox.net/download/), [Windows](https://www.waterfox.net/download/)* > - [Mercury](https://thorium.rocks/mercury) | *[Linux](https://github.com/Alex313031/Mercury/releases), [Windows](https://github.com/Alex313031/Mercury/releases)* ## Fire & Forget First, we'll illustrate what *not* to do, which is fortunately not incredibly difficult to avoid. Taking an image straight out of your editing software at a massive size will often bloat the size & resolution to something that isn't generally usable for a website regardless of the codec you're using & its quality per bit. It can be argued there are specific use cases that demand incredible resolution & fidelity coexist on the Web, but we won't be covering those here. Here's an example of a bloated image: *exported straight from Darktable at JPEG q90, with no scaling* ![bloated_jpeg](/img/_DSC8466.jpg) **2.2 MB** ## Massive Improvement The easiest way to have a large improvement without doing much work is to simply resize the image before serving it. Even if you exported a lossy JPEG, resizing should remove a lot of artifacts. The way to perceive a worst-case for an image's size on a site is to inspect the image element's width & height, which should give us an estimate of how large we should make our image. Any larger than this value is unreasonable since we're overfilling the element's size for no reason & the image is being scaled down anyway. ![box-size-mac](/img/box-size-mac.avif) *Inspect Element in Firefox. The Mac used to take this screenshot has a relatively high display resolution of 2560x1664. Because Macs scale things differently, we're probably going to want to double the horizontal resolution here.* The width is the most important value here, so our new image is going to be exported with a width of 1699 pixels. This new image, encoded at JPEG q90 with `cjpegli`, looks like this: ![smaller_jpeg](/img/_DSC8466-smaller.jpg) Obviously, there's lost fidelity compared to the original, but considering this is *so much smaller*, it is worth the trade-off for many. It is also worth noting we are using an improved jpeg encoder in the form of `cjpegli`, although that is secondary to the resize. If it doesn't look as good as you want it to, you can always scale the resolution up a bit, though currently, it looks plenty passable for its size. 2.2 MB -> **233 kB** > **Chart — Page-weight reduction in the article example.** Resizing and re-encoding reduces the example image to roughly one tenth of its original transfer size. Data: Original: 2200 kB; Optimized: 233 kB. ### Lazy Loading A bonus tip is to add the `loading="lazy"` attribute to your picture tag to allow the image to load only when scrolled to by a user. This doesn't save bandwidth, but it improves the user experience by loading images further down the page only when necessary. An example may look like this: ```html alt text ``` ## New Codecs If you desire further improvement, it may be time to consider using a newer codec like [AVIF](/codecs/images/avif/) or [JPEG-XL](/codecs/images/jxl/). These options will compress far more effectively than JPEG, with the only trade-off being browser support. We're not going to consider [WebP](/docs/images/WebP) or [HEIC](/docs/images/HEIC), since WebP is not competitive enough with JPEG for photographic imagery (often being worse) & HEIC has been superseded by AVIF - which sees greater support anyhow - & is not royalty free, effectively preventing widespread Web adoption forever. Again, we're just considering *lossy* compression for *photographic* images; it is a different story with WebP elsewhere, as it performs well on non-photographic content & is almost always better than PNG for 8-bit lossless compression. So, we are left with JXL & AVIF for now. ### Fallbacks AVIF sees widespread support, but JPEG-XL isn't quite there yet with Web support as Google continues to push AVIF (it is debatable if it ever will be outside the Apple ecosystem). Even with AVIF, adoption isn't remotely close to JPEG, so it is worth providing a fallback. This can look like the following example: ```html alt text ``` Here is a JXL falling back to an AVIF falling back to a WebP falling back to a JPEG. Pretty intense to have this many fallbacks unless you're really after the ultimate compression ratio, but it is certainly an option. AVIF & JPEG alone will probably be enough for most. ### Compression Efficacy Let's look at how our image examples compare to the original with our new codec selection. We'll be aiming for high visual fidelity, so around the same quality as our initial JPEG encoded with `cjpegli` (which scores ~`83.01` with the [SSIMULACRA2](/docs/metrics/SSIMULACRA2) visual fidelity metric). ![smaller_jxl](/img/_DSC8466-smaller.jxl) **137.0 kB** *JPEG-XL image, encoded with `cjxl lossless.png out.jxl -d 1.49 -e 9`. Score: ~`83.04`* *3.06s user time* ![smaller_avif](/img/_DSC8466-smaller.avif) **124.8 kB** *AVIF image, encoded with `avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=16 -a tune=ssim lossless.png out.avif`. Score: ~`83.03`* *7.54s user time* JXL also supports lossless transcoding of JPEG images. This means every pixel is identical, the image just has a smaller filesize than the original JPEG; if you can use JXL, you can transcode existing JPEGs losslessly on your site & save some bandwidth that way. The JPEG transcode below gives a higher SSIMULACRA2 score than the original for some reason, but I'll chalk that up to a decoding inconsistency between how the `ssimulacra2` program decodes JPEG & JXL. Either way, the scores are fairly close. ![smaller_jxl_jpeg-recomp](/img/_DSC8466-smaller-recomp.jxl) **189.4 kB** *JPEG-XL image from JPEG, encoded with `cjxl input.jpg input-recomp.jxl -d 0.0 -e 9 --brotli_effort=11`. Score: ~`84.92` (???)* *0.67s user time* The final trick we can use, while not a new codec at all, still increases quality per bit. Encoding an XYB JPEG with `cjpegli` encodes with the perceptual XYB colorspace using an ICC profile to modify the original JPEG colors, avoiding JPEG's normal YCbCr which isn't perceptually optimized for the human visual system. Using XYB, we can afford identical quality with less bitrate than normal JPEG. This has universal compatibility, but not every application understands how to handle the XYB color profile (although color-managed modern browsers should be fine). ![smaller_jpeg_xyb](/img/_DSC8466-smaller-xyb.jpg) **208.3 kB** *XYB JPEG, encoded with `cjpegli lossless.png out.jpg --xyb -d 1.155`. Score: ~`83.04`* *0.10s user time* In this particular instance, AVIF seems to be the overall winner. This isn't always the case due to JXL's superiority at higher fidelity & with more detailed images, but according to SSIMULACRA2, AVIF has the best quality per bit with this image. You can use your own eyes to further clarify your choice, though. It is worth mentioning that as these were encoded from a 16-bit source PNG, the JXL image is the only one that maintains the full original bit depth, & AVIF isn't fast to encode. ## Responsive Images Displaying an image that is too large for a viewport is a waste of bandwidth, & displaying an image that's too small for the viewport leaves fidelity to be desired. Luckily, we have the [Responsive Image Linter](https://ausi.github.io/respimagelint/) that can help us figure out which image sizes we should be using. ![responsive_image_linter](/img/responsive_image_linter.avif) In our fire & forget example, we see that we are serving an image that is far too large. We already know that, but now we can see that given various viewport sizes we could be serving images that have respective widths of 270px, 958px, 1350px, 1660px, & 1916px to optimize for delivery to a variety of different devices. Here's how we'd write that in HTML: ```html alt text ``` It is worth noting that this example above & the example below aren't perfect implementations of a responsive image given the conditions of this site, but the general concept still applies. Some things to note: - `srcset` = the images available to your browser to serve, & their respective widths - `sizes` = the conditions given to the browser explaining under what conditions should it serve which image - `(min-width: XXXpx) YYYpx` = Given the viewport is at least XXX wide, serve an image of YYY horizontal resolution. The browser will pick an image from srcset that is CSS pixels \* display scaling. - `calc(100vw - 24px)` = Usually preceded by a (min-width) condition. Specifies a value the browser should calculate on its own to pick the closest option from the srcset. Let's say we have `(min-width: 997px) calc(75vw - 257px)`. This means given the viewport is at least 997px wide, calculate 0.75 \* the current viewport resolution - 257 to find the closest image in the srcset to fit the number of pixel specified. alt text That's all! Massive thanks to Auto-Rez Media Technologies for the inspiration behind this article & explicit permission to use their [Reduce Your Page's Image Load](https://autocompressor.net/blog/reduce-image-load) blog post when writing this entry. I have [confirmed](https://autumn.revolt.chat/attachments/GtFGuwNfeRdcwUN0MWzhDCAiiadWOk88XXC3pQv6RI) with their leadership that this wiki entry can be safely licensed under CC BY-SA 4.0. --- # Encoding Animation with SVT-AV1: A Deep Dive Canonical URL: https://encode.wiki/articles/svt-av1-deep-dive/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 8ec837249aa678752f367a55d42085da4fa2acacca0740072e1146dd1489afa5 # Introduction This blog post is based on a series of visual quality benchmarks with SSIMULACRA2 and speed benchmarks of SVT-AV1 1.8.0 on a corpus of animated clips. The resources available will range from ***graphs*** to ~~**image comparisons**~~ (WIP). The ***former*** has the advantage of being easily understandable, showcasing pure efficiency comparisons between encoder parameters using metrics as the reference, while the **latter** are image samples from the encoded files during the tests that enable you to check quality for yourself and add another layer of subjective interpretation to these comparisons. The testing methodology involves using relatively short video samples from a wide range of modern anime genre, which have been either losslessly encoded with `x264 --qp 0` for ease of use or losslessly cut from their source. These lossless files are then pipped into SvtAv1EncApp *directly*, meaning we are measuring the performance of a single encoder instance and not leveraging chunked encoding like any actual final AV1 encoding pipeline should. Once an encode is done, SSIMULACRA2 scores are calculated using the [Zig implementation](https://github.com/dnjulek/vapoursynth-ssimulacra2) and lots of useful data are aggregated to make the graphs for this benchmark, including encoding time, encode size (bitrate), and SSIMULACRA2 scores. Bits per pixel scores (BPP) are calculated so that the `Metric / BPP` graphs may represent the closest we have to real efficiency. The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy. SvtAv1EncApp was compiled directly from the [v1.8.0 source code](https://gitlab.com/AOMediaCodec/SVT-AV1/-/releases/v1.8.0) using the provided `Build/linux/build.sh` script, Clang 16.0.6, and Profile-Guided Optimization (PGO). The testing machine is comprised of an i7 8750H running at 35W with 16GB of 2666MHz DDR4 RAM in Arch Linux with kernel 6.6.6 and the performance governor enabled. All encodes have been made in the same session without rebooting. This testing was conducted within the [AV1 Weeb Edition](https://discord.gg/83dRFDFDp7) Discord server, which is focused on encoding animated content in AV1. ## Samples The samples are as follows: - 11s `Blame!` clip which sports 3DCG action with lots of grain, effects and high-contrast elements. Most complex source of this set. - 13s `Blue Lock` clip which sports rapid camera movements and rotations + high-contrast elements. - 15s `Fate/Grand Order: Babylonia` relatively slow-paced clip with lots of effects still. Easiest source of this set but easy sources still give interesting data. - 22s `Jigokuraku (Hell's Paradise)` flashback clip with huge static grain in a very dark scenery and some action. - 14s `Kaguya-sama` opening sequence with lots of effects and fast change of scenery. The resolution of every clip is 1080p, except for the first one which is 1920x804. > **All clips have been encoded in a wide quality range, from `--crf 8` to `--crf 43`.** Without further ado, let's start with the first comparisons! ## Presets comparisons (-1 -> 13) **In the following graphs, you may find comparisons between all SVT-AV1 presets, ranging from the slowest `--preset -1` to the fastest `--preset 13`.** *Please remember that these two extreme presets are meant for development purposes and as such should not be used in normal encoding conditions. You will soon understand why.* `--preset X` is the only parameter used here, in conjunction with the CRF values. That means everything else is default. The defaults worth mentioning are: - `--tune 1`: tune PSNR - `--aq-mode 2`: variance deltaq - `--enable-qm 0`: quantisation matrices disabled - `--irefresh-type 2`: closed GOP - `--enable-tf 1`: temporal filtering enabled And more, like CDEF and restoration enabled, overlays and film-grain disabled... - First of all, here are the efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). This could be too much information. - Now the same graphs but focusing on the "high quality" range (CRF8 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Same again but without presets 9 to 13 for better clarity: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Now for the "low quality" range (CRF28 -> 43): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Same but without presets 9 to 13 for better clarity: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - **Let's now see speed comparisons between all presets:** ![speed_overall](https://raw.githubusercontent.com/av1-community-contributors/images/main/svt-trix-blogpost/speed_overall.webp) As we can see, preset -1 is so abysmally slow it makes the graph unusable (BTW, notice the `1e6` in the lower right corner, it is obviously not encoding at 1 to 4 ms, but at 1 000 000 to 4 000 000 ms) - Here is what it looks like with a logarithmic scale: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - **Now the speed graphs but with SSIMU2 on the y-axis instead of BPP: (logarithmic scale)** > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Here are speeds graphs for preset 1 to 6 with a linear scale: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). One interpretation we can have is that **presets 2 to 4** have actually pretty close scores (pretty much the same at HQ, 2 points at max in the low quality range) but **preset 2** is **2x slower than preset 4**. The quality gap between **preset 2** and **preset 1** is even narrower but the speed penalty is also ~2x. That makes it hard to recommend **preset 1**, while **preset 3** is in a strange middle ground between **2** and **4** where it makes little sense to choose it over those two: it's better to choose **preset 2** for the higher efficiency at 1.5x slower speeds, or **preset 4** for the 1.33 to 1.5x higher speeds. At least the option exists. A similar observation can be made between **preset 5** and **6**. They can be so close in scores in many samples (especially at HQ) while also being close in speeds that **preset 5** becomes rather obsolete most of the time. ### TLDR **Clear quality gains can be observed as we decrease presets, until the very last one, however the effectiveness of dropping presets becomes less and less impressive the higher in quality you go.** - For instance, in worst-case scenario, we observe that (for the CRF23 to CRF8 range), __**preset 4**__ only loses at maximum **2** SSIMU2 **points** compared to __**preset -1**__ while being 50-60x faster. Though to be fair, the speed loss from __**preset 4**__ to __**preset 0**__ is "only" 5-10x for a maximum SSIMU2 difference close to **1.5 point**. - From CRF43 to CRF28, the difference between __**preset 4**__ and __**preset -1**__ can be as much as **5** SSIMU2 **points**, so lower presets become more attractive. ## Tunes comparisons **In the following graphs, you may find comparisons between SVT-AV1 tunes, from the default `--tune 1` (PSNR) to the other two tunes: `--tune 0` (VQ) and `--tune 2` (SSIM).** Except for the tunes, `--preset 4` is set due to its good balance of quality and speed, in conjunction with the CRF values. That means everything else is default. The defaults have been mentioned earlier above. - **Let's compare the efficiency of every tunes:** > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Now let's focus on the "high quality" range (CRF8 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - And the "low quality" range (CRF28 -> 43): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - And here is the speed difference: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Graphs comparing the tunes individually between each others will be made available soon. - The image comparisons will make the conclusion quite more nuanced, stay tuned (heheh) for that. ### TLDR **SSIMU2 favors __tune 1 and 2__ above __tune 0__. At high quality, __tune 1 and 2__ are matched, while at low quality __tune 2__ takes the edge. __Tune 0__ is sometimes a match for the other two on certain clips and other times fare pretty badly.** **For some reason, __tune 0__ is quite a bit faster now, compared to 1 and 2 which are basically the same.** Keep in mind that I have observed multiple times in the past that __tune 0__ kept more fine detail and was sharper than the other tunes, at the expense of potential artifacting, like ringing and distortion. It may very well explain why the metric doesn't like its results. The image comparisons may give a different interpretation than what we concluded here, please stay *tuned* for these subjective comparisons. ## Parameters comparisons **In the following graphs, you may find comparisons between many SVT-AV1 parameters.** *Additional graphs focusing on the high and low qualities will be made available later down the line* `--preset 4` is used here due to its good balance of quality and speed, in conjunction with the CRF values. That means everything else is default. The defaults have been mentioned earlier above. ### `--tile-rows 1 --tile-columns 1` vs default `--tile-rows 0 --tile-columns 0` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __tiles__ here are both slightly harmful and slower. ### `--aq-mode 0` vs default `--aq-mode 2` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Except for the Jigokuraku clip, __aq-mode 0__ is harmful in the eyes of SSIMU2, while being slower at low CRF levels, and sometimes a match or faster at high CRF levels. ### `--aq-mode 1` vs default `--aq-mode 2` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __aq-mode 1__ fares closer to __aq-mode 2__ than __aq-mode 0__ did, both in quality and speed, but is still overall inferior according to SSIMU2 ### `--enable-cdef 0` vs default `--enable-cdef 1` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > According to SSIMU2, disabling CDEF barely impact efficiency. But as its a pretty demanding tool, there's a slight speed benefit of having it disabled too. I advise you to take these results with a grain of salt until the image comparisons, because in anime particularly, CDEF *can* be beneficial for the line-art. ### `--enable-dg 0` vs default `--enable-dg 1` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __Dynamic GoP control__ yields bit-perfect results in all clips except for Blue Lock and Jigokuraku. There is no speed benefit to disabling it except in clips where it is in use. Let's not jump to conclusions too easily, the image comparisons will tell if it's "safe" to keep the setting disabled at all times or not. ### `--enable-dlf 0` vs default `--enable-dlf 1` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __Deblocking loop filter__ can be slightly beneficial in some scenarios. In reverse, it is never harmful, so it is recommended to keep it default. ### `--fast-decode 1` vs default `--fast-decode 0` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __fast-decode 1__ is pretty harmful in the Fate clip and slightly harmful in the rest. There is a speed benefit of enabling it though. ### `--irefresh-type 1` vs default `--irefresh-type 2` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Finally something interesting to analyse! - __irefresh-type 1__ is either a match or beneficial compared to __irefresh-type 2__ at high CRF levels. - __irefresh-type 1__ either wins or lose to __irefresh-type 2__ at low CRF levels depending on the clip. As such, it is NOT recommended to blindly set __irefresh-type__ to __1__ at "high quality" as it might be harmful depending on the content. - __irefresh-type 1__ is always slower compared to __irefresh-type 2__, so I might argue it is more safe to leave __irefresh-type__ default at "high quality" than not. - According to the content type of those clips, it appears that __irefresh-type 1__ may benefit *extremely grainy* content while default __irefresh-type 2__ is better suited for *cleanish* content. This needs to be confirmed with *moar* testing though. ### `--lookahead 0` vs default `--lookahead -1` (auto) - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). ### `--lookahead 60` vs default `--lookahead -1` (auto) - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). ### `--lookahead 120` (max) vs default `--lookahead -1` (auto) - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __lookahead__ seems to behave strangely when set... - __lookahead 0__ shifts quality around a lot and it is difficult to draw conclusions but there's a clear speed drawback of disabling lookahead. - __lookahead 60__ is perfectly bit-perfect to every clips EXCEPT for some damn reason in Jigokuraku at CRF23 - __lookahead 120__ is somehow also bit-perfect, but this time in every clips and every CRF levels. Both 60 and 120 don't see much speed differences. > Soooo.... this behavior is so odd I don't advise to set any lookahead value. Let the encoder decide. ### `--enable-overlays 1` vs default `--enable-overlays 0` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > __overlays__ do not seem to either improve efficiency or performance. ### `--enable-qm 1` vs default `--enable-qm 0` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Enabling __quantization matrices__ alone increase efficiency at "high quality" with no real speed impact. ### `--enable-qm 1 --qm-min 0` vs `--enable-qm 1` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Setting __qm-min__ to __0__ on top of enabling __quantization matrices__ can be beneficial in some clips at no added compute time. I will re-tests many QMs ranges in the future, but I doubt it changed much from v1.7.0 where 0 was the most appropriate choice for most content. ### `--enable-restoration 0` vs default `--enable-restoration 1` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Even though the efficiencies are very similar, nothing is bit-perfect here. So according to SSIMU2, the __loop restoration filter__ isn't necessarily useful. However, just like CDEF, it's a pretty demanding tool, so disabling it yields some performance improvements. Let's take these with a grain of salt until the image comparisons. ### `--scm 0` vs default `--scm 2` (content adaptive) In all the clips, the results are bit-perfect and there is no notable performance difference. ### `--scm 1` vs default `--scm 2` (content adaptive) - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Interestingly enough, __screen content tools__ seem to improve efficiency according to SSIMU2, at the cost of a huge performance regression. After the image comparisons are published, I will conduct additional testing on this. ### `--enable-tf 0` vs default `--enable-tf 1` - Efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-deep-dive.json). > Disabled __temporal filtering__ *can* sometimes improve efficiency slightly at "high quality", however it is very much clip dependent. It also improves performance slightly. The image comparisons will give another perspective to these results. ### `--enable-tpl-la 0` vs default `--enable-tpl-la 1` In all the clips, the results are bit-perfect and there is no notable performance difference. ### `superres`: Kinda lazy to make, share and comment so much graphs for something that can be told in two lines... All superres variants are freaking useless as they: - do not improve efficiency - decrease encoding speeds - decrease decoding speeds when either bitrate or quality normalized. ### __Early TLDR on parameters results:__ ***For a previous test with SVT-AV1 1.7.0 I did all parameters tests with `--preset 6 --tune 2`, now I did them with `--preset 4 --tune 1`. Mostly everything tested on v1.7.0 still stands today, but now we have more valuable data. Images comparisons are still needed to give more context to some results, so the conclusion presented here remains early as they are 100% based on SSIMU2 results and will require more analysis down the line.*** **Here is a quick run down of how each parameter affect encoding:** - `--tile-rows --tile-columns` should never be used (except for decreasing decode complexity) - `--aq-mode 2` is the most efficient / fastest - `--enable-cdef 0` *might* improve performance at almost no efficiency loss *(needs more thorough testing)* - `--enable-dg` and `--enable-dlf` barely do anything - `--fast-decode 1` decreases efficiency, improves encoding times, and the decoding gains still need to be determined. - `--irefresh-type` should be kept default at high CRF values and for cleanish content at low CRF values, but can be set to 1 at low CRF values for extremely grainy content - `--lookahead` should be kept default - `--enable-overlays 1` does not improve efficiency, slight speed regression as well - `--enable-qm 1 --qm-min 0` should be set for increased efficiency especially at low CRF values at no perf cost - `--enable-restoration` barely does anything but disabling yields better performance *(needs visual confirmation)* - `--scm 1` screen content tools can improve efficiency with a big performance trade-off *(needs more thorough testing)* - `--enable-tf 0` is a mixed bag efficiency wise but improves performance - `superres & resize` please don't. ## Conclusion The extensive benchmarking reveals that obviously the slower presets provide better efficiency, with diminishing returns past preset 2. However, preset 4 provides a good balance of quality and speed for most use cases. Presets 6 and 8 are good options for the people that find 4 to be too slow for their liking. The default settings tend to provide good results, but some tweaks like enabling quantization matrices can further optimize efficiency. Parameters like tile encoding and super resolution modes are not beneficial. Overall, SVT-AV1 continues to be a competitive option for encoding animation in AV1, with its speed being a notable strength. Further testing with more content samples would help solidify these findings. The image comparisons will also provide additional subjective evaluation to complement the objective metrics used here, and we can discover the potential usefulness of Tune 0 that may betray what the metrics suggest. Overall, this comprehensive deep dive should give encoders a helpful starting point for choosing settings when encoding animation with the latest SVT-AV1 1.8.0. Thanks for reading! {} --- # Deep Dive into SVT-AV1's Evolution (Part 1): Presets Analysis from v2.0 to v3.0 Canonical URL: https://encode.wiki/articles/svt-av1-fourth-deep-dive-p1/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: b5faa036017e5c781c2459c99f31f92b12ca39a11f752a8d69fe8a63a1b9349b # Introduction It's been almost a year since SVT-AV1 v2.0.0 dropped in March 2024, and we finally got v3.0.0 in late February of this year. Minor versions v3.0.1 and v3.0.2 came along afterward with some bug fixes and ARM SIMD improvements, but they didn't meaningfully alter encoding results. So what's actually different between these versions? I've been wanting to run tests across all the major releases from v2.0.0 to v3.0.x to see how the speed vs quality trade-offs have evolved this past year. Using SSIMULACRA2, Butteraugli, XPSNR, and VMAF (plus some methodology tweaks I'll get into), I'll break down what each version brought to the table and in a second part, we'll also deep dive a few specific options that appeared in the encoder since my first blog post release so you can figure out what you may want to use for your projects. That includes variance boost, fast decode, temporal filtering strength and a few others... ## Feedback Although the reception was warm, I got less feedback than usual this time around, but honestly, that hasn't prevented me from wanting more. And better! I'm keeping the stuff that seemed well-received: the diverse test clips, visual comparisons, and my ongoing attempt to stay as objective as possible. In my last post's conclusion, I mentioned some frustrations: XPSNR was acting up, metrics were taking forever to calculate, and there was just too much data to make sense of. Good news is I've tackled all of these. I fixed the XPSNR issue (it expected mod-8 inputs), started using the new [Vship](https://codeberg.org/Line-fr/Vship) to speed up metrics calculation, and found a cleaner way to present all the data points at just two quality levels, like I used to do. Also, it needs to be pointed out I've been way too optimistic about how often I can get these posts out. Especially when I revamp the methodology each time. I'm sorry about that, again. Though, my ambition may also be at fault: the size of this article got so out of hand I eventually decided to split it into two parts for convenience's sake. ## Methodology You'll find both graphs and visual comparisons in this analysis. The graphs give you objective data on encoder efficiency across different settings, using various metrics. The image comparisons show the actual encoded samples so you can judge quality for yourself, adding that subjective element that numbers alone can't capture. This time, I'm using a new tool called [*Metrics*](https://github.com/psy-ex/metrics) from the self-proclaimed Psychovisual Experts group, which provides scripts for measuring and comparing video codecs. I've heavily modified these scripts for my specific needs, but if you want to run your own tests locally, definitely check out the original *Metrics* toolkit! --- Here's how the testing works: I use relatively short video samples covering a wide range of content types, all converted to uncompressed y4m format (if they weren't already!) for consistency. These lossless files go straight into *SvtAv1EncApp*, so we're measuring single-instance encoder performance here. Keep in mind that serious AV1 encoding pipelines should probably use chunked encoding (with a tool like [Av1an](https://github.com/rust-av/Av1an)), especially on higher core count systems. Once encoding is done, we run multiple full-reference metrics comparing the encodes against the original source. Using several different metrics helps compensate for each one's weaknesses and gives a more complete picture of the actual visual differences between encodes. I try to stray away from the industry standard metrics which tend to have poor correlation to *Mean Opinion Scores* (MOS), and instead focus more on psychovisual metrics that better represent actual visual quality. I calculate *SSIMULACRA2* and *Butteraugli* scores using [Vship](https://codeberg.org/Line-fr/Vship), an accurate GPU-accelerated port of the Zig implementation I used before that's also much faster. *XPSNR* and *VMAF* scores come from their respective ffmpeg filters, but with a twist! If you remember, last time I started making use of Harmonic Mean for SSIMU2 to better account for inconsistent scoring behaviors within a video. We'll be doing it again today. This time sees the addition of the (in)famous VMAF metric, but in a radically different form than you've probably ever seen until now. Indeed, it is well known VMAF is rather [unreliable](https://arxiv.org/pdf/2107.04510), but with a few modifications to the scoring method, we can try to make it better. I'm computing scores across all three color planes (instead of just luma as VMAF isn't chroma aware), then weighting them with this formula: `((4.0 * vmaf_y) + vmaf_u + vmaf_v) / 6.0`. This approach is inspired by the now-unavailable better-vmaf mod. I also chose to use [VMAF's neg model](https://github.com/Netflix/vmaf/blob/master/resource/doc/models_v0.md#disabling-enhancement-gain-neg-mode) and to disable the motion component (`motion.motion_force_zero=true`) since it's notorious in our niche encoding communities for inflating scores during motion and producing nonsensical results (bad looking frames having near perfect scores...). I'll refer to this metric as W-VMAF in the rest of this post. XPSNR gets similar treatment: motion component disabled ([by commenting out this line](https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_xpsnr.c#L240)) and the same weighting formula: `((4.0 * xpsnr_mse_y) + xpsnr_mse_u + xpsnr_mse_v) / 6.0`. As you may glimpse from said formula, XPSNR's dB values are converted back to linear MSE, we then calculate the weighted average and convert back to dB. I'll refer to this metric as W-XPSNR in the rest of this post. Butteraugli runs in stock configuration except for the intensity multiplier set to 203 nits instead of the currently default 80 (based on an [industry reference](https://www.itu.int/dms_pub/itu-r/opb/rep/R-REP-BT.2408-3-2019-PDF-E.pdf), used by MPV for instance). I believe these modifications produce more meaningful results than stock metrics, but proving that is beyond this post's scope. You'll have to trust the methodology or test it yourself. All this data gets aggregated to create the benchmark graphs. The "speed graphs" plot *Constant Rate Factor* (CRF) against encoding time to show speed efficiency at different quality targets. For compression efficiency, I plot metric scores against output file size. Since *Metrics* doesn't use *bits per pixel* (BPP) and I had received feedback a few blog posts back that this metric could be confusing, I decided to drop it this time around. To achieve more accurate efficiency curves with fewer probes, it is more effective to prioritize probes in the higher-quality (lower CRF) range of the quality spectrum, since bitrate will increase quicker. I used the following formula, courtesy of better-vmaf's author, to determine ten CRF values from 10 to 50 to use for the testing: `crfs = [min_q + (max_q - min_q) * ((step / (q_steps - 1)) ** scaling_factor) for step in range(q_steps)]`, then equally split into *High Quality* and *Low Quality* graphs and visual comparisons. How to read the graphs? For the compression efficiency graphs, higher and further left is better, except for Butteraugli, which is a distance metric where lower scores mean better quality, so you want bottom-left instead of top-left. For the encoding speed graphs, further left means faster. One important caveat about speed measurements: while the efficiency results should be reproducible regardless of your hardware, measuring encoding speed is trickier. The performance numbers you see here will likely differ on your setup depending on your specific hardware configuration. Please take them with a grain of salt. --- The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy. As the testing started a good while ago, about when v3.0.1 released in fact, and many encodes were already completed by the time v3.0.2 was out, I used the following encoder versions for this test: **v2.0.0**, **v2.1.2**, **v2.2.1**, **v2.3.0** and **v3.0.1**. The different *SvtAv1EncApp* binaries were compiled directly from their respective [source codes](https://gitlab.com/AOMediaCodec/SVT-AV1/-/releases/) found on the release section of the SVT-AV1 Gitlab repository, using **Clang 19.1.7** and the provided `Build/linux/build.sh` script with the following command: `build.sh cc=clang cxx=clang++ jobs=$(nproc) enable-lto static native release`. The testing machine is now comprised of an **i7 12700F**, whose E-cores have been disabled to avoid scheduler-related issues, with **2x8GB** of TOTL 3200MHz CL14 DDR4 RAM, in **Arch Linux** with kernel 6.12.17 and the performance governor enabled. I tried my best to run most encodes in the same session without rebooting, but a few issues made it so I had to re-run some in a new session. However, this does not concern this part of the article, so we'll revisit this in the next one. There is an exception to one of my above statement. As you may know, there is a feature that significantly impacted SVT-AV1's competitiveness: [**variance boost**](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/Appendix-Variance-Boost.md)! This feature can provide good efficiency improvements by increasing quality in low-contrast areas in frames, at little to no performance cost when properly bitrate normalized. This feature was in the work during the v1.7.0 days, but it only got merged to git relatively shortly after v2.0.0 released. As I wanted to enable *varboost* by default for this entire testing (as I had expressed in the conclusion of the last blog post), I decided to manually patch the v2.0.0 source code as provided on the release page with the following two commits: ["Introduce the variance boost feature"](https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/cb6de1134e9695e48f547a799e3818115f18ae93) and ["Do not adjust picture QP/qindex value with variance boost on"](https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/128712bf0aa83e4f680352b87c4b75ac3419364d). Of course, the feature slightly evolved since then, but this little modification still allowed me to test the evolution of the encoder performance with this key feature on. We're almost ready to deep dive into an ocean of metrics, graphs, and revelations! (Okay, maybe not revelations... but hopefully a few surprises.) ## Samples The samples are the same as last time: - 17s `Avatar The Way Of Water (trailer 3)` clip sourced from [thedigitaltheater.com](https://thedigitaltheater.com/category/movie-trailers/lossless-movie-trailers/) (1920x808p *with 4 columns and 4 rows of pure black borders to fix XPSNR*, 23.976fps). - 6s `Ducks Take Off` clip sourced from [xiph.org](https://media.xiph.org/video/derf/) (1280x720p, 50fps). - 3s `Fallout 4` clip sourced from another encoder fellow (1920x1080p, 60fps). - 8s `Minecraft` clip sourced from [xiph.org](https://media.xiph.org/video/derf/) (1920x1080p, 60fps). - 8s `Sol Levante` HDR clip sourced from [opencontent.netflix.com](http://download.opencontent.netflix.com/?prefix=SolLevante/) (3840x2160p, 24fps). This one is pretty educative as SVT-AV1's behavior isn't influenced by the existence (or lack thereof) of HDR metadata in a source. - 21s `Suzume (trailer 2)` clip sourced from [thedigitaltheater.com](https://thedigitaltheater.com/category/movie-trailers/lossless-movie-trailers/) *(seems to have been deleted since)* (1920x808p, 23.976fps). - 13s `The Mandalorian (trailer 2)` clip sourced from [thedigitaltheater.com](https://thedigitaltheater.com/category/movie-trailers/lossless-movie-trailers/) (1920x800p, 23.976fps). ## Visual comparisons ![comp_showcase](/img/comp_showcase.webp) Throughout this blog post, you’ll find *slow.pics* links that provide various visual comparisons between presets. The *"High Quality"* (noted HQ) and *"Low Quality"* (noted LQ) links showcase comparisons at two different quality targets. These have been carefully handcrafted to be as size-normalized as possible, given the available encodes. We want to be focusing on encodes with minimal bitrate deviation for a fair comparison. Feel free to double-check the bitrate of each frame or scene (as written in the top-left corner of each screenshot) to make a more informed observation, keeping the size difference in mind when comparing the encodes. Use the arrow keys and numpad to navigate between screenshots. Alternatively, you can click on "Slider comparison" and select two sources if you prefer comparing this way. ## Quality Target & Encoding Settings > **All clips have been encoded in a wide quality range, from `--crf 10` to `--crf 50`, with values determined using the previously described formula.** `--preset X --enable-variance-boost 1` are the only parameters used here, in conjunction with the CRF values. We will compare various varboost parameters' combinations in Part 2 of the article, but for now, the defaults are used. Obviously, it wouldn't be fun without me realizing, as I'm writing these words, that I forgot to use `--hierarchical-levels 4` like last time, but well, it is what it is. Nothing *that* significant. Else, the SVT-AV1 defaults were used. The ones worth mentioning are: - `--tune 1`: tune PSNR - `--aq-mode 2`: variance deltaq - `--enable-qm 0`: quantisation matrices disabled - `--enable-tf 1`: temporal filtering enabled - `--tf-strength 3`: default temporal filtering strength - `--sharpness 0`: default deblock and rate distortion mode decision - `--fast-decode 0`: decode optimizations disabled And more, like CDEF and restoration enabled, overlays and film-grain disabled... Anyway, time to kick things off with the presets comparisons! ## SVT-AV1 v3.0.x Presets Comparisons (-1 -> 10) *If all you care about is what preset to use in the latest available version, this is the section for you!* **In the following graphs, you may find comparisons between all SVT-AV1 v3.0.x presets, ranging from the slowest `--preset -1` to the current fastest `--preset 10`.** > Since **v2.3.0** (and unlike in **v2.1.x** and **v2.2.x**), preset 6 made its return as its own separate preset: it is not mapped to preset 7 anymore. However, the maximum preset has become 10, and anything above it is effectively mapped to 10. ### Efficiency ---> Here are the full efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). Quite a lot of data eh? So much so that readability is impacted. Hence why we'll also focus on two quality targets to better understand what's exactly going on. ---> Thus, let's look at the same data but zooming in on the *"high quality"* range (defined here as CRF10 through 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> And now, let's zoom in on the *"low quality"* range (defined here as CRF28 through 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). As we can see, **presets 9 and 10** can be quite unpredictable depending on the clip. Their efficiency curves are also clearly not monotonic, with unexpected efficiency regressions happening at very low CRFs (high bitrates), especially visible on SSIMU2. This behavior is likely enhanced by the harmonic scoring, but it is also observable on Butteraugli and W-VMAF to a lesser extent. When looking at **preset 10** in the low quality range, the curves can be so chaotic on the SSIMU2 side we can easily conclude there's a quality consistency issue at play, that is to say that certain frames score significantly lower than others. --- ---> Therefore, let's remove **presets 9 and 10** from the equation and have a better look at the "usable" presets, from **8 and below**: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Same, but focusing on the *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> And the *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). Efficiency wise, we instantly notice **presets 2 and below** are grouped together very tightly, while every other presets appear to be more evenly spaced. Consistency, and thus SSIMU2 harmonic scores in particular, tend to get really bad below *CRF38* or *44* depending on the clip, which at least teaches us that SVT-AV1 begins to struggle around these quality targets whatever the chosen preset (in its almost-stock configuration). Something interesting seems to be happening with **preset 8** where it scores better than *preset 7* on certain clips, even going neck and neck with *preset 6* at times according to W-VMAF. That is something worth investigating visually, in the following visual comparisons between the three presets: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/rxAIcuu7) | [Avatar (LQ)](https://slow.pics/c/8LCApWef) | | [Ducks (HQ)](https://slow.pics/c/jIDYwjXn) | [Ducks (LQ)](https://slow.pics/c/K0fQfPew) | | [Fallout (HQ)](https://slow.pics/c/zvLeJCdH) | [Fallout (LQ)](https://slow.pics/c/dsrtj4aN) | | [Minecraft (HQ)](https://slow.pics/c/Yc1tnPhN) | [Minecraft (LQ)](https://slow.pics/c/WSE7zC8B) | | [Sol Levante (HQ)](https://slow.pics/c/zmUvEM6J) | [Sol Levante (LQ)](https://slow.pics/c/kiP6AOYN) | | [Suzume (HQ)](https://slow.pics/c/OepN2y8E) | [Suzume (LQ)](https://slow.pics/c/lvkQehjN) | | [Mandalorian (HQ)](https://slow.pics/c/XknHyTna) | [Mandalorian (LQ)](https://slow.pics/c/M6D29CR1) | Do you agree with W-VMAF's numbers here? I think I do, to an extent. --- ---> Let's take a closer look at **presets 4 and below** to see if we can better observe the differences between the slower modes: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> At *"high quality"* (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> And at *"low quality"* (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). It is better, but still clear the differences are relatively small. Though, we can also put BD-rate numbers to these differences thanks to psy-ex's *metrics*! --- The following table show the BD-rate regressions of preset 0 through 4, with **preset -1** as the reference, when averaged across all four metrics and all seven clips: | Preset | BD-rate regression (vs P-1) | |:-:|:-:| | 0 | 0.97% | | 1 | 3.79% | | 2 | 7.82% | | 3 | 17.08% | | 4 | 22.41% | **Presets 0 through 2** are pretty close to the **reference preset -1** and we notice a huge jump from **preset 2** to **3**, with **preset 4** relatively scoring very close to **preset 3**. We will put these numbers into perspective after having looked at the performance of every presets. ### Speed Efficiency numbers are great, but they only tell half the story. The relative speed differences between presets can paint a drastically different picture of the situation and change our entire interpretation of the results so far. In the following graphs, you may find speed comparisons of the different presets, with either a linear or a logarithmic scale. The latter is useful to better visualize the large speed variations between presets, as it compresses the scale and makes smaller differences more apparent. I must reiterate my previous disclaimer that these numbers should be taken with a grain of salt. They only represent the speed of the given presets at each CRF value *over a single run* on a given machine. Making multiple runs could help eliminate small, undesired variations, but they'd come at the cost of time and electricity. It is not reasonable here considering the scale of this blog post, sacrifices must unfortunately be made. To help mitigate this issue, the processor temperature is closely monitored and the fans actively adjust to prevent overheating. I also make sure no background tasks are running and that only a single encoder instance is running at all times. This won't prevent some outliers from passing through, but they are usually easier to detect. ---> That said, let's begin by comparing the performance of all presets, from **-1 to 10**: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). First thing first, SVT-AV1 scales exceptionally well, from well above realtime speeds, to painfully placebo speeds. Looking at the graphs with logarithmic scales, we can see that encoding speeds, the slower presets we go, tend to increase in an exponential manner. Unfortunately for us, efficiency doesn't usually follow that trend! We do notice a substancial gap between **presets 4 and 5** however. ---> Let's also look at the performance of just **presets -1 through 4**, considered the slower, non-realtime presets: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). On the logarithmic scale, the presets are impressively evenly spaced, except for **preset -1** which is somewhat closer to **preset 0**. Now then, we need to combine what we learned about efficiency and speed to interpret the results in an informative way! ### Interpretation (TLDR) Despite the repeated presets shifting, some things never change. For *good* modes, **presets 2 and 4** still offer the best bang for your buck in balancing efficiency and speed. **Preset 2** is usually slightly more than 2x faster than *preset 1* and about 2x slower than *preset 3* while offering close to *preset 1* efficiency and largely better efficiency than *preset 3*. **Preset 4** happens to be the "slowest" good preset that's still very competitive, with little efficiency differences against *preset 3*, making it a valuable choice for anyone wanting a good balance of quality and speed for non-realtime usecases. For *realtime* modes, **any preset from 5 and above** will do as long as it reaches realtime on your system! However, consistency takes a huge hit at **presets 9 and 10**, rendering those two presets unappealing compared to hardware solutions that are likely to be quite competitive for this usecase. **Preset 8**, which strikes an excellent efficiency to speed ratio for what it is, is the last preset I would deem truly usable in SVT-AV1 v3.0.x. ## SVT-AV1 v2.0.0 -> v3.0.x Initial Presets Comparisons (-1 -> 10) In this section, we will first examine the *presets -1 to 10* range, independently of the presets shifting happening between the tested encoder versions. In a following section, we'll make more targeted comparisons taking into account the shifts. ### `preset -1`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Efficiency wise, **preset -1** has not improved from v2.0.0 to v3.0.x, rather it has stagnated. At worst, it regressed of exactly 2.0% BD-rate on one metric in one given clip. On average, the regression is closer to 0.5%. The good news though is that the preset became *up to 65% faster vs v2.0.0*! That's the best case scenario, as on average across the studied quality range, it has gotten *faster of about 37.5%*, which is still an amazing trade-off! It is fascinating to see how very different the presets' speed behave with CRF between versions, as at CRF50, the preset is barely faster on v3.0.x than it was on v2.0.0. Now, feel free to look at the following comparisons to see if you can spot much of a difference between the different versions at preset -1! ---> Preset -1 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/7snxpOZv) | [Avatar (LQ)](https://slow.pics/c/h3cE9Pva) | | [Ducks (HQ)](https://slow.pics/c/9Q2mes9k) | [Ducks (LQ)](https://slow.pics/c/VVtIWwgO) | | [Fallout (HQ)](https://slow.pics/c/CmFkc6g5) | [Fallout (LQ)](https://slow.pics/c/xpE2mZiX) | | [Minecraft (HQ)](https://slow.pics/c/i6xsIUa1) | [Minecraft (LQ)](https://slow.pics/c/F3KSC9w2) | | [Sol Levante (HQ)](https://slow.pics/c/pb9UcwUd) | [Sol Levante (LQ)](https://slow.pics/c/yF6fN4b9) | | [Suzume (HQ)](https://slow.pics/c/2ySf85bn) | [Suzume (LQ)](https://slow.pics/c/R8DBurLN) | | [Mandalorian (HQ)](https://slow.pics/c/cl1c54cP) | [Mandalorian (LQ)](https://slow.pics/c/CAWDE2f7) | --- ### `preset 0`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Here, we have an interesting case where **preset 0** efficiency got slightly better since v2.0.0 (by about 1.75%), though it remained largely unchanged after v2.1.x. There's one funny exception: according to all metrics, it peaked at *HQ* on the Minecraft clip during v2.1.x and regressed ever since. Speed wise, the preset took a hit in v2.1.x, improved a lot in v2.2.x almost closing the gap with v2.0.0, and regressed again in v2.3.0 and especially in v3.0.x. The regressions are not a good look, even though by in large, they're rather insignificant. Can you spot the differences in the comparisons below though? ---> Preset 0 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/u9OZOSxK) | [Avatar (LQ)](https://slow.pics/c/HXc5J1WU) | | [Ducks (HQ)](https://slow.pics/c/sv71x9XV) | [Ducks (LQ)](https://slow.pics/c/feQLSoX6) | | [Fallout (HQ)](https://slow.pics/c/u8KQnRuG) | [Fallout (LQ)](https://slow.pics/c/eYe84LaK) | | [Minecraft (HQ)](https://slow.pics/c/gtL1DkE4) | [Minecraft (LQ)](https://slow.pics/c/bcd8dyr6) | | [Sol Levante (HQ)](https://slow.pics/c/zsXcTpz4) | [Sol Levante (LQ)](https://slow.pics/c/tvaZVwuT) | | [Suzume (HQ)](https://slow.pics/c/XeKhvH4u) | [Suzume (LQ)](https://slow.pics/c/tVrFmswl) | | [Mandalorian (HQ)](https://slow.pics/c/sFL9h8zi) | [Mandalorian (LQ)](https://slow.pics/c/z5zW6yRP) | --- ### `preset 1`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: We start seeing more diverse results with **preset 1**. Versions after v2.0.0 tend to fare better efficiency wise, but each version present different strengths and weaknesses depending on the clip and metric. The speed graphs are quite chaotic, though the preset seems to have consistently gotten slightly slower and slower since the v2.0.0 days. Taking all of this into account, the overall interpretation would still be that the preset behavior is *mostly unchanged* from before. Let's play at "spot the differences"! After looking at the following comps, tell me what you think of the new trade-offs! ---> Preset 1 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/E72LcsSl) | [Avatar (LQ)](https://slow.pics/c/KFuurqAg) | | [Ducks (HQ)](https://slow.pics/c/FPId4Zqe) | [Ducks (LQ)](https://slow.pics/c/qMqqL0Lz) | | [Fallout (HQ)](https://slow.pics/c/vwixXHUX) | [Fallout (LQ)](https://slow.pics/c/k0nicdyT) | | [Minecraft (HQ)](https://slow.pics/c/gyer6fDr) | [Minecraft (LQ)](https://slow.pics/c/BRGkTPoT) | | [Sol Levante (HQ)](https://slow.pics/c/WtxUfd7I) | [Sol Levante (LQ)](https://slow.pics/c/ePf4GiMQ) | | [Suzume (HQ)](https://slow.pics/c/SjuspSWV) | [Suzume (LQ)](https://slow.pics/c/IVDIqGK9) | | [Mandalorian (HQ)](https://slow.pics/c/E2BqCUTc) | [Mandalorian (LQ)](https://slow.pics/c/FuS5w2bn) | --- ### `preset 2`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: On Minecraft, we notice (especially at *HQ*) a trend of **preset 2** progressively becoming worse after v2.1.x, by about 2.5% in fact vs v3.0.x. On other clips, it has either stagnated or slightly improved, except for v3.0.x where it seems to score consistently last or second to last. Relatively, *the differences are small*, so broadly speaking, the preset is again very similar to what it used to be. Well, except for its consistent *speed improvement of 10-40%* since v2.0.0, making the new trade-off considerably more appealing. Does the following comparisons help quantify the speed improvements in the new version? Well, no... they can't, but it's nice to have them anyway! Check them out! ---> Preset 2 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/GmKOaffP) | [Avatar (LQ)](https://slow.pics/c/XsTfILBQ) | | [Ducks (HQ)](https://slow.pics/c/1MapPnXt) | [Ducks (LQ)](https://slow.pics/c/Dr7IoQ6j) | | [Fallout (HQ)](https://slow.pics/c/86DRKP0R) | [Fallout (LQ)](https://slow.pics/c/VZNSVtdy) | | [Minecraft (HQ)](https://slow.pics/c/MKfT4aR7) | [Minecraft (LQ)](https://slow.pics/c/clfDCYS0) | | [Sol Levante (HQ)](https://slow.pics/c/9JkuBBUQ) | [Sol Levante (LQ)](https://slow.pics/c/gKOHHyxj) | | [Suzume (HQ)](https://slow.pics/c/uCTa7ncr) | [Suzume (LQ)](https://slow.pics/c/auIbFheo) | | [Mandalorian (HQ)](https://slow.pics/c/t9x5kOGV) | [Mandalorian (LQ)](https://slow.pics/c/q4kmtobA) | --- ### `preset 3`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: At last, efficiency changes that can be considered significant! The presets shifting is at play here, and it will only become more evident as we progress towards the faster presets. With v3.0.x having fewer presets than some earlier versions, the SVT-AV1 team spread them out more to fill the gaps, completing the shifting initiated in v2.3.0. **Preset 3** in v3.0.x almost consistently scores last, though that is less pronounced at *Low Quality*. On average, *its BD-rate is 2.7% lower* in v3.0.x vs v2.0.0. However, partly thanks to that, it received a hefty *speed boost of about 24%* since! Take a look at the graphs below, does this seem worth it to you? ---> Preset 3 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/bswlVaS9) | [Avatar (LQ)](https://slow.pics/c/bVlBtF5U) | | [Ducks (HQ)](https://slow.pics/c/WMB8HiIa) | [Ducks (LQ)](https://slow.pics/c/MRtHNOAV) | | [Fallout (HQ)](https://slow.pics/c/BEuwKWeU) | [Fallout (LQ)](https://slow.pics/c/CLIwIDQA) | | [Minecraft (HQ)](https://slow.pics/c/6nU4h6Av) | [Minecraft (LQ)](https://slow.pics/c/noTwMunx) | | [Sol Levante (HQ)](https://slow.pics/c/PMWrAwJ6) | [Sol Levante (LQ)](https://slow.pics/c/0yKDNkSG) | | [Suzume (HQ)](https://slow.pics/c/oXePAVWf) | [Suzume (LQ)](https://slow.pics/c/3RPadLsV) | | [Mandalorian (HQ)](https://slow.pics/c/9lwUBV2H) | [Mandalorian (LQ)](https://slow.pics/c/cjyYORDs) | --- ### `preset 4`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Same goes for **preset 4**! Our slow preset of choice has become *29% faster on average*, at the cost of a *3.0% efficiency loss*. With that in mind and the comps available right below, have your opinion of **preset 4** changed? ---> Preset 4 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/tyQdj3fH) | [Avatar (LQ)](https://slow.pics/c/RTSn1mek) | | [Ducks (HQ)](https://slow.pics/c/TSnz1fH3) | [Ducks (LQ)](https://slow.pics/c/JYB28hs1) | | [Fallout (HQ)](https://slow.pics/c/myrv1Wug) | [Fallout (LQ)](https://slow.pics/c/8RO9hvGH) | | [Minecraft (HQ)](https://slow.pics/c/Z3LOcUlJ) | [Minecraft (LQ)](https://slow.pics/c/jrTsw9Ph) | | [Sol Levante (HQ)](https://slow.pics/c/9rZCYMS6) | [Sol Levante (LQ)](https://slow.pics/c/bnpxfOo8) | | [Suzume (HQ)](https://slow.pics/c/cyUqkiWq) | [Suzume (LQ)](https://slow.pics/c/xTBouY62) | | [Mandalorian (HQ)](https://slow.pics/c/8Fao1kY8) | [Mandalorian (LQ)](https://slow.pics/c/7SVJpFiU) | --- ### `preset 5`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Unsurprisingly, the spreading continues! The gaming clips are the most affected here, showing an efficiency decrease of 6.1% and 14.3% respectively. Then again, the average across all seven clips is quite a bit lower at *4.5% lower efficiency*. *Speed is improved by 44%*, so the trade-off sounds reasonable. I'm starting to lose inspiration to introduce the comparisons, so simply have a look! ---> Preset 5 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/fwRG6aYK) | [Avatar (LQ)](https://slow.pics/c/93jKW8uB) | | [Ducks (HQ)](https://slow.pics/c/xZq2VS4u) | [Ducks (LQ)](https://slow.pics/c/iHY2I6Pp) | | [Fallout (HQ)](https://slow.pics/c/5k7S6JjP) | [Fallout (LQ)](https://slow.pics/c/VWAZyZLS) | | [Minecraft (HQ)](https://slow.pics/c/XV1if9np) | [Minecraft (LQ)](https://slow.pics/c/N4CugkJp) | | [Sol Levante (HQ)](https://slow.pics/c/pNqh7oLp) | [Sol Levante (LQ)](https://slow.pics/c/ia89EKhf) | | [Suzume (HQ)](https://slow.pics/c/ICbt1UM9) | [Suzume (LQ)](https://slow.pics/c/nJQCjkvK) | | [Mandalorian (HQ)](https://slow.pics/c/DQjLIrE2) | [Mandalorian (LQ)](https://slow.pics/c/6gpzZo9B) | --- ### `preset 6`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: More of the same! Though, the shift seems to have been initiated in v2.1.x, so **preset 6** in v3.0.x is quite different from its older sibling from v2.0.0. **Compared to v2.0.0**, *preset 6* is now both *47% faster* and *8.8% lower efficiency*, while **compared to v2.1.2 instead**, it is roughly *27% faster* and *2.6% lower efficiency*. The gaming clips are the most affected again, showcasing that complex content suffers more as you move to higher presets. Find the visual comparisons right after, if you're interested. ---> Preset 6 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/NfMp2Qxb) | [Avatar (LQ)](https://slow.pics/c/fvRtLfXJ) | | [Ducks (HQ)](https://slow.pics/c/k68hC9TU) | [Ducks (LQ)](https://slow.pics/c/8YpIfbGP) | | [Fallout (HQ)](https://slow.pics/c/936r0JxK) | [Fallout (LQ)](https://slow.pics/c/QXc8lc0V) | | [Minecraft (HQ)](https://slow.pics/c/LhqYPsqK) | [Minecraft (LQ)](https://slow.pics/c/NusQfYg0) | | [Sol Levante (HQ)](https://slow.pics/c/H5TkQgQO) | [Sol Levante (LQ)](https://slow.pics/c/ZF0aLLti) | | [Suzume (HQ)](https://slow.pics/c/lt3HXK9x) | [Suzume (LQ)](https://slow.pics/c/H1qa9TTW) | | [Mandalorian (HQ)](https://slow.pics/c/0mDTEvnx) | [Mandalorian (LQ)](https://slow.pics/c/mRRQt2Ky) | --- ### `preset 7`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: With **preset 7**, it is clear the downward shift in efficiency was progressively set in motion since v2.3.0. **Compared to v2.0.0**, *v3.0.x's preset 7* sees an *11.6% efficiency hit*, traded for a *68% speed improvement*, not too bad. **Versus v2.3.0**, the *new preset 7* has a *5% worse efficiency* and only *14% better speeds*. This difference is arguably the least appealing of every presets so far... How did the preset evolve visually though? The answer is in the following comparisons! ---> Preset 7 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/UVDaSDcb) | [Avatar (LQ)](https://slow.pics/c/zh84Mp6e) | | [Ducks (HQ)](https://slow.pics/c/0DrdFnYu) | [Ducks (LQ)](https://slow.pics/c/86F1oqdy) | | [Fallout (HQ)](https://slow.pics/c/IHZ5zSsa) | [Fallout (LQ)](https://slow.pics/c/PGWRkxiF) | | [Minecraft (HQ)](https://slow.pics/c/MAcpzaop) | [Minecraft (LQ)](https://slow.pics/c/hCwQbbLe) | | [Sol Levante (HQ)](https://slow.pics/c/nSfCm0lW) | [Sol Levante (LQ)](https://slow.pics/c/1CKfXF9T) | | [Suzume (HQ)](https://slow.pics/c/v8GZVrQ8) | [Suzume (LQ)](https://slow.pics/c/V9aFc5d3) | | [Mandalorian (HQ)](https://slow.pics/c/hMmMbBBS) | [Mandalorian (LQ)](https://slow.pics/c/0HgfcQto) | --- ### `preset 8`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: **Preset 8**'s trade-offs are significantly altered in v3.0.x. This is relatively proportionate to the gain in speed, but we may have expected better. Efficiency wise, it is *15.8% lower* and *48% faster* than in v2.0.0. To be fair, as the presets are starting to stray too much from each other, we should probably compare it to a faster v2.0.0 preset like 9. Look up the visual comparisons while keeping that in mind. ---> Preset 8 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/j1HlBkwz) | [Avatar (LQ)](https://slow.pics/c/DTxPsMDM) | | [Ducks (HQ)](https://slow.pics/c/Y5MifaMo) | [Ducks (LQ)](https://slow.pics/c/qQN9QNWU) | | [Fallout (HQ)](https://slow.pics/c/RitaiAEa) | [Fallout (LQ)](https://slow.pics/c/PHsPKE03) | | [Minecraft (HQ)](https://slow.pics/c/N9Gg6dvo) | [Minecraft (LQ)](https://slow.pics/c/rNVmLiEz) | | [Sol Levante (HQ)](https://slow.pics/c/b0LHyz5w) | [Sol Levante (LQ)](https://slow.pics/c/FJH9IhKi) | | [Suzume (HQ)](https://slow.pics/c/mvPN2u37) | [Suzume (LQ)](https://slow.pics/c/Ei7Wbvk9) | | [Mandalorian (HQ)](https://slow.pics/c/g2I52mPH) | [Mandalorian (LQ)](https://slow.pics/c/kz6qcrTm) | --- ### `preset 9`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Similar to *preset 8*, it is becoming really hard to make a fair comparison between very disparate presets, especially when it is starting to break at extreme CRF values. It is clear **v3.0.x's preset 9** is built differently than *v2.0.0's*, it plays in a completely different league. I will not elaborate further at this point, as we will revisit this preset in the next section. There are still visual comparisons, if you're curious: ---> Preset 9 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/OdUkbCaJ) | [Avatar (LQ)](https://slow.pics/c/l9QdddJH) | | [Ducks (HQ)](https://slow.pics/c/cjgpua5F) | [Ducks (LQ)](https://slow.pics/c/9VCjfwSs) | | [Fallout (HQ)](https://slow.pics/c/T7CeIQxx) | [Fallout (LQ)](https://slow.pics/c/4oWIQ96e) | | [Minecraft (HQ)](https://slow.pics/c/cS9XPPvJ) | [Minecraft (LQ)](https://slow.pics/c/OC42ixqX) | | [Sol Levante (HQ)](https://slow.pics/c/GLY1TNke) | [Sol Levante (LQ)](https://slow.pics/c/O4wQ8qI5) | | [Suzume (HQ)](https://slow.pics/c/g12kkzKH) | [Suzume (LQ)](https://slow.pics/c/XgzXw5do) | | [Mandalorian (HQ)](https://slow.pics/c/roJJVsLE) | [Mandalorian (LQ)](https://slow.pics/c/wSPqbIjf) | --- ### `preset 10`: v2.0.0 -> v3.0.x ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Unlike the previous two presets, the new **preset 10** can somewhat be compared to its previous iterations. After disregarding *v2.2.1's* which is the one to behave distinctly here, **v3.0.x's preset 10** show similar characteristics across the quality range. For what it's worth, the preset is *38.5% less efficient* but *39.1% faster* compared to v2.0.0. It should be noted that as we go faster, the gap between fast presets is more likely to shrink on shorter clips due to the low encode times (in the order of a few seconds). As the time taken by the encoder to initialize isn't quite negligible anymore, using longer clips would likely increase the lead of **preset 10** in v3.0.x vs v2.0.0 as encoding speeds would have gotten enough time to stabilize. This is obviously out of the scope of this blog post, but still, the analysis in the next section will help shed light on how the fastest preset in v3.0.x compares to the fastest one in v2.0.0. I invite you to take a look at the visual comparisons if you want to see how *preset 10* evolved from v2.0.0 to v3.0.x. ---> Preset 10 Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/TzFCXBiX) | [Avatar (LQ)](https://slow.pics/c/1ynK7ErT) | | [Ducks (HQ)](https://slow.pics/c/phOYk2Un) | [Ducks (LQ)](https://slow.pics/c/vjEFuqlA) | | [Fallout (HQ)](https://slow.pics/c/TTUG4Oha) | [Fallout (LQ)](https://slow.pics/c/BY7pZJdR) | | [Minecraft (HQ)](https://slow.pics/c/7TwJykvh) | [Minecraft (LQ)](https://slow.pics/c/rYIW5Flt) | | [Sol Levante (HQ)](https://slow.pics/c/NJLhXART) | [Sol Levante (LQ)](https://slow.pics/c/5UJnwTuq) | | [Suzume (HQ)](https://slow.pics/c/hlDThBg0) | [Suzume (LQ)](https://slow.pics/c/rmDWXipa) | | [Mandalorian (HQ)](https://slow.pics/c/QqkHhYsk) | [Mandalorian (LQ)](https://slow.pics/c/vwbowkAs) | ## SVT-AV1 v2.0.0 vs v3.0.x Selective Presets Comparisons Some presets cannot be directly compared anymore, as the reduction in presets since v2.0.0 means the gap had to be filled, and the fastest presets were the most affected by this move. I decided to focus on SVT-AV1 v3.0.x's *presets 8*, *9* and *10* here, then we'll proceed do something new for this blog post series... ### v3.0.x's `preset 10` vs v2.0.0's `preset 13` How does v3.0.x's fastest preset fares against v2.0.0's own? ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: Well, the answer is quite nicely! Depending on the clip and metric, v3.0.x is overall either lightly ahead or lightly behind, while staying equally fast! The trade-offs are mildly different, as v3.0.x is capable of being even slightly faster, at the cost of some efficiency. It's fair to say the dev team managed to preserve the performance of the fastest mode despite said reduction in presets. *Preset 13* was always considered an experimental mode reserved for convex hull purposes, so it's my assumption that the dev team had no real intention of making the fastest preset even faster if accuracy was impacted further than it already is. Here are some visual comparisons to visualize the situation! Did you miss them? ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/CeJKaDn9) | [Avatar (LQ)](https://slow.pics/c/gixmjNww) | | [Ducks (HQ)](https://slow.pics/c/uOYxGIh5) | [Ducks (LQ)](https://slow.pics/c/4JxasZSn) | | [Fallout (HQ)](https://slow.pics/c/RYSBtmDW) | [Fallout (LQ)](https://slow.pics/c/paBShIhM) | | [Minecraft (HQ)](https://slow.pics/c/SceyUJPY) | [Minecraft (LQ)](https://slow.pics/c/XqhPRl2m) | | [Sol Levante (HQ)](https://slow.pics/c/S9PRwEUf) | [Sol Levante (LQ)](https://slow.pics/c/3JgTLYcs) | | [Suzume (HQ)](https://slow.pics/c/SFKjq59f) | [Suzume (LQ)](https://slow.pics/c/kZdSkYaw) | | [Mandalorian (HQ)](https://slow.pics/c/VhGBR0G6) | [Mandalorian (LQ)](https://slow.pics/c/qtucNxVy) | --- ### v3.0.x's `preset 9` vs v2.0.0's `presets 11 & 12` ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: In a nutshell, **preset 9** in v3.0.x fares in between v2.0.0's presets 11 and 12 in efficiency, though usually closer to preset 11 overall, while usually being closer in speed to preset 12 (notable exception on the gaming clips)! That's a free efficiency or speed boost for any non-gamer upgrading! Screenshots... screenshots everywhere! Find them right below: ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/cO94QQdI) | [Avatar (LQ)](https://slow.pics/c/9cjrP2O4) | | [Ducks (HQ)](https://slow.pics/c/AlNsI0Zj) | [Ducks (LQ)](https://slow.pics/c/FtcbLYlh) | | [Fallout (HQ)](https://slow.pics/c/yM0zgRkY) | [Fallout (LQ)](https://slow.pics/c/aKvQww2A) | | [Minecraft (HQ)](https://slow.pics/c/LqYh01UD) | [Minecraft (LQ)](https://slow.pics/c/ZDe8R4de) | | [Sol Levante (HQ)](https://slow.pics/c/nhdlzESm) | [Sol Levante (LQ)](https://slow.pics/c/GRv94zWH) | | [Suzume (HQ)](https://slow.pics/c/JLSRi3aX) | [Suzume (LQ)](https://slow.pics/c/7FnyFHAn) | | [Mandalorian (HQ)](https://slow.pics/c/Fi79eSMN) | [Mandalorian (LQ)](https://slow.pics/c/USPgAC2s) | --- ### v3.0.x's `preset 8` vs v2.0.0's `presets 9 & 10` ---> Compression efficiency graphs, *full* quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). ---> Interpretation: As with *preset 9* previously, **preset 8** in v3.0.x usually competes closer to *v2.0.0's preset 9* in efficiency, while performing one tier higher with speeds comparable to *v2.0.0's preset 10*. The results are more chaotic on the gaming clips but one clear advantage of *preset 8* is its robust consistency, unlike *v2.0.0's preset 10* (and *v3.0.x's two faster presets*). Here's our last set of comps for the day: ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/ldcvqdEf) | [Avatar (LQ)](https://slow.pics/c/NdmLe6Bh) | | [Ducks (HQ)](https://slow.pics/c/lhwMU6m4) | [Ducks (LQ)](https://slow.pics/c/EegUibz1) | | [Fallout (HQ)](https://slow.pics/c/t1gd5pQE) | [Fallout (LQ)](https://slow.pics/c/3P4lmgXC) | | [Minecraft (HQ)](https://slow.pics/c/0XcVB7M0) | [Minecraft (LQ)](https://slow.pics/c/7zo5ZuCT) | | [Sol Levante (HQ)](https://slow.pics/c/gcMqaiP5) | [Sol Levante (LQ)](https://slow.pics/c/B3mjiCJ9) | | [Suzume (HQ)](https://slow.pics/c/ANtnJzta) | [Suzume (LQ)](https://slow.pics/c/91YCoS9b) | | [Mandalorian (HQ)](https://slow.pics/c/YZErMped) | [Mandalorian (LQ)](https://slow.pics/c/nIoc7RN4) | ## SVT-AV1 v2.0.0 -> v3.0.x General BD-rate Evolution Before we wrap up on the presets analysis, I'm going to attempt an exercise widely used for comparing video encoder performance in both academic research and industry benchmarks. *BD-rate* (Bjøntegaard Delta rate) calculates the bitrate savings between two encoders at equivalent quality levels, giving you a single percentage that represents compression efficiency gains. You've already seen these numbers throughout the post. While BD-rate has its limitations (it assumes rate-distortion curves follow specific mathematical models and can struggle with very different encoder behaviors), it remains the most widely accepted metric for encoder comparisons. Psy-ex's *metrics* conveniently outputs BD-rate numbers when running its benchmarking scripts, so this served as the basis for what we're about to do. The graphs in this section plot *BD-rate* (%) against *encoding time* (ms), showing you the classic speed-vs-efficiency trade-off that defines encoder development. The first batch of graphs uses SVT-AV1 *v2.0.0's preset -1* as the reference: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). We can instantly see the limitations of this approach. Using accurate results is vital for these graphs to make any sense, and having such bumpy curves is proof that despite my efforts in ensuring the measured encoding times were correct, this methodology is too sensitive to even the smallest of deviations. That being said, I'm not helped by the very behavior of the encoder I'm benchmarking: as we've seen in earlier speed graphs, SVT-AV1's performance doesn't scale linearly with CRF values. That is usually the expected behavior of encoders, and prior SVT-AV1 versions used to be like this too (cf. first deep dive article). This entails that when calculating the average BD-rate across the entire quality range, the shape of the speed curve can seriously throw off the results. Anyway, let's try to interpret these results a bit. The faster presets can be found at the top left of the graph, and the slower ones in the bottom right. We can still generally notice the brighter colors tend to be above the darker ones (representing the newer versions), meaning the trade-offs did in fact improve *overall*. If we take **preset -1** as an easy-to-analyze example, we can confirm our previous findings that it did in fact get faster and faster with versions at little to no efficiency impact. What happens if we take a radically different preset as the reference though? Well, I got you covered with the following graphs, using *SVT-AV1 v2.0.0's preset 10* (the default if unspecified) as the reference: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p1.json). In this case, it tends to straighten the curves and make any sort of analysis harder. It does not help the situation and there's not much more to say with these graphs. At least, it gives a different perspective of the results. All in all, let's just say this was a fine experiment and a good learning opportunity. I'll think of ways I can do this better by next time. ## SVT-AV1 v2.0.0 -> v3.0.x Conclusion So what do you think of SVT-AV1's evolution from v2.0.0 to v3.0.x? Do you find it underwhelming? Well, that's almost expected. The SVT-AV1 dev team has been hard at work to reduce the amount of presets. As that's been a focus for quite some time, I'm not sure the initial reasons for this are still valid today. Anyway, what we're mostly seeing are small but free speed improvements from smart trade-off decisions. It's worth noting that versions 2.0.0 and 3.0.0 weren't actually major feature milestones. The version numbers jumped because of breaking API changes that aren't backward compatible with previous releases. Beyond the incremental performance gains, analysis of the different changelogs reveals a clear development strategy focused on specific areas. The dev team has invested heavily in ARM optimizations and memory requirements reduction, worked on the fast-decode feature to further reduce decoding cycles, and streamlined the architecture by removing the decoder component entirely. While many of these changes might not influence quality metrics, they represent important steps toward broader SVT-AV1 adoption by facilitating integration for actors in the industry and ensuring cross-platform consistency. The biggest changes since v2.0.0 probably lie in the parameters originating from *SVT-AV1-PSY* that were recently introduced. ## Presets Analysis TLDR So SVT-AV1 v3.0.0 delivers some nice speed gains across the board. I will refrain from giving numbers to which exact presets possess the best efficiency-to-speed trade-offs due to my speed accuracy concerns, but I can confidently say **presets 2 and 4** remain the efficiency champions, giving you excellent quality without completely destroying your encode times. **Presets 5-8** strike a good balance, trading a bit of efficiency for significantly quicker encodes. This should give you a good foundation for picking your go-to preset(s) in v3.0.x. ## SVT-AV1 v3.0.x Parameters Revisited This blog post is already pretty long... Although we won't revisiting *every* encoder parameter like we did back in the first deep dive on v1.8.0, we will concentrate on a few important ones, some of them coming straight from the *SVT-AV1-PSY* project! **EDIT:** This section has been developed further in Part 2 of the article, now [available here](/articles/svt-av1-fourth-deep-dive-p2/)! ## Closing Thoughts Today, we tested five SVT-AV1 versions, ranging from v2.0.0 to v3.0.x, to quantify their relative efficiency and speed. New metrics were introduced and the format from the last blog post entry was iterated upon to improve the overall quality and flow of this present article. We used this opportunity to experiment and learn encoding knowledge along the way, I hope it was valuable to you too! This is only the first part of this benchmarking session, as we'll deep dive variance boost and a few other exciting features in the next one. Revisiting certain key parametres in such a context has been on my mind for quite a while, so I hope you will look forward to Part 2! Your feedback and suggestions are always welcome as I work to improve this blog format. Do you have a request for me? Something you'd like to see fixed or added? Let me know what you think on socials or in the different communities I'm active in! Thanks for reading and see you soon! I want to extend my thanks to the people who contributed, directly or indirectly, to the making of this article, including [Gianni Rosato (gb82)](https://github.com/gianni-rosato), [Line (Lumen)](https://codeberg.org/Line-fr), [Soda](https://github.com/GreatValueCreamSoda), [Emre](https://github.com/emrakyz), [Bolu](https://github.com/0xb01u/), [Julio Barba](https://github.com/juliobbv-p), the people behind slow.pics for hosting thousands of screenshots each time I make these articles, the SVT-AV1 dev team for the work on this amazing encoder, and probably others I'm forgetting... --- *Consider supporting me by making a donation on [my Ko-Fi page](https://ko-fi.com/nekotrix), to reward my efforts and to compensate for the electricity bills of weeks of non-stop encoding.* {} --- # Deep Dive into SVT-AV1's Evolution (Part 2): Encoder Parameters Revisited Canonical URL: https://encode.wiki/articles/svt-av1-fourth-deep-dive-p2/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 1ffc4477abb4037f233b1abecdd233daee5d2038f1dbdfe1623b7107940200e0 # Introduction Welcome to the second part of my SVT-AV1 testing analysis! I received lots of kind words, including constructive feedback to improve on my methodology, I sincerely appreciate it! While I couldn’t incorporate those suggestions for this follow-up, stay tuned, because you’ll likely hear more from me before summer ends. Before we continue, if you haven’t seen Part 1 yet, I recommend giving it a quick look! It covers the test methodology, sample clips, and base encoder settings in detail. The only change to this Part is my decision to remove the *full* graphs in an attempt to cut down on bloat. If you really want to, you can still access them on the repository where we upload pictures for the blog posts [here](https://github.com/av1-community-contributors/images). Now, without further ado, let’s dive right back in where we left off! ## SVT-AV1 v3.0.x Parameters Revisited Today, we are looking at **8 encoder features** present in SVT-AV1 v3.0.x! Although we won't be revisiting *every* encoder parameter like we did back in the first deep dive on v1.8.0, we are going to concentrate on a few important ones, some of them coming straight from the *SVT-AV1-PSY* project! Let's start with the feature you are likely most exciting about: *varboost*! ### Variance Boost The author of variance boost made a highly visual explanation of the feature in the official SVT-AV1 documentation you can find [here](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/Appendix-Variance-Boost.md). This overview will probably do a better job than me in describing what it does, but in a few words, *varboost* allocates more bits to low-contrast areas in a frame. You can enable varboost via the `--enable-variance-boost` parameter and control its behavior by changing its `--variance-boost-strength` or its `--variance-octile` setting. Basically, the strength controls how much areas are to be boosted, while octile controls how much of the area needs to be deemed low-contrast before being boosted. > *In every graph of this section, you will notice `vbs_o_` in the legend. The `vb` stands for `varboost`, the `s` stands for `strength` and lastly the `o` stands for octile. In the last subsection, you will even see `vbs_o_c_` with the `c` standing for `curve`. To take one example, `vbs1o4c2` means `varboost at strength 1 and octile 4 on the 2nd curve`.* --- #### Varboost Strength We're going to test all 4 available *strengths* values, with *octile* as the variable... ##### ...starting with `--variance-boost-strength 1`! ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). At HQ, on almost all metrics (excepted VMAF which is more inconsistent here), all octile values are more efficient than *varboost disabled*. With one notable exception in *octile 8* which often scores the same or worse than *no varboost* depending on the clip. The lower octiles seem to score increasingly better here. ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). At LQ, the results tend to be the same, however there are more outliers and overall the octile value seem to be more negligible. *No varboost* often scores better than many octile values on VMAF, which is an interesting behavior to observe. ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). In terms of speed, we won't be surprised to learn that the lower octile you go, the slower the encoding instance will be, as the output filesize is bigger and that *tends* to slow down SVT-AV1. By bitrate normalizing, we would get closer performance between all options. ---> Interpretation: `1` is the most conservative strength value, so we expect reasonable results from varboost here. All octile values consistently provide efficiency improvements except for `--variance-octile 8`. It is counter-intuitive to see lower octiles score better in efficiency so far, but we'll see if that changes as we increase the strength. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/mVtRwupm) | [Avatar (LQ)](https://slow.pics/c/Qoa3amhs) | | [Ducks (HQ)](https://slow.pics/c/LpnpWeCh) | [Ducks (LQ)](https://slow.pics/c/Ei6cY7hT) | | [Fallout (HQ)](https://slow.pics/c/HzcmlT95) | [Fallout (LQ)](https://slow.pics/c/H4lJgTSZ) | | [Minecraft (HQ)](https://slow.pics/c/EcRac2q1) | [Minecraft (LQ)](https://slow.pics/c/7zucoraP) | | [Sol Levante (HQ)](https://slow.pics/c/Ra474TKQ) | [Sol Levante (LQ)](https://slow.pics/c/dfo9gMIX) | | [Suzume (HQ)](https://slow.pics/c/HF6ls9AW) | [Suzume (LQ)](https://slow.pics/c/RaZ0Q6SM) | | [Mandalorian (HQ)](https://slow.pics/c/0vwqi52J) | [Mandalorian (LQ)](https://slow.pics/c/9dBsnQaJ) | ##### Let's continue with `--variance-boost-strength 2`, the default *strength* value in every iteration of SVT-AV1: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: The picture is consistent with what we've previously seen, so I won't repeat myself. However, we must keep in mind that the higher strength and the lower octile we go, the more the curve is shifted to the top right, which *may not* paint the most fair picture of the situation and might skew our interpretation in favor of the one end of the graph or another. That being said, except for VMAF which behaves a bit differently with varboost, we see the default of `--variance-boost-strength 2 --variance-octile 6` always increases efficiency over *no varboost* across basically all clips and metrics, so we can confidently say it does live up to its promise. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/ij7c8xbT) | [Avatar (LQ)](https://slow.pics/c/I8VX0gmb) | | [Ducks (HQ)](https://slow.pics/c/SRIm1hoy) | [Ducks (LQ)](https://slow.pics/c/SR3f3YWp) | | [Fallout (HQ)](https://slow.pics/c/7NeqER2Z) | [Fallout (LQ)](https://slow.pics/c/CQFdBB9C) | | [Minecraft (HQ)](https://slow.pics/c/crBjBTvx) | [Minecraft (LQ)](https://slow.pics/c/hfA0CXrp) | | [Sol Levante (HQ)](https://slow.pics/c/3l2dOaoE) | [Sol Levante (LQ)](https://slow.pics/c/cSDSKnv7) | | [Suzume (HQ)](https://slow.pics/c/pSVfrZSK) | [Suzume (LQ)](https://slow.pics/c/1B0Af0Ta) | | [Mandalorian (HQ)](https://slow.pics/c/t6MEfwHh) | [Mandalorian (LQ)](https://slow.pics/c/UJmCWT1T) | ##### Now for `--variance-boost-strength 3`: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: It's more of the same, varboost at *strength 3* tends to perform better in efficiency at higher quality targets though there can still be gains at lower qualities depending on the content at hand. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/l1PSZy1m) | [Avatar (LQ)](https://slow.pics/c/6XyZa5m2) | | [Ducks (HQ)](https://slow.pics/c/HfgyEtI6) | [Ducks (LQ)](https://slow.pics/c/vNb218sk) | | [Fallout (HQ)](https://slow.pics/c/lTxBGS4h) | [Fallout (LQ)](https://slow.pics/c/Zb1SsMXo) | | [Minecraft (HQ)](https://slow.pics/c/t7pX8SGu) | [Minecraft (LQ)](https://slow.pics/c/dKcQDN48) | | [Sol Levante (HQ)](https://slow.pics/c/q98rqqlx) | [Sol Levante (LQ)](https://slow.pics/c/fiTECJdJ) | | [Suzume (HQ)](https://slow.pics/c/xAcvt1gg) | [Suzume (LQ)](https://slow.pics/c/pZxBf9s9) | | [Mandalorian (HQ)](https://slow.pics/c/0dhkMBkj) | [Mandalorian (LQ)](https://slow.pics/c/ab9AOw54) | ##### And lastly, `--variance-boost-strength 4`, which will print you a warning that this may be too aggressive in some usecases: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: So, what does this unusual varboost strength have in store for us? Well, nothing as dangerous as we're led to believe. In fact, by in large, we observe the same things as the other three strengths. You may have noticed this yourself, as we've increased the strength, Butteraugli has tended to highlight *lower octiles* more and more at high quality and *no varboost* more and more at low quality. Still, *varboost on* often provides great gains and the optimal octile depends a lot on the sample. The default *octile 6* is a fine, safe default. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/qUhijQqk) | [Avatar (LQ)](https://slow.pics/c/fLvog0BW) | | [Ducks (HQ)](https://slow.pics/c/DCX9HCno) | [Ducks (LQ)](https://slow.pics/c/M8PcEi4T) | | [Fallout (HQ)](https://slow.pics/c/yfF24i4L) | [Fallout (LQ)](https://slow.pics/c/1aCFxgND) | | [Minecraft (HQ)](https://slow.pics/c/bTHqdZS3) | [Minecraft (LQ)](https://slow.pics/c/0vk3SJ0C) | | [Sol Levante (HQ)](https://slow.pics/c/hYRDUXh1) | [Sol Levante (LQ)](https://slow.pics/c/DcIxPyX2) | | [Suzume (HQ)](https://slow.pics/c/FaTSgkNC) | [Suzume (LQ)](https://slow.pics/c/1exzpYov) | | [Mandalorian (HQ)](https://slow.pics/c/RARbCDOr) | [Mandalorian (LQ)](https://slow.pics/c/SMe5q0Kv) | --- #### Varboost Octile Now, let's do the contrary and test all 8 *octile* values while solely adjusting the *strength*... ##### ...starting with `--variance-octile 1`! ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: For an octile value that supposedly should boost too many blocks in frames, `1` scores pretty consistently better than *no varboost*, especially at high quality. What's more, even though it can be argued all varboost curves are extremely close to one another, higher strengths are ever so slightly better performing than lower ones at high quality, while the contrary is more often observed at low quality though not always! Fallout seems to benefit less from varboost, heck it can be harmful at low quality according to some metrics. You will be unsurprised to hear that the higher the strength the slower encoding tends to be. Again, speed is affected by the resulting output filesize. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/GzZ0xiAW) | [Avatar (LQ)](https://slow.pics/c/ZuIOSRed) | | [Ducks (HQ)](https://slow.pics/c/jUNbTEHV) | [Ducks (LQ)](https://slow.pics/c/bknKSeOY) | | [Fallout (HQ)](https://slow.pics/c/zvc0jqvt) | [Fallout (LQ)](https://slow.pics/c/ongtADY8) | | [Minecraft (HQ)](https://slow.pics/c/NKeFM3q4) | [Minecraft (LQ)](https://slow.pics/c/8LTGKKGa) | | [Sol Levante (HQ)](https://slow.pics/c/CB3GU6g5) | [Sol Levante (LQ)](https://slow.pics/c/h3jJaTAG) | | [Suzume (HQ)](https://slow.pics/c/kCJj6jDP) | [Suzume (LQ)](https://slow.pics/c/fpzkIsUb) | | [The Mandalorian (HQ)](https://slow.pics/c/ZeVlmDlS) | [The Mandalorian (LQ)](https://slow.pics/c/PVvi6EWQ) | ##### Let's look at `--variance-octile 2`: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: **Octile 2**'s results are similar to *1*, that is to say close curves between all strengths, with a tendency for higher strengths to be better at high quality, for lower strengths to be at times preferable at low quality, and for varboost to overall be beneficial over it disabled, except according to VMAF. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/ImqHaKb7) | [Avatar (LQ)](https://slow.pics/c/EhkP5ZaF) | | [Ducks (HQ)](https://slow.pics/c/0CRmG4s9) | [Ducks (LQ)](https://slow.pics/c/lkB0foeJ) | | [Fallout (HQ)](https://slow.pics/c/TlQCQoeH) | [Fallout (LQ)](https://slow.pics/c/ZSxW0PTv) | | [Minecraft (HQ)](https://slow.pics/c/KuqBscxE) | [Minecraft (LQ)](https://slow.pics/c/fZL3hiWK) | | [Sol Levante (HQ)](https://slow.pics/c/eLZy8Fjv) | [Sol Levante (LQ)](https://slow.pics/c/B77FJMuy) | | [Suzume (HQ)](https://slow.pics/c/zvkKQTHO) | [Suzume (LQ)](https://slow.pics/c/5B82eyRP) | | [The Mandalorian (HQ)](https://slow.pics/c/80ofSD0W) | [The Mandalorian (LQ)](https://slow.pics/c/ZWSi8W6G) | ##### Let's test `--variance-octile 3`: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: It can be summarized the same as *octile 1* and *2*. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/WriHrzHL) | [Avatar (LQ)](https://slow.pics/c/bXA4FPus) | | [Ducks (HQ)](https://slow.pics/c/RwggDVq4) | [Ducks (LQ)](https://slow.pics/c/Z8MERQqW) | | [Fallout (HQ)](https://slow.pics/c/Cnsx12Ct) | [Fallout (LQ)](https://slow.pics/c/KUFUrvhX) | | [Minecraft (HQ)](https://slow.pics/c/SxnyNTDr) | [Minecraft (LQ)](https://slow.pics/c/GTydHu65) | | [Sol Levante (HQ)](https://slow.pics/c/iJFd4qzD) | [Sol Levante (LQ)](https://slow.pics/c/gVNXWh7C) | | [Suzume (HQ)](https://slow.pics/c/kYGB1IWf) | [Suzume (LQ)](https://slow.pics/c/yGLJnHAB) | | [The Mandalorian (HQ)](https://slow.pics/c/ryzIi6Ky) | [The Mandalorian (LQ)](https://slow.pics/c/er2a3xNi) | ##### And `--variance-octile 4`: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: I swear it's not just me getting lazy, there's not much more to say than as already been said! ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/bn1iUk8T) | [Avatar (LQ)](https://slow.pics/c/S40uaj7w) | | [Ducks (HQ)](https://slow.pics/c/5auZYNLd) | [Ducks (LQ)](https://slow.pics/c/OH59amm8) | | [Fallout (HQ)](https://slow.pics/c/PppdU8Cp) | [Fallout (LQ)](https://slow.pics/c/LWF9w3KG) | | [Minecraft (HQ)](https://slow.pics/c/Z3Rl51sK) | [Minecraft (LQ)](https://slow.pics/c/UdptXCxk) | | [Sol Levante (HQ)](https://slow.pics/c/ySMBddAM) | [Sol Levante (LQ)](https://slow.pics/c/tNrgSJSR) | | [Suzume (HQ)](https://slow.pics/c/9FVioL5x) | [Suzume (LQ)](https://slow.pics/c/WWCDsBGc) | | [The Mandalorian (HQ)](https://slow.pics/c/pOCex4aG) | [The Mandalorian (LQ)](https://slow.pics/c/2IiO8loM) | ##### And `--variance-octile 5`! ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: Cf. previous interpretations. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/xfux0wc3) | [Avatar (LQ)](https://slow.pics/c/4wUXeENY) | | [Ducks (HQ)](https://slow.pics/c/byPkfKGV) | [Ducks (LQ)](https://slow.pics/c/jhr4SelN) | | [Fallout (HQ)](https://slow.pics/c/XlWDEMlG) | [Fallout (LQ)](https://slow.pics/c/PoElCuV3) | | [Minecraft (HQ)](https://slow.pics/c/gFx0OGXz) | [Minecraft (LQ)](https://slow.pics/c/5mFpmbqq) | | [Sol Levante (HQ)](https://slow.pics/c/XcpyeAfl) | [Sol Levante (LQ)](https://slow.pics/c/elhMkf3h) | | [Suzume (HQ)](https://slow.pics/c/pBXnSGgh) | [Suzume (LQ)](https://slow.pics/c/1ZmKPN5Y) | | [The Mandalorian (HQ)](https://slow.pics/c/xSNu2fAK) | [The Mandalorian (LQ)](https://slow.pics/c/ICwJ27DI) | ##### What about `--variance-octile 6`, the default *octile* value in every iteration of SVT-AV1: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: The default octile value tends, again, to perform better on higher strengths at high quality, and sometimes lower strengths at low quality. We notice it is getting easier to distinct between strengths as we've increased octile, which can be good depending on how you see things. Something we haven't discussed until now is how higher strengths tend to be more inconsistent according to SSIMU2 as you get closer to the *CRF40-50* range, which could be problematic depending on your usecase, and thus something to keep in mind. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/uh1kwBCy) | [Avatar (LQ)](https://slow.pics/c/BhXRytzk) | | [Ducks (HQ)](https://slow.pics/c/45p8EDsI) | [Ducks (LQ)](https://slow.pics/c/7xCXSvO2) | | [Fallout (HQ)](https://slow.pics/c/4xqrsZJ9) | [Fallout (LQ)](https://slow.pics/c/lvmSA3XE) | | [Minecraft (HQ)](https://slow.pics/c/w2OOsRQF) | [Minecraft (LQ)](https://slow.pics/c/VCsRWnKr) | | [Sol Levante (HQ)](https://slow.pics/c/9Cbo7ua5) | [Sol Levante (LQ)](https://slow.pics/c/r1tP3s25) | | [Suzume (HQ)](https://slow.pics/c/oZ61IBgw) | [Suzume (LQ)](https://slow.pics/c/UhzGesBH) | | [The Mandalorian (HQ)](https://slow.pics/c/u5atwJ8W) | [The Mandalorian (LQ)](https://slow.pics/c/eXAZ1p0g) | ##### Following with `--variance-octile 7` tests: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: Cf. previous interpretations. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/dPYDMxXt) | [Avatar (LQ)](https://slow.pics/c/1bOEH4Ts) | | [Ducks (HQ)](https://slow.pics/c/22BsOu46) | [Ducks (LQ)](https://slow.pics/c/gzuNDayw) | | [Fallout (HQ)](https://slow.pics/c/LnrHCX4b) | [Fallout (LQ)](https://slow.pics/c/JOTbCI5m) | | [Minecraft (HQ)](https://slow.pics/c/RfMprkEY) | [Minecraft (LQ)](https://slow.pics/c/whW5d55t) | | [Sol Levante (HQ)](https://slow.pics/c/qcR6vp2Q) | [Sol Levante (LQ)](https://slow.pics/c/cGoytwwE) | | [Suzume (HQ)](https://slow.pics/c/f7cPlcI0) | [Suzume (LQ)](https://slow.pics/c/S20TviAL) | | [The Mandalorian (HQ)](https://slow.pics/c/x736Wk54) | [The Mandalorian (LQ)](https://slow.pics/c/f3hZchK5) | ##### And lastly `--variance-octile 8` testing: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: Well, here we are, *octile 8* barely provides any gains and can be (quite) harmful depending on the clip and metric, especially at low qualities, so this can hardly ever be recommended. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/qsJDQuH9) | [Avatar (LQ)](https://slow.pics/c/sSjtrfV7) | | [Ducks (HQ)](https://slow.pics/c/AkPNi9tB) | [Ducks (LQ)](https://slow.pics/c/Ovc74G5T) | | [Fallout (HQ)](https://slow.pics/c/wftx5BS3) | [Fallout (LQ)](https://slow.pics/c/zhaFrhou) | | [Minecraft (HQ)](https://slow.pics/c/DgxYKGC3) | [Minecraft (LQ)](https://slow.pics/c/QUTIHXJo) | | [Sol Levante (HQ)](https://slow.pics/c/H9IjcZdh) | [Sol Levante (LQ)](https://slow.pics/c/D3gA2BNr) | | [Suzume (HQ)](https://slow.pics/c/OfIcrFq3) | [Suzume (LQ)](https://slow.pics/c/o28ioyrm) | | [The Mandalorian (HQ)](https://slow.pics/c/rdZTAUJn) | [The Mandalorian (LQ)](https://slow.pics/c/RVprUbv4) | --- #### Varboost Curve We will be comparing all 3 available varboost curves... ##### ...with the default settings first: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: We observe that all three curves tend to be blurred together, with the notable exception of *curve 2* scoring non-negligibly better on a few clips, consistently across all metrics. Speed-wise, I would argue it's a tie as all three curves are sometimes situationally slightly slower or faster. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/F44iAZvt) | [Avatar (LQ)](https://slow.pics/c/OsPifnca) | | [Ducks (HQ)](https://slow.pics/c/UdPvmsZd) | [Ducks (LQ)](https://slow.pics/c/MWvE2PhS) | | [Fallout (HQ)](https://slow.pics/c/mE4DPv4b) | [Fallout (LQ)](https://slow.pics/c/imXOhRu3) | | [Minecraft (HQ)](https://slow.pics/c/0M03K6Td) | [Minecraft (LQ)](https://slow.pics/c/vtXMiDX4) | | [Sol Levante (HQ)](https://slow.pics/c/0HkpGRVh) | [Sol Levante (LQ)](https://slow.pics/c/eUFlYP4S) | | [Suzume (HQ)](https://slow.pics/c/4pPMQEHl) | [Suzume (LQ)](https://slow.pics/c/tc6t8142) | | [Mandalorian (HQ)](https://slow.pics/c/CrjP7X1t) | [Mandalorian (LQ)](https://slow.pics/c/hdxcVqCC) | ##### I also conducted the tests with a different combination of varboost settings I have used on occasion: ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: The winner is less clear here, this time we can say it's a tie. Such results are why testing with a different combination of settings can be educative. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/PPInVFpH) | [Avatar (LQ)](https://slow.pics/c/3ddsAMj8) | | [Ducks (HQ)](https://slow.pics/c/r4N2mCXD) | [Ducks (LQ)](https://slow.pics/c/eVvOPXBr) | | [Fallout (HQ)](https://slow.pics/c/6EzFC57R) | [Fallout (LQ)](https://slow.pics/c/EUZWdHKu) | | [Minecraft (HQ)](https://slow.pics/c/Gz32ytoJ) | [Minecraft (LQ)](https://slow.pics/c/UKpTJkzI) | | [Sol Levante (HQ)](https://slow.pics/c/DkIz3Ti1) | [Sol Levante (LQ)](https://slow.pics/c/agKlz4Ul) | | [Suzume (HQ)](https://slow.pics/c/1XL15C39) | [Suzume (LQ)](https://slow.pics/c/daIcH6Bh) | | [Mandalorian (HQ)](https://slow.pics/c/52CQl8aI) | [Mandalorian (LQ)](https://slow.pics/c/gFNSbm5v) | --- #### Varboost Conclusion We have seen that, in SVT-AV1 v3.0.x, **__varboost on__ more often than not provides consistent improvements whatever the combination of settings**, though we see better gains at high qualities and/or with higher strengths and lower octiles, less gains at low qualities and/or with lower strengths and higher octiles. > It is best to stay on the **default `--variance-boost-strength 2 --variance-octile 6`** unless you want to hyper-tune for your content, or you want to encode at lower than *CRF20* where I guess a combination like `--variance-boost-strength 3 --variance-octile 3` *can* provide more consistent efficiency gains. `--variance-boost-strength 4` is still clearly better than *no varboost* and sometimes lower strengths too, but it often performs worse than *3*. `--variance-octile 1 & 2` often came out on top, but they may be too aggressive or inflate filesizes too much, so a more conservative value is advised. > My last recommendation would be to remember to **adjust __CRF__ in consequence**, as enabling varboost, increasing the strength and decreasing octile can drastically boost filesizes. If you understand how these graphs work, then you know you aren't compromising anything by increasing *CRF* to compensate for the filesize increase. --- ### `--tune` The original SVT-AV1 implementation offers three tunes to choose from: - **`--tune 0` (VQ - Visual Quality)**: Favors sharper decisions, reducing blur but potentially introducing artifacts. - **`--tune 1` (PSNR - Peak Signal-to-Noise Ratio)**: The **default** in mainline SVT-AV1. - **`--tune 2` (SSIM - Structural Similarity Index Measure)**: The previous efficiency champion. Has the dynamics between tunes changed since we last tested them on SSIMULACRA2 almost a year and a half ago? Let's see! ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: In most cases, on most metrics, *tune 1* comes out on top in efficiency, with *tune 2* sometimes trading blows and even winning. Tune 0 is usually quite a bit behind, except on XPSNR. In terms of speed, all tunes seem to perform closely from one another, without a clear winner. For most users, **sticking with the default (tune 1)** is recommended, as it provides the best balance of everything. However, **tune 0** can be worth experimenting with if you prefer sharper outputs. Just be aware of the potential trade-offs in artifacting. >#### **Specialized Tunes in SVT-AV1 Forks** >The SVT-AV1-PSY based forks introduced new tunes to catter to new usecases. We won't be testing these forks today but I'm mentioning them for reference purposes. Note that the following two `--tune 3` modes are entirely different depending on the encoder variant! >- **`--tune 3` (_SVT-AV1-PSY(EX)_ exclusive)**: A psychovisual enhancement of `--tune 2`, borrowing some features from `--tune 0` and other tweaks. A general-purpose psychovisual tune for a wide range of content. >- **`--tune 3` (_SVT-AV1-HDR_)**: Acts as a *grain-optimized* mode, disabling CDEF, restoration, and temporal filtering while applying aggressive psychovisual adjustments. Best suited for **noisy live-action content**. We may want to confirm our findings with a visual analysis on the encoded samples: ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/KoZYGlBL) | [Avatar (LQ)](https://slow.pics/c/CDMgkOVT) | | [Ducks (HQ)](https://slow.pics/c/eiklaXqz) | [Ducks (LQ)](https://slow.pics/c/kLeCUR5k) | | [Fallout (HQ)](https://slow.pics/c/3lzjEFEX) | [Fallout (LQ)](https://slow.pics/c/u9Wx37CU) | | [Minecraft (HQ)](https://slow.pics/c/gQPbq4j0) | [Minecraft (LQ)](https://slow.pics/c/FGGpyjeZ) | | [Sol Levante (HQ)](https://slow.pics/c/9B9lOQ1c) | [Sol Levante (LQ)](https://slow.pics/c/Nr2DBM1m) | | [Suzume (HQ)](https://slow.pics/c/RTlTjyW4) | [Suzume (LQ)](https://slow.pics/c/kV5XUUi1) | | [Mandalorian (HQ)](https://slow.pics/c/CO81FwdN) | [Mandalorian (LQ)](https://slow.pics/c/IxDW70kW) | Lastly, my own opinion regarding the tunes is that it probably does not matter as much as you think. Again, unless you're willing to hyper-tune, which implies testing parameters each time you encode a new source, no one can guess in advance what's going to be best for that content because it depends on an almost infinite amount of variables. The thing with psychovisual approaches is that metrics may be saying something and your eyes something else. Plus it is wildly subjective: my eyes may disagree with yours, or anyone else's. Don't fret too much over the tune. --- Let's continue with the newly introduced parameters of v3.0.0, starting out with an important feature in the context of AV1: luma bias! ### `--luminance-qp-bias` Anyone familiar with AV1 for long enough is aware that its encoders have struggled on dark scenes forever. As the encoder implementations mature, performance in such scenes naturally improve, however in many cases they persist in allocating insufficient bits to these darker scenes. This setting changes (almost) everything! It effectively applies a dumb qp offset to frames of lower overall brightness. The higher the value, the stronger the effect is. This implementation has one advantage and one weakness: it gives the user control over the bitrate balancing between bright and dark frames, however if only parts of the frame are dark and the rest is fairly bright, it may not fix cases of localized detail loss or blurring. So let's see the effect it can have on efficiency and visuals! ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: On the HQ graphs, we can see no luma-bias tends to score last or at least lower than conservative luma-bias values. On the LQ graphs, we usually observe the opposite, that is to say that no luma-bias tends to barely come out on top, with increasing luma-bias values decreasing efficiency ever so slightly. If we look at performance, the impact of luma-bias is negligible at lower *CRF* values, but gradually increases with higher *CRF* values for the simple reason that the frames whose QP will be reduced will take a bit more time to encode, as is the expected behavior of any encoder. The effect on QP, and thus performance, is going to be more important the higher base *CRF* you start with. Let's look at some BD-rate numbers directly, to try and clarify the graphs: | Luma-Bias Effect | LB0 | LB10 | LB20 | LB30 | LB40 | LB50 | LB60 | LB70 | LB80 | LB90 | LB100 | |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:| | Avatar | 0% | +0.18% | +0.59% | +0.74% | +0.89% | +1.11% | +1.28% | +1.45% | +1.63% | +1.73% | +1.94% | | Ducks | 0% | +1.33% | +2.34% | +3.04% | +3.76% | +4.43% | +5.02% | +5.74% | +6.29% | +6.82% | +7.43% | | Fallout | 0% | -0.31% | +0.79% | +1.29% | +1.84% | +2.56% | +2.72% | +3.04% | +3.40% | +3.73% | +4.06% | | Minecraft | 0% | +0.69% | +1.43% | +1.99% | +2.64% | +3.10% | +3.47% | +3.87% | +4.26% | +4.59% | +5.06% | | Sol Levante | 0% | -0.09% | -0.12% | -0.02% | +0.21% | +0.29% | +0.44% | +0.42% | +0.53% | +0.60% | +0.75% | | Suzume | 0% | -0.45% | -0.43% | -0.34% | -0.31% | -0.30% | -0.21% | -0.11% | -0.07% | +0.02% | +0.08% | | Mandalorian | 0% | +0.57% | +0.91% | +1.29% | +1.56% | +1.80% | +2.24% | +2.30% | +2.53% | +2.78% | +3.03% | Unfortunately, I couldn't easily separate the BD-rate numbers at low quality and the ones at high quality, so we can't exactly reproduce what I interpreted from the graphs earlier. Still, this gives insightful data, which showcase that the BD-rate across the entire quality range is improved the most on the anime clips and tends to be harmful on live action and gaming clips, again across the entire quality range, not when isolating a smaller range where we could make luma-bias look way more appealing. Luma-bias is no magic, it simply offsets a frame QP depending on its average luminance. So unless rate control is badly tuned, the feature is not expected in theory to bring any efficiency improvements. My advice for choosing a luma-bias value is to simply isolate a test sample in your source and test out a few values (like 10, 30, 50...), look at the bitrate balance between dark and bright frames by checking the impact on visuals and filesizes, and finally decide what you prefer. On that note, you cannot go wrong by using a conservative value below 50 in my opinion. Beware that the author of the feature warned me it isn't suited for the PQ transfer used in most UHD blu-rays! Use luma-bias exclusively on SDR and HDR HLG videos. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/NUsH5vl0) | [Avatar (LQ)](https://slow.pics/c/dxkop5sv) | | [Ducks (HQ)](https://slow.pics/c/oWY5nMIr) | [Ducks (LQ)](https://slow.pics/c/Ai0GXaAZ) | | [Fallout (HQ)](https://slow.pics/c/cWT7QhiL) | [Fallout (LQ)](https://slow.pics/c/hskxyERo) | | [Minecraft (HQ)](https://slow.pics/c/yRa03RAb) | [Minecraft (LQ)](https://slow.pics/c/DLp6cOsW) | | [Sol Levante (HQ)](https://slow.pics/c/Z8XcDPxn) | [Sol Levante (LQ)](https://slow.pics/c/kUemyTCw) | | [Suzume (HQ)](https://slow.pics/c/WZPOKhQt) | [Suzume (LQ)](https://slow.pics/c/kc2Zgskv) | | [Mandalorian (HQ)](https://slow.pics/c/D1QSRGhy) | [Mandalorian (LQ)](https://slow.pics/c/eXysNAjI) | --- ### `--sharpness` Sharpness is a straighforward parameter, though it may not do exactly what you'd assume it to do. It does not affect the encodes' clarity but rather impacts the deblocking filter sharpness, which can lead to increased fidelity. ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: Like luma-bias, this feature can only be useful if the deblocking filter isn't properly tuned for all usecases to begin with. On surface, it looks like there is not much room for improvements. It does depend on the metric and clip, but in many of these, all sharpness values perform extremely closely from each another, apart from 3 (a bit lower or higher) and 4+ (usually noticeably lower). Speed wise, negative sharpness values perform closely to the default of 0, and increasing positive values become slower and slower, but as we can see on the x axis, it's rather negligible. Again, there are so many data points that a BD-rate table will help visualize stuff differently: | Sharpness Effect | -7 | -6 | -5 | -4 | -3 | -2 | -1 | 0 | +1 | +2 | +3 | +4 | +5 | +6 | +7 | |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:| | Avatar | 0% | -0.04% | -0.05% | -0.04% | -0.16% | -0.04% | -0.03% | -0.02% | -0.03% | -0.01% | +1.07% | +3.44% | +6.05% | +8.28% | +9.65% | | Ducks | 0% | -0.05% | -0.09% | -0.10% | -0.14% | -0.10% | -0.09% | -0.06% | +0.04% | +0.06% | +2.14% | +8.94% | +14.24% | +18.19% | +20.74% | | Fallout | 0% | -0.05% | -0.07% | -0.13% | -0.16% | -0.15% | -0.13% | -0.13% | +0.01% | +0.01% | +0.99% | +5.62% | +10.23% | +13.68% | +16.00% | | Minecraft | 0% | -0.04% | -0.08% | -0.12% | -0.17% | -0.11% | -0.08% | -0.01% | +0.21% | +0.26% | -1.14% | +0.88% | +3.88% | +5.60% | +6.83% | | Sol Levante | 0% | -0.03% | -0.07% | -0.10% | -0.02% | -0.01% | -0.17% | -0.06% | -0.05% | +0.04% | -3.61% | -3.61% | -2.65% | -1.89% | -1.53% | | Suzume | 0% | -0.03% | -0.07% | -0.15% | -0.13% | -0.17% | -0.20% | -0.13% | -0.75% | -0.77% | -0.46% | +2.73% | +5.87% | +7.89% | +8.91% | | Mandalorian | 0% | -0.02% | -0.04% | -0.01% | -0.17% | -0.13% | -0.06% | -0.20% | -0.95% | -1.07% | -1.86% | -0.49% | +1.09% | +2.16% | +2.65% | The same disclaimer is applicable: BD-rate numbers across such a wide quality range are bound to not be representative of smaller ranges, and doing such average across all 4 metrics could reduce the impact of outliers and make the situation look better than it actually is. Which is why this information is important to pair with the individual graphs, and the visual comparisons. Anyway, on average we can see in the table above that negative sharpness values (-7 to -1) generally show minor BD-rate improvements or neutral effects, while positive sharpness values (+1 to +7) can show decent, consistent BD-rate improvements or neutral effects with 1 & 2, and either bigger gains or significant degradations with 3 and above. We notice that *"Sol Levante"* loves sharpness (any value really), whereas the gaming clips and *"ducks take off"* rather dislike it. So, what effect does sharpness have on visuals: ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/ygCCh0bq) | [Avatar (LQ)](https://slow.pics/c/Y4qtod2v) | | [Ducks (HQ)](https://slow.pics/c/iNLdFEQl) | [Ducks (LQ)](https://slow.pics/c/Ks9N1GI0) | | [Fallout (HQ)](https://slow.pics/c/pI7ijRcy) | [Fallout (LQ)](https://slow.pics/c/E8dfmTz7) | | [Minecraft (HQ)](https://slow.pics/c/gJPslpwr) | [Minecraft (LQ)](https://slow.pics/c/XWc5abmS) | | [Sol Levante (HQ)](https://slow.pics/c/SMMOEbcj) | [Sol Levante (LQ)](https://slow.pics/c/F0invSsk) | | [Suzume (HQ)](https://slow.pics/c/J1esnTPE) | [Suzume (LQ)](https://slow.pics/c/Wc22o61r) | | [Mandalorian (HQ)](https://slow.pics/c/65LP89UX) | [Mandalorian (LQ)](https://slow.pics/c/g0vG4p8B) | --- ### `--tf-strength` Temporal filtering in SVT-AV1 combines information from multiple nearby video frames to create cleaner reference pictures with reduced noise, which helps improve compression quality especially for noisy source material. The feature was often considered too strong and often created unavoidable blocking on keyframes, so we historically disabled temporal filtering. Thankfully, a strength parameter has been introduced which has allowed to tame its effects. ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: On this corpus of clips, we instantly notice *tf-strength 4* performs very poorly and should probably never be used. Lower strengths than the default 3 tend to score increasingly better, though we seem to hit a ceiling below 2. *No temporal filtering* is competitive with low *tf-strength*s, though it still gets beaten slightly at times. There is an interesting outlier in Minecraft where low strengths and especially no tf perform noticeably worse for some reason. The difference in speed between all options is overall negligible. From this, it is recommended at a minimum to **reduce `--tf-strength`** from its default `3` **to 1**, or below, to at least completely eliminate the tf blocking issue. > #### **Additional Parameter in SVT-AV1 Forks** > The SVT-AV1-PSY based forks include an additional `--kf-tf-strength` parameter which decouples tf strength on keyframes, and allows the user to concurrently fix the blocking issue and use a stronger tf strength on all other frames if they want. In mainline SVT-AV1, tf-strength is the same between keyframes and other frames, unless you use tune 0 in which case tf-strength on keyframes will be 1 value lower than on other frames (so for example *1* on keyframes if `--tune 0 --tf-strength 2` is set, the strength being capped at a minimum of 0 in any case). ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/SgZuDVbk) | [Avatar (LQ)](https://slow.pics/c/WHh361VL) | | [Ducks (HQ)](https://slow.pics/c/LbanU7Yl) | [Ducks (LQ)](https://slow.pics/c/pbfa7YFg) | | [Fallout (HQ)](https://slow.pics/c/tL86eXXv) | [Fallout (LQ)](https://slow.pics/c/fVKdbsw4) | | [Minecraft (HQ)](https://slow.pics/c/juHcIEFk) | [Minecraft (LQ)](https://slow.pics/c/9M9u6Srw) | | [Sol Levante (HQ)](https://slow.pics/c/DsXaFYfg) | [Sol Levante (LQ)](https://slow.pics/c/Et2cKCt0) | | [Suzume (HQ)](https://slow.pics/c/LIwgjH4w) | [Suzume (LQ)](https://slow.pics/c/nOLPdxus) | | [Mandalorian (HQ)](https://slow.pics/c/CVqDrlrX) | [Mandalorian (LQ)](https://slow.pics/c/zHxREyku) | --- ### `--lossless` SVT-AV1 finally added a lossless mode in v3.0.0! Until then, aomenc was the only AV1 software encoder capable of doing lossless encoding, as both rav1e and SVT-AV1 wouldn't allow you to set *Q0*/*CRF0*. It probably shouldn't be expected to see better lossless compression out of SVT-AV1, rather, the encoder aims to achieve feature parity with aomenc, but can it do it fast? I initially intended this section to compare aomenc to SVT-AV1, but complications quickly arose from my testing as you'll soon be able to see. Obviously, there is no question of efficiency graphs here as the metrics are expected to always be maxed out with lossless enabled. Anyway, no graphs will be needed here. In fact, SVT-AV1 lossless mode isn't actually *mathematically* lossless. Rather, the feature was designed to reach a PSNR of 100 more often than not (not inf!), as we can see in the [Merge Request](https://gitlab.com/AOMediaCodec/SVT-AV1/-/merge_requests/2312) that introduced the feature. Running `--preset 4 --lossless 1` proves it on my test samples: | "Lossless" Test | PSNR (average) | PSNR (min) | PSNR (max) | |:-:|:-:|:-:|:-:| | Avatar | 96.616006 | 86.680008 | inf | | Ducks | inf | inf | inf | | Fallout | 104.378402 | 92.930594 | inf | | Minecraft | 96.263052 | 87.993916 | inf | | Sol Levante | 119.890723 | 106.868980 | inf | | Suzume | 116.226350 | 102.759654 | inf | | Mandalorian | 102.546893 | 91.381363 | inf | > *XPSNR and W-VMAF are not included here as the numbers were nonsensical due to the chroma weighting at play. As for Butteraugli's different intensity multiplier and SSIMULACRA2's harmonic scoring, they are not expected to affect the score of a lossless frame, however the GPU implementation itself isn't 100% score accurate to their original's so they can't be considered reliable for this specific usecase. Therefore, regular ffmpeg's PSNR filter was used here.* Sure, the scores are high, but not lossless-like! Funnily enough, the anime clips' average score a bit higher. That makes SVT-AV1 more lossless on anime than on live action! Is this confirmation that SVT-AV1 is a weeb encoder? You tell me. Anyway, for some reason, *"ducks take off"* is the only clip that is properly lossless out of SVT-AV1 here. What's even more concerning is that even though the outputs are not mathematically lossless, the filesizes are often bigger than `libx264 -preset veryfast -qp 0`, the latter of which is properly lossless: | "Lossless" Test | x264 Filesize | SVT-AV1 Filesize | |:-:|:-:|:-:| | Avatar | 282,198,409 o | 280,441,784 o (-0.6%) | | Ducks | 244,077,992 o | 234,884,891 o (-3.8%) | | Fallout | 272,838,887 o | 278,457,794 o (+2.1%) | | Minecraft | 382,935,638 o | 473,884,851 o (+23.8%) | | Sol Levante | 438,207,769 o | 448,937,567 o (+2.4%) | | Suzume | 372,844,589 o | 384,223,861 o (+3.1%) | | Mandalorian | 182,526,455 o | 185,680,771 o (+1.7%) | Needless to say the AV1 encodes took at least 10x longer to encode as well! Considering that *"Ducks"* is mathematically lossless and a few percents smaller, it shows that the format is in fact capable of compression gains in this department. Simply, the current iteration is either broken or misleading in its true intentions. While the encodes are not *mathematically* lossless, I doubt anyone can see any difference with our bare eyes, but I still made comps so you can see for yourself. ---> Visual Comparisons: - [Avatar](https://slow.pics/c/B5ITVoCV) - [Ducks](https://slow.pics/c/yAjveP75) - [Fallout](https://slow.pics/c/M9F6GgKB) - [Minecraft](https://slow.pics/c/Hqx6vb5X) - [Sol Levante](https://slow.pics/c/Hqx6vb5X) - [Suzume](https://slow.pics/c/F3wXKV3g) - [Mandalorian](https://slow.pics/c/Makytom1) --- ### `--fast-decode` SVT-AV1 ships with its own built-in method for reducing decoding bottlenecks by smartly tuning down or disabling specific internal tools that trade off some efficiency for decoding performance. The encoder offers two `--fast-decode` levels, with `2` being more aggressive. The default is `0`. Three presets have been tested with the feature to quantify what the effects would be for different usecases. Due to some hardware-related issues, all the encodes in this section were re-run, so the speed numbers of disabled *fast-decode* encodes are different from before, however this ensured there would be no impact on my subsequent analysis. #### Preset 2 ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: The efficiency impact of *fast-decode* is clearly visible, be it at low or high quality targets. Beware of the much different scales on the y-axix between the HQ and LQ graphs, which could mislead you into thinking the effect at low quality is lesser, even though that's not the case. We observe that *fast-decode* can non-negligibly influence the speed of your encoding instances, with **1** appearing slower than *0* and **2** being faster than both. There's an exception in *"Sol Levante"* where *0* and *1* perform the same. Let's look at the effects on decoding speed with data I nicely aggregated into tables! >Decoding speeds were collected using ffmpeg 7.1.0 from Arch's official repository using: `ffmpeg -hide_banner -benchmark -i "" -f null - > /dev/null`. The tests were repeated 5 times and the performance numbers you'll see are the average of these 5 runs. I can say with confidence the speed deviation was *largely* negligible, but better safe than sorry! | Preset 2 Decoding Test (CRF12) | FD0 | FD1 | FD2 | |:-:|:-:|:-:|:-:| | Avatar | 197fps | 204fps *(+3.6%)* | 212fps *(+7.6%)* | | Ducks | 118fps | 126fps *(+6.8%)* | 125fps *(+5.9%)* | | Fallout | 65fps | 67fps *(+3.1%)* | 68fps *(+4.6%)* | | Minecraft | 82fps | 85fps *(+3.7%)* | 91fps *(+11.0%)* | | Sol Levante | 98fps | 105fps *(+7.1%)* | 107fps *(+9.2%)* | | Suzume | 360fps | 401fps *(+11.4%)* | 397fps *(+10.3%)* | | Mandalorian | 326fps | 348fps *(+6.7%)* | 340fps *(+4.3%)* | In our *high quality* target, `--fast-decode 1` provides a `+6%` decoding performance increase on average, while `--fast-decode 2` provides `+7.5%`. | Preset 2 Decoding Test (CRF33) | FD0 | FD1 | FD2 | |:-:|:-:|:-:|:-:| | Avatar | 429fps | 465fps *(+8.4%)* | 482fps *(+12.4%)* | | Ducks | 341fps | 366fps *(+7.3%)* | 342fps *(+0.3%)* | | Fallout | 131fps | 140fps *(+6.9%)* | 140fps *(+6.9%)* | | Minecraft | 182fps | 211fps *(+15.9%)* | 217fps *(+19.2%)* | | Sol Levante | 194fps | 211fps *(+8.8%)* | 210fps *(+8.2%)* | | Suzume | 684fps | 766fps *(+12.0%)* | 770fps *(+12.6%)* | | Mandalorian | 885fps | 912fps *(+3.1%)* | 946fps *(+6.9%)* | At *CRF33*, `--fast-decode 1`'s decoding speeds were faster of about `+8.9%` and `--fast-decode 2` of about `+9.5%`. Sure, `--fast-decode 2` doesn't provide substancial decoding benefits over `1`, heck even over `0` at times, however it also acts as a "fast-encode" parameter, so it can be argued it has its purpose. Before we continue with the visual comparisons, I will add that we are not on a low powered device, so the importance of such gains may not be immediately visible nor perfectly accurate to be honest. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/mHlwNgCj) | [Avatar (LQ)](https://slow.pics/c/3W7ELwA1) | | [Ducks (HQ)](https://slow.pics/c/BeiJQRSs) | [Ducks (LQ)](https://slow.pics/c/rj4jFQGb) | | [Fallout (HQ)](https://slow.pics/c/WPqNeTjN) | [Fallout (LQ)](https://slow.pics/c/ZcKpdvBR) | | [Minecraft (HQ)](https://slow.pics/c/1ElnTugb) | [Minecraft (LQ)](https://slow.pics/c/ODFp1HmP) | | [Sol Levante (HQ)](https://slow.pics/c/E84J2Jdc) | [Sol Levante (LQ)](https://slow.pics/c/xKIDF42q) | | [Suzume (HQ)](https://slow.pics/c/vaQ57EY2) | [Suzume (LQ)](https://slow.pics/c/UJCsXDmV) | | [Mandalorian (HQ)](https://slow.pics/c/Qy1roRVu) | [Mandalorian (LQ)](https://slow.pics/c/vwAg1glu) | --- #### Preset 4 ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: It can be observed the gaming clips's efficiency is less affected by the *fast-decode* feature, if at all. Even then, `--fast-decode 2` continue to provide encoding performance benefits. As we move to faster presets, the resulting streams are expected to be less complex and thus easier to decode. Is that assumption correct? And if so, does it imply that *fast-decode* will have a smaller impact when using preset 4? | Preset 4 Decoding Test (CRF12) | FD0 | FD1 | FD2 | |:-:|:-:|:-:|:-:| | Avatar | 185fps | 202fps *(+9.2%)* | 203fps *(+9.7%)* | | Ducks | 117fps | 126fps *(+7.7%)* | 121fps *(+3.4%)* | | Fallout | 67fps | 71fps *(+6.0%)* | 70fps *(+4.5%)* | | Minecraft | 87fps | 95fps *(+9.2%)* | 94fps *(+8.0%)* | | Sol Levante | 91fps | 100fps *(+9.9%)* | 99fps *(+8.8%)* | | Suzume | 324fps | 372fps *(+14.8%)* | 360fps *(+11.1%)* | | Mandalorian | 312fps | 336fps *(+7.7%)* | 326fps *(+4.5%)* | Looking at the FD0 column of the below and above table, compared to the *preset 2* ones, my claim seems to barely apply and only consistently at low quality levels at that. We'll see how it goes for *preset 6* in the next sub-section. Anyway, the average decoding speed impact at *HQ* is as follow: - `+9.2%` for `--fast-decode 1`, - `+7.1%` for `--fast-decode 2`. | Preset 4 Decoding Test (CRF33) | FD0 | FD1 | FD2 | |:-:|:-:|:-:|:-:| | Avatar | 432fps | 494fps *(+14.4%)* | 499fps *(+15.5%)* | | Ducks | 354fps | 394fps *(+11.3%)* | 375fps *(+5.9%)* | | Fallout | 143fps | 156fps *(+9.1%)* | 146fps *(+2.1%)* | | Minecraft | 197fps | 258fps *(+31.0%)* | 254fps *(+28.9%)* | | Sol Levante | 200fps | 224fps *(+12.0%)* | 217fps *(+8.5%)* | | Suzume | 696fps | 811fps *(+16.5%)* | 763fps *(+9.6%)* | | Mandalorian | 902fps | 981fps *(+8.8%)* | 999fps *(+10.8%)* | `--fast-decode 1` gives a hefty `+14.7%` increase in decoding performance at *lower quality* targets, as certain clips like *"Minecraft"* greatly benefit from it. `--fast-decode 2` brings a more modest `+11.6%` improvement. The trade-offs are quite a bit different than at *preset 2* which is quite fascinating, but `--preset 4` seems to draw more benefit from *fast-decode*! ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/z063QyOe) | [Avatar (LQ)](https://slow.pics/c/fjOkslFE) | | [Ducks (HQ)](https://slow.pics/c/mUdjpzQE) | [Ducks (LQ)](https://slow.pics/c/PJFYyxwC) | | [Fallout (HQ)](https://slow.pics/c/pBUEwMuo) | [Fallout (LQ)](https://slow.pics/c/Rk9yKcOL) | | [Minecraft (HQ)](https://slow.pics/c/0M6hKQxg) | [Minecraft (LQ)](https://slow.pics/c/ZNovZDlt) | | [Sol Levante (HQ)](https://slow.pics/c/sodLtXq2) | [Sol Levante (LQ)](https://slow.pics/c/5Oqc0ljs) | | [Suzume (HQ)](https://slow.pics/c/IdFzXbrh) | [Suzume (LQ)](https://slow.pics/c/pUqrSDSp) | | [Mandalorian (HQ)](https://slow.pics/c/HvpNGjj7) | [Mandalorian (LQ)](https://slow.pics/c/JKLlXygr) | --- #### Preset 6 ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: The efficiency situation here is the same as on *preset 4*, however we notice `--fast-decode 1` is no longer slower than disabled. Instead, it places itself in-between *0* and *2* in terms of performance, offering another kind of trade-off again! What influence on decoding speeds can we expect at `--preset 6`? | Preset 6 Decoding Test (CRF12) | FD0 | FD1 | FD2 | |:-:|:-:|:-:|:-:| | Avatar | 218fps | 216fps *(-0.9%)* | 228fps *(+4.6%)* | | Ducks | 128fps | 127fps *(-0.8%)* | 130fps *(+1.6%)* | | Fallout | 76fps | 74fps *(-2.6%)* | 77fps *(+1.3%)* | | Minecraft | 110fps | 106fps *(-3.6%)* | 112fps *(+1.8%)* | | Sol Levante | 100fps | 104fps *(+4.0%)* | 107fps *(+7.0%)* | | Suzume | 382fps | 381fps *(-0.3%)* | 394fps *(+3.1%)* | | Mandalorian | 343fps | 343fps *(0.0%)* | 357fps *(+4.1%)* | This time, be it at high or low quality, the base decoding speed without fast-decode is almost always faster. At *CRF12*, `--fast-decode 1` and `--fast-decode 2` respectively deliver a `-0.6%` decoding speed regression and a `+3.4%` speed improvement. | Preset 6 Decoding Test (CRF33) | FD0 | FD1 | FD2 | |:-:|:-:|:-:|:-:| | Avatar | 468fps | 480fps *(+2.6%)* | 508fps *(+8.5%)* | | Ducks | 358fps | 364fps *(+1.7%)* | 382fps *(+6.7%)* | | Fallout | 169fps | 168fps *(-0.6%)* | 177fps *(+4.7%)* | | Minecraft | 278fps | 303fps *(+9.0%)* | 335fps *(+20.5%)* | | Sol Levante | 209fps | 215fps *(+2.9%)* | 221fps *(+5.7%)* | | Suzume | 667fps | 694fps *(+4.0%)* | 715fps *(+7.2%)* | | Mandalorian | 878fps | 912fps *(+3.9%)* | 950fps *(+8.2%)* | The gains at low qualities are more appealing, at `+3.4%` for `--fast-decode 1` and `+8.8%` for `--fast-decode 2`. `--preset 6` benefits less from *fast-decode*, especially mode *1*, probably due to a shift in the decoding bottlenecks past this point. ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/P1ghVYLa) | [Avatar (LQ)](https://slow.pics/c/FtUPq8PL) | | [Ducks (HQ)](https://slow.pics/c/sNribSBQ) | [Ducks (LQ)](https://slow.pics/c/MYYl1GBK) | | [Fallout (HQ)](https://slow.pics/c/3GXE1MYP) | [Fallout (LQ)](https://slow.pics/c/YtIF51Xn) | | [Minecraft (HQ)](https://slow.pics/c/ccPFfGsI) | [Minecraft (LQ)](https://slow.pics/c/3hsSuzSD) | | [Sol Levante (HQ)](https://slow.pics/c/0MBOG9TK) | [Sol Levante (LQ)](https://slow.pics/c/NOIjeZkg) | | [Suzume (HQ)](https://slow.pics/c/6ghzI8QM) | [Suzume (LQ)](https://slow.pics/c/ktntjOqm) | | [Mandalorian (HQ)](https://slow.pics/c/0hv7XLQv) | [Mandalorian (LQ)](https://slow.pics/c/kVc5GHuR) | --- #### Fast Decode Conclusion In performance constrained scenarios, like low power ARM devices, fast-decode could come in handy to help smooth out your playback experience, at the possible cost of some efficiency. Keep in mind it has been observed that the output of *fast-decode* modes can be more prone to macro-blocking depending on source characteristics, so proceed with caution. --- ### Tiles AV1 tiles are a straightforward method of splitting the video frame into independent tiles of equal size to hopefully increase encoding and decoding thread-ability. In SVT-AV1, tiles don't increase encoding speeds but they can help devices (especially low-powered ones) to software decode AV1 more easily. We are going to challenge these claims. > *In the following graphs and tables, `c` refers to `--tile-columns` and `r` refers to `--tile-rows`. For instance, `c2r1` can be read as `--tile-columns 2 --tile-rows 1`.* ---> Compression efficiency graphs, *"high quality"* range (CRF10 -> 23): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Compression efficiency graphs, *"low quality"* range (CRF28 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-fourth-deep-dive-p2.json). ---> Interpretation: Except on *"Suzume"*, and to some extent on *"Mandalorian"* at low qualities, where we see the higher number of tiles start to impact efficiency, the effect of tiles on efficiency is by in large negligible. It does seem like encoding speeds aren't particularly impacted by tiles in this encoder. Let's also look at decoding speeds, as we did with *fast-decode*! What follows are the relative decoding speed improvements relative to `c0r0`, starting with tests at *CRF12*: | Tiles Decoding Test (CRF12) | c0r0 | c1r0 | c1r1 | c2r0 | c2r1 | |:-:|:-:|:-:|:-:|:-:|:-:| | Avatar | 185fps | 335fps *(+81%)* | 446fps *(+141%)* | 477fps *(+158%)* | 487fps *(+163%)* | | Ducks | 117fps | 224fps *(+91%)* | 330fps *(+182%)* | 348fps *(+197%)* | 410fps *(+250%)* | | Fallout | 67fps | 125fps *(+87%)* | 186fps *(+178%)* | 202fps *(+201%)* | 223fps *(+233%)* | | Minecraft | 87fps | 164fps *(+89%)* | 223fps *(+156%)* | 256fps *(+194%)* | 267fps *(+207%)* | | Sol Levante | 91fps | 162fps *(+78%)* | 205fps *(+125%)* | 219fps *(+141%)* | 217fps *(+138%)* | | Suzume | 324fps | 544fps *(+68%)* | 679fps *(+110%)* | 732fps *(+126%)* | 729fps *(+125%)* | | Mandalorian | 312fps | 520fps *(+67%)* | 631fps *(+102%)* | 681fps *(+118%)* | 688fps *(+121%)* | The same tests were conducted at *CRF33* to quantify the differences in impact between two very distinctive quality targets: | Tiles Decoding Test (CRF33) | c0r0 | c1r0 | c1r1 | c2r0 | c2r1 | |:-:|:-:|:-:|:-:|:-:|:-:| | Avatar | 432fps | 698fps *(+62%)* | 780fps *(+81%)* | 816fps *(+89%)* | 758fps *(+75%)* | | Ducks | 354fps | 650fps *(+84%)* | 845fps *(+139%)* | 860fps *(+143%)* | 920fps *(+160%)* | | Fallout | 143fps | 259fps *(+81%)* | 355fps *(+148%)* | 375fps *(+162%)* | 386fps *(+170%)* | | Minecraft | 197fps | 346fps *(+76%)* | 432fps *(+119%)* | 489fps *(+148%)* | 481fps *(+144%)* | | Sol Levante | 200fps | 295fps *(+48%)* | 285fps *(+43%)* | 312fps *(+56%)* | 292fps *(+46%)* | | Suzume | 696fps | 991fps *(+42%)* | 1020fps *(+47%)* | 1082fps *(+55%)* | 1008fps *(+45%)* | | Mandalorian | 902fps | 1183fps *(+31%)* | 1164fps *(+29%)* | 1207fps *(+34%)* | 1128fps *(+25%)* | No, you aren't dreaming! We immediately realize that the impact of tiles is significantly greater than either of the *fast-decode* modes, an especially appealing outcome given their relatively minor effect on compression efficiency. Decoding speeds improve even further at higher quality settings, with the smallest gain reaching **+67%** at *CRF12*. That kind of difference can turn an unplayable file into one that runs smoothly. A single additional tile can have important consequences. *Tile-rows* on average offer less benefit than *tile-columns*, and in some cases even cause slight regressions at lower qualities. Still, seeing up to +250% improvements in decoding speed is nothing to scoff at. [Dav1d](https://code.videolan.org/videolan/dav1d/)’s performance is genuinely impressive, now exceeding **1000fps** on certain clips! ---> Visual Comparisons: | HQ | LQ | |:-:|:-:| | [Avatar (HQ)](https://slow.pics/c/6ZctTOwN) | [Avatar (LQ)](https://slow.pics/c/bHKBXQsC) | | [Ducks (HQ)](https://slow.pics/c/psvcfJR0) | [Ducks (LQ)](https://slow.pics/c/dYIJoDX9) | | [Fallout (HQ)](https://slow.pics/c/9cImaJgO) | [Fallout (LQ)](https://slow.pics/c/lfD5I54d) | | [Minecraft (HQ)](https://slow.pics/c/TG61F71o) | [Minecraft (LQ)](https://slow.pics/c/HVRceUiN) | | [Sol Levante (HQ)](https://slow.pics/c/rWdmd8dP) | [Sol Levante (LQ)](https://slow.pics/c/I9H4Akva) | | [Suzume (HQ)](https://slow.pics/c/uC13Y3J7) | [Suzume (LQ)](https://slow.pics/c/zL4abWSW) | | [Mandalorian (HQ)](https://slow.pics/c/cnUI9sTu) | [Mandalorian (LQ)](https://slow.pics/c/5WlGC4R6) | If I had to give recommendations based on what we saw, for a good balance between losses in efficiency and increased decoding performance, consider the following tile settings: - `--tile-columns 1 --tile-rows 0`: for 1080p and above - `--tile-columns 2 --tile-rows 0`: for 4K and above Of course, if decoding speed isn’t a concern at all, you can stick with the default `--tile-columns 0 --tile-rows 0`. But even then, I believe enabling tiles is worth considering for future-proofing purposes. One can combine *fast-decode* and tiles to decrease decoding complexity further. ## Statistics Who doesn't love a bunch of random and useless stats? Well, if you recognize yourself in this, I got you. I compiled a few, though I only included stuff that ended up being used in these two parts. Yes, test encodes, test graphs or test comps figures are *not* included here. Here's this deep dive in numbers: - **10577** total AV1 encodes... - ...which account for a total size of **294GB**... - ...which makes for an average of about **28MB** per file - **3928** total graphs - **2030** total decoding runs - **483** slow.pics comps... - ...for a grand total of about *12000* distinct screenshots uploaded! Even I was flabbergasted when I discovered the actual scope of this benchmarking session! ## Conclusion In this second part, we took a deep dive into several key SVT-AV1 encoding parameters, re-evaluating them in today's context. The goal was mainly to explore how these settings impact efficiency and encoding speeds, but at times also decoding performance or practical usability. While I could have revisited even more parameters, the sheer time investment required means I had to draw the line somewhere. I made sure to go **really in-depth** with the ones that mattered most in my opinion. As always, your mileage may vary regarding any kind of speed figures. The key is to **test things yourself**, with your own clips, workflows, and goals in mind. What works for one setup might not suit another, and that’s what makes this kind of testing both challenging... and somewhat rewarding too. This wraps up Part 2, but there’s more to come. I'm always thinking about what to explore next, and your feedback helps shape that direction. So if you have suggestions, requests, or thoughts to share, I’d love to hear them, in the usual places. Thanks again for reading, and I hope you found this deep dive insightful! ## Future ~~First of all, I do not consider this testing complete. I have been told some comps had issues, and while re-generating them, I ended up rate-limited by slow.pics again, though only temporarily fortunately. As you can imagine, even with scripts, preparing and uploading these takes quite a while already, but I'm taking even more precautions to avoid getting banned again. I decided not to delay this Part further, so you wouldn't have to wait more, but I'll ask you to be patient a bit longer for me to add the fixed comps for Part 1 and the varboost octiles comps for this Part. I expect to be able to update both articles before the end of the week.~~ **EDIT (July 4):** These concerns have been addressed. Thanks for your patience. As I alluded to in the last section, I have ideas for future blog posts. First of all, **SVT-AV1 v3.1.0 is right around the corner**, and it is touted to be quite an update especially for VBR and realtime usecases! It could be the occasion for me to do the long awaited *target bitrate* tests and compare efficiency with CRF on our test samples... Second, most of the ideas I proposed in past articles remain valid possibilities and I still want to test other AV1 encoders in this format. AV2 is approaching too so it would be interesting to make some early comparisons of AV1 with AVM, if there's enough time for that. Third, there's still so much that could be improved in my methodology, to make it more robust and precise, so I will continue experimenting as usual to find a better overall formula! Lastly, I’ve also been working on a few AV1-related projects behind the scenes, including one that involves SVT-AV1 directly... I’m looking forward to sharing more about it in the future, so stay tuned! I want to extend my thanks a second time to the people who contributed, directly or indirectly, to the making of this article, including [Gianni Rosato (gb82)](https://github.com/gianni-rosato), [Line (Lumen)](https://codeberg.org/Line-fr), [Soda](https://github.com/GreatValueCreamSoda), [Emre](https://github.com/emrakyz), [Bolu](https://github.com/0xb01u/), [Julio Barba](https://github.com/juliobbv-p), the people behind slow.pics for hosting thousands of screenshots each time I make these articles, the SVT-AV1 dev team for the work on this amazing encoder, and probably others I'm forgetting... --- *Consider supporting me by making a donation on [my Ko-Fi page](https://ko-fi.com/nekotrix).* {} --- # Observing SVT-AV1 v2.1.0's improvements: A New Deep Dive Canonical URL: https://encode.wiki/articles/svt-av1-second-deep-dive/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: c7c4326f63e8aadab45acae69ff7cf8e54ce7a3d7b78e8a5f4383c12de5c5c53 # Introduction SVT-AV1, the most scalable AV1 encoder, has received a new update and one may wonder if the old presets recommendation still holds today. We will delve into that in this blog post, based on a series of speed and visual quality benchmarks with SSIMULACRA2 and XPSNR of SVT-AV1 2.1.0 on a corpus of varied animated clips. ## Feedback I unfortunately never got to update the previous blog post with the image comparisons, and some people expressed concerns that this testing may not be representative of live action content. For the former, I will have to ask for your patience again, because this blog post won't contain image comparisons either, but they are being actively worked on. As for the latter, please be reassured that this testing in its entirety is perfectly representative of any modern content people typically encode: the diversity of japanese animation is rich and the content specifically chosen for this benchmark is relatively complex. From 3DCG to extremely noisy clips, we are far from the easy-to-compress static scenes of some slice-of-life show. I have also decided to complement this benchmark of another psychovisually-driven metric (XPSNR) so that double-checking is made easier. Thus, each graph possesses a SSIMULACRA2 version and a XPSNR version. Don't hesitate to switch between one another! ## Methodology The resources available will range from ***graphs*** to ~~**image comparisons**~~ (WIP). The ***former*** has the advantage of being easily understandable, showcasing pure efficiency comparisons between encoder parameters using metrics as the reference, while the **latter** are image samples from the files encoded during the tests that enable you to check quality for yourself, adding another layer of subjective interpretation to these comparisons. The testing methodology involves using relatively short video samples from a wide range of modern anime genre, which have been either losslessly encoded with `x264 --qp 0` for ease of use or losslessly cut from their source. These lossless files are then pipped into SvtAv1EncApp *directly*, meaning we are measuring the performance of a single encoder instance and not leveraging chunked encoding like any actual final AV1 encoding pipeline should. Once an encode is done, SSIMULACRA2 scores are calculated using the [Zig implementation](https://github.com/dnjulek/vapoursynth-ssimulacra2), XPSNR scores on the other hand are calculated using a [ffmpeg filter](https://github.com/fraunhoferhhi/xpsnr), and lots of useful data are aggregated to make the graphs for this benchmark, including encoding time, encode size (bitrate), and metrics scores. Bits per pixel scores (BPP) are calculated so that the `Metric / BPP` graphs may represent the closest we have to real efficiency. The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy. SvtAv1EncApp was compiled directly from the [v2.0.0 and v2.1.0 source code](https://gitlab.com/AOMediaCodec/SVT-AV1/-/releases/) using the provided `Build/linux/build.sh` script, Clang 16.0.6, and Profile-Guided Optimization (PGO). The testing machine is comprised of an i3 12100 with 16GB of 3200MHz CL14 DDR4 RAM in Arch Linux with kernel 6.7.7 and the performance governor enabled. All encodes have been made in the same session without rebooting. ## Samples The samples are as follows: - 11s `Blame!` clip which sports 3DCG action with lots of grain, effects and high-contrast elements. - 13s `Blue Lock` clip which sports rapid camera movements, complex geometry and high-contrast elements. - 5s `Spy x Family` first ending sequence with an extremely high amount of dynamic noise. New most complex source of this set. - 12s `Jigokuraku (Hell's Paradise)` flashback clip with huge static grain in a very dark scenery and some action. - 5s `The Garden of Sinners` clean but fast-paced 3DCG scene with explosions. The resolution of every clip is 1080p, except for the first one which is 1920x804. > **All clips have been encoded in a wide quality range, from `--crf 6` to `--crf 46`, by increments of 4.** Without further ado, let's start with the first comparisons! ## Presets comparisons (-1 -> ~~13~~ 12) **In the following graphs, you may find comparisons between all SVT-AV1 presets, ranging from the slowest `--preset -1` to the fastest `--preset 12`.** > Yes, you heard that right. Preset 7 and 13 are no more in __v2.1.0__. This new update, like the previous one, mostly consisted of optimizing the presets trade-offs. The devs have made the choice to map `preset 7` to `preset 6` and `preset 13` to `preset 12` due to the lack of spacing between the new presets. We will discuss the implications of this further ahead. `--preset X` is the only parameter used here, in conjunction with the CRF values. That means everything else is default. The defaults worth mentioning are: - `--tune 1`: tune PSNR - `--aq-mode 2`: variance deltaq - `--enable-qm 0`: quantisation matrices disabled - `--irefresh-type 2`: closed GOP - `--enable-tf 1`: temporal filtering enabled And more, like CDEF and restoration enabled, overlays and film-grain disabled... ### Efficiency - First of all, here are the full efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). This is all very cool, but visually bloated. - Now the same graphs but focusing on the "high quality" range (CRF6 -> 22): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Same, but now focusing on the "low quality" range (CRF26 -> 46): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - If we now focus on __presets 4__ and below, where it's more difficult to discern the differences between presets, we get this at "high quality": > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - And the following at "low quality": > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). ### Speed - **Let's now see speed comparisons between all presets:** > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Once is not custom, __preset -1__ is so abysmally slow it makes the graph unusable. - Same, but without the placebo __preset -1__: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Lastly, here is what it looks like with a logarithmic scale: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). ### Interpretation As for interpreting the results, it would seem like **preset 2** and **preset 4** remain all-around very balanced presets, with **preset 3** being in a nice in-between spot, an improvement over v2.0.0's **preset 3** in a way. We will better understand the reasons for this when we'll compare the new version to the last, a bit after. The quality gap between **preset 2** and **preset 1** is usually pretty narrow, however the speed penalty from going to **preset 1** is ~2x, when the penalty of going from **preset 3** to **preset 2** is closer to ~1.5x. As such, **preset 1** enters placebo territory, and considering the very little benefits of going any lower than it, compared to the huge performance loss of even lower presets, I advise you not to waste encoding resources on **preset 0** and **preset -1**. This applies especially at medium to high quality, however at extremely low quality like the CRF40 range, we can still see some small gains from these placebo presets. When we start talking about faster presets though, things are pretty different from previous versions: **presets 5 to 9** behave similarly on the graphs and seem to stand apart from their slower counterparts by just a bit. If you can bear the speed of **preset 4**, you should definitely be going for it, however if fast encoding is a necessity, for example in the case of realtime transcoding or streaming, **presets 5 through 9** will serve you right with great efficiency/speed trade-offs between one another. No preset in that range particularly stands out from the others, so simply pick one depending on your performance needs. **Presets 10 to 12** are pretty inefficient, and to be avoided if possible. They can still be of use in a convex-hull scenario, but in the case of realtime transcoding, you may be better off with a hardware encoder like the ones found in RTX 4000 or Arc GPUs, especially since SVT-AV1's target bitrate mode is even less efficient than CRF mode. ### TLDR The same conclusions as the previous blog post can be made: **clear quality gains can be observed as we decrease presets, until __preset 2__, however the effectiveness of dropping presets is noticeably less and less important as quality is increased.** In the next part, we will evaluate the differences in efficiency and speed of every presets when updating from SVT-AV1 2.0.0 to 2.1.0, which should enable an increase of nuance from the previous results alone. ## SVT-AV1 v2.0.0 vs v2.1.0 presets comparisons: Two months ago, I conducted a similar test to this one to compare the presets evolution between versions __1.8.0__ and __2.0.0__. The results were pretty unsatisfying: I noticed that **presets -1 to 8** in __v2.0.0__ performed like the old **presets 0 to 9** did in __v1.8.0__. We basically saw an efficiency regression at a given preset, and speedups did not follow suit as well as we would have anticipated. All in all, it wasn't all that bad, it suffice to say you could simply drop a preset from before and you were good to go again. What was more concerning however is that the release note claimed important speedups that did not impact efficiency and my testing proved otherwise. My theory is that due to the dev team testing methodology, which consist of mostly pretty low resolution clips and non-psychovisual metrics like PSNR, SSIM or bad psychovisual metrics like VMAF, it's very well possible they were tricked into thinking they introduced improvements as they tweaked the presets when in reality the metrics simply didn't notice the quality degradation. Such issue is an additional reason why the industry should adopt more competent metrics, ones that better correlate with the human vision, to improve encoders in more impactful ways and better avoid pointless regressions. So the question for today's testing is: have the SVT-AV1 devs redeemed themselves and actually improved the presets trade-offs this time around? Let's find out! ### `preset -1`: v2.0.0 vs v2.1.0 - **Let's start off with a battle of the placebos, with the efficiency at "high quality":** > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - And the efficiency at "low quality": > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Yes, this is a bit underwhelming, but you can't just improve the best an encoder has to offer with just tweaking right? - Now, let's compare their respective speeds: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Let's be grateful it became ever so slightly faster, I guess. ### `preset 0`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Overall, efficiency wise, this new **preset 0** places itself in-between old **preset -1** and **0** - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Interestingly enough, its speed is much closer to the old **preset 0** than to the old **preset -1**. This means **preset 0** was genuinely improved over v2.0.0! ### `preset 1`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). In efficiency, this new **preset 1** is often equal to old **preset 0**, else in-between old **preset 0 and 1**. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). We observe that the new preset is a bit closer to old **preset 1** speeds than it is to old **preset 0** speeds. Good news! ### `preset 2`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Oh well, that's awkward. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Speed was left untouched too, meaning **preset 2** is unchanged in v2.1.0. ### `preset 3`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). The new **preset 3**'s efficiency is the same as the old one. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). However, the preset got slightly faster, so this is a speedup! ### `preset 4`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). We can observe that **preset 4** got slightly to moderately worse efficiency wise. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Fortunately, the consequence of that slight efficiency decrease is a big performance improvement! ### `preset 5`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). **Preset 5** seems to have gotten ever so slightly worse efficiency wise. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Yet it became slightly faster, this is overall a good trade-off. ### `preset 6`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). The new **preset 6** has a huge responsibility: being able to compensate in the absence of its **preset 7** sibling. It seems to performs in-between old **preset 6 and 7**, usually closer to old **7**. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). **Preset 6** is now ever so slightly slower to old **7**, this is an interesting trade-off, overall a win over old **7**. ### `preset 7`: v2.0.0 vs v2.1.0 Again, there is no preset 7. Actually, it's preset 6 that disappeared but I'm not remaking the graphs just for fun. If you select preset 6, you will be granted the following message: `Svt[warn]: Preset M6 is mapped to M7.` ![always_has_been](/img/preset_7_meme.webp) ### `preset 8`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). In efficiency, this new **preset 8** is sometimes equal or slightly worse to the old **8**, and sometimes equal or slightly worse than old **7**... - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Overall, the speed is pretty much unchanged from old **8**. It looks like a slight regression, that's pretty disappointing. ### `preset 9`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). The new **preset 9** is the same as ever, ever so slightly better in some scenario but nothing groundbreaking. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Its speed remains the same, sometimes ever so slightly slower. Basically the preset is pretty much unchanged, which may as well be a relief, as the last usable preset of the encoder. ### `preset 10`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). **Preset 10** is slightly to moderately worse efficiency wise. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Its speed is mostly the same, sometimes ever so slightly faster. It's a wash, avoid this preset at all costs! ### `preset 11`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). **Preset 11**'s efficiency is untouched. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). **Preset 11**'s speed is unchanged as well. ### `preset 12`: v2.0.0 vs v2.1.0 - Efficiency graphs, high quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). - Efficiency graphs, low quality: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). Just as **preset 6**, **preset 12** is now mapped to **13**, and unsurprisingly, its efficiency is equal to old **13**. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-second-deep-dive.json). The speeds seem to be in-between old 12 and 13, so potentially a slight speedup. Still, nothing to get excited at. ### TLDR From these extensive comparisons, it appears that some presets have received genuine improvements in their respective efficiency/speed trade-off. Some presets, like **-1**, **0**, **1** and **3**, received the most significant improvements, followed by **4**, **5** and **6** with overall beneficial new trade-offs. **Preset 12** got slightly faster too. On the other hand, **presets 8 and 10** seemed to have regressed slightly, and **presets 2 and 9** are perfectly unchanged from v2.0.0. ## Conclusion SVT-AV1 2.1.0 introduced some welcomed improvements. **Presets 2 through 4** remain the king of optimal AV1 encoding, while **presets 5 through 9** stand as good options for the people that find **2-4** to be too slow for their liking. Let's be honest a second, not much as changed in SVT-AV1 since the first blog post. There was no need to redo all the parameter testing for the simple reason that their behavior remained the same, as did the conclusions drawn from them. I hope this article wasn't disappointing in a sense... Still, be reassured, this was just an appetizer, there will be more in the near future! By the way, did you know that the **[SVT-AV1-PSY project](https://github.com/gianni-rosato/svt-av1-psy/)** was initiated a few months ago? Its defaults were tailored according to the testing done in the last blog post, allowing a free efficiency boost for anyone not keen to tweak their encoders. Furthermore, SVT-AV1-PSY introduced a sharpness parameter to control distortion, a quarter-step quantizer for more CRF precision, a new subjective SSIM tune, Dolby Vision support, frame luma bias, and some other knobs to improve the appeal and consistency of your encodes. It is actively maintained by a group of talented people, including the main dev of the aom-av1-lavish fork of aomenc. Some of the changes are being backported to mainline SVT-AV1 due to the increased interest of the mainline devs. Please check it out! Hopefully, this comprehensive second deep dive should give you a helpful new starting point for choosing settings when encoding with the latest SVT-AV1(-PSY) 2.1.0. ## Future My plans for the future regarding the blog post include: - polishing this blog post. - a follow-up article in the relatively near future about giving you encoding tips and explaining common AV1 encoding knowlegde, for instance showcasing why film grain synthesis is a game-changer or why chunked encoding can prove beneficial to your encoding pipeline. - an article focused on observing the evolution of SVT-AV1 since the beginning of its development, as well as comparisons with current aomenc, rav1e and SVT-AV1-PSY, including a quick look at the current state of AVM (development ground for AV2) in comparison to VVC's state. Thanks for reading! {} --- # Better late than never: SVT-AV1 v2.2.x Deep Dive Canonical URL: https://encode.wiki/articles/svt-av1-third-deep-dive/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: 367eab4d9b2ca7646c5087c7a639ae964c99657da7e471e73f2d96c16f8e9c7c # Introduction SVT-AV1 v2.2.0 was released in late August and a minor version v2.2.1 followed suit to adress some bugs. This blog post will focus on comparing this new encoder version to the last, on the basis of benchmarks and visual comparisons. We will quantify the new trade-offs between compression efficiency and encoding speed, so you can choose the right balance for your projects. Our metrics of choice today will be SSIMULACRA2 and XPSNR, used in conjonction with a revised methodology. ## Feedback The biggest missed opportunity of the previous SVT-AV1 deep dives was the absence of visual comparisons. Indeed, metrics may be convenient for easily quantifying differences between encoder versions or encoding parameters, but they fail to give the information of how much these differences matter for your eyes. However, making properly useful visual comparisons isn't an easy task. Comparing two encodes of varying bitrates will bias the result against one or the other, which is not desirable. Figuring out the best way to present these comparisons and making the appropriate scripts took me weeks. Gathering all the necessary data, crafting the comparisons and proceeding to a double-checking pass took me another few weeks. These reasons explain why this blog post took so long to release, but I hope it will have been worth the wait! On that note though, I have been uploading thousands of png screenshots to *slow.pics* and I ended up rate-limited. This has prevented me from uploading the visual comparisons for preset 11, 12 and some of 10. I'm actively trying to fix this situation, so please understand. Another reason for this taking so long was my decision to increase the amount of video samples while increasing the amount of CRF values tested. A grand total of 3682 encodes were done for this blog post alone, in the span of around two weeks, where my PC would be exclusively encoding 24/7. Send help. Also, the graphs will now be using the harmonic mean instead of the arithmetic mean. Indeed, arithmetic mean scores fail to account for deviations and outliers. Using the harmonic mean implies low-scoring frames have more weight towards the final score, which adds a consistency component to the picture. That's not it though! Consistency is capital in an enjoyable watching experience. As such, each graph now possesses a SSIMULACRA2 (Harmonic) version, a XPSNR (Harmonic) version and a SSIMULACRA2 (Standard Deviation) version, in order to closely monitor variations in consistency between presets. Lastly, the biggest complaint I have received is me exclusively using anime clips. The reason for that is pretty simple: I mostly encode anime content on my free time and am myself very little interested in other types of media. However, these blog posts have grown in exposure and I understand a majority of people is more concerned about the performance of encoders on live action content or gaming clips. Thus, this new blog post is comprised of 3 live action clips, 2 gaming clips and 2 anime clips! See the sacrifices I'm making for y'all? ## Methodology The resources provided will include both graphs and image comparisons. Graphs offer a straightforward, objective look at efficiency across encoder parameters, using metrics as benchmarks for performance. In contrast, image comparisons display actual samples from encoded files, allowing you to assess quality firsthand. This adds a subjective dimension to the comparisons, giving you a more nuanced understanding of each preset's impact on visual quality. The testing methodology involves using relatively short video samples with a wide range of content types, uncompressed to the y4m file format for ease of use. These lossless files are *directly* fed to SvtAv1EncApp, implying the performance of a single encoder instance is what's being measured here. A more serious AV1 encoding pipeline should probably be leveraging a chunked encoding approach, especially on higher core count systems. Once an encode is done, SSIMULACRA2 scores are calculated using the [Zig implementation](https://github.com/dnjulek/vapoursynth-ssimulacra2), XPSNR scores on the other hand are calculated using a [ffmpeg filter](https://github.com/fraunhoferhhi/xpsnr), and the data is then aggregated into a final Harmonic or Standard Deviation score to create the graphs for this benchmark. The Constant Rate Factor (CRF) is plotted against encoding time, and the metrics scores against encode size (bitrate). The former may represent the efficiency as defined by the speed achieved at a certain quality target. For the latter, Bits Per Pixel scores (BPP) are calculated so that the `Metric / BPP` graphs may represent the compression efficiency, normalized by resolution. How to read the graphs? For the compression efficiency ones, the closer to the top left the better. For the encoding speed ones, the closer to the left the faster. For the standard deviation ones, the closer to the bottom left the better. The clips used in this test were acquired legally. The Codec Wiki and its contributors do not endorse media piracy. SvtAv1EncApp was compiled directly from the [v2.1.2 and v2.2.1 source code](https://gitlab.com/AOMediaCodec/SVT-AV1/-/releases/) using Clang 18.1.8 and the provided `Build/linux/build.sh` script with the following command: `build.sh cc=clang cxx=clang++ jobs=$(nproc) enable-lto static native release`. The testing machine is comprised of an i3 12100 in its stock configuration, with 2x8GB of 3200MHz CL14 DDR4 RAM, in Arch Linux with kernel 6.9.12 and the performance governor enabled. All encodes have been made in the same session without rebooting. I want to give a disclaimer concerning encoding speeds. Contrary to the efficiency results which should be reproducible independantly of the machine, measuring speed is a pretty difficult endeavor, with increased risks for errors. The performance numbers I mention may differ for you depending on the hardware configuration at hand. ## Samples & Encoding Settings The samples are as follows: - 17s `Avatar The Way Of Water (trailer 3)` clip sourced from [thedigitaltheater.com](https://thedigitaltheater.com/category/movie-trailers/lossless-movie-trailers/) (1916x804p, 23.976fps). - 6s `Ducks Take Off` clip sourced from [xiph.org](https://media.xiph.org/video/derf/) (1280x720p, 50fps). - 3s `Fallout 4` clip sourced from another encoder fellow (1920x1080p, 60fps). - 8s `Minecraft` clip sourced from [xiph.org](https://media.xiph.org/video/derf/) (1920x1080p, 60fps). - 8s `Sol Levante` HDR clip sourced from [opencontent.netflix.com](http://download.opencontent.netflix.com/?prefix=SolLevante/) (3840x2160p, 24fps). This one is pretty educative as SVT-AV1's behavior isn't influenced by the existence (or lack thereof) of HDR metadata in a source. - 21s `Suzume (trailer 2)` clip sourced from [thedigitaltheater.com](https://thedigitaltheater.com/category/movie-trailers/lossless-movie-trailers/) (1920x808p, 23.976fps). - 13s `The Mandalorian (trailer 2)` clip sourced from [thedigitaltheater.com](https://thedigitaltheater.com/category/movie-trailers/lossless-movie-trailers/) (1920x800p, 23.976fps). > **All clips have been encoded in a wide quality range, from `--crf 10` to `--crf 50`, by increments of 2, with the exception of preset -1 that uses increments of 4.** `--preset X --hierarchical-levels 4` are the only parameter used here, in conjunction with the CRF values. I have been asked to use `--hierarchical-levels 4` by fellow SVT-AV1-PSY developers to force smaller mini GOPs, more appropriate for testing. Else, the SVT-AV1 defaults were used. The ones worth mentioning are: - `--tune 1`: tune PSNR - `--aq-mode 2`: variance deltaq - `--enable-qm 0`: quantisation matrices disabled - `--irefresh-type 2`: closed GOP - `--enable-tf 1`: temporal filtering enabled And more, like CDEF and restoration enabled, overlays and film-grain disabled... ## Visual comparisons ![comp_showcase](/img/comp_showcase.webp) Throughout this blog post, you’ll find *slow.pics* links that provide various visual comparisons between presets. - The “full” links offer comparisons across the entire quality range for each source. - The HQ (High Quality), MQ (Medium Quality), and LQ (Low Quality) links showcase more targeted comparisons. These have been carefully handcrafted to be as size-normalized as possible, given the available encodes. We want to be focusing on encodes with minimal bitrate deviation for a fair comparison. Feel free to double-check the bitrate of each frame or scene to make a more informed observation, keeping the size difference in mind when comparing the encodes. Use the arrow keys and numpad to navigate between screenshots. Alternatively, you can click on "Slider comparison" and select two sources if you prefer comparing this way. Without further ado, let's start with the first comparisons! ## Presets comparisons (-1 -> 12) **In the following graphs, you may find comparisons between all SVT-AV1 presets, ranging from the slowest `--preset -1` to the fastest `--preset 12`.** > Just like in __v2.1.x__, preset 6 and 13 do not exist in __v2.2.x__ and are instead mapped to preset 7 and 12 respectively. ### Efficiency - First of all, the complete efficiency graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). You may notice something odd going on with the Avatar results using XPSNR. I have tried to understand the cause, without success. For the remainder of this blog post, the Avatar XPSNR results will be omitted. I will continue investigating and aim to have a workaround in place for next time. Anyway, this graph may be impressive, but difficult to read. So let's analyse different quality targets. - The same graphs but focusing on the "high quality" range (CRF10 -> 22): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Same, but now focusing on the "medium quality" range (CRF24 -> 36): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - And lastly, focusing on the "low quality" range (CRF38 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - If we now focus on __presets 4__ and below, where it's more difficult to discern the differences between presets, we get this at "high quality": > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - This at "medium quality": > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - And the following at "low quality": > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). ### Speed - **Let's now compare the speed of all presets:** > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Unusable, right? - Then, here is what it looks like with a logarithmic scale: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). ### Interpretation It appears as if once again **preset 2** through **preset 4** remain the most balanced presets all-around in an efficient encoding scenario, with **preset 3** not offering much improvements over **preset 4** in average scores but nicely improving on consistency instead, and **preset 2** offering a nice efficiency and consistency uplift on top. In this release again, the quality gap between **preset 2** and **preset 1** is pretty narrow, and the speed penalty from **preset 1** onward continuously increases, ending up close to ~2x. In comparison, the penalty of going from **preset 3** to **preset 2** is closer to ~1.5x. As such, using **preset 1** is entering placebo territory and it is usually not recommended to waste precious encoding resources on **preset 0** and **preset -1**. This especially applies at medium to high quality, though at extremely low quality targets, like the CRF40-50 range, we can still see appreciable gains from these placebo presets in some clips. As for the faster presets, **presets 5 to 10** are usually grouped on the graphs focusing on average scores and the ones focusing on consistency. They tend to stand apart from their slower counterparts by just a bit. Though **preset 10** can be worryingly close to **preset 11** on some occasions. They are all viable for your real-time needs. The rule is the same as usual: **go the slowest you can bear that still achieves your goal!** **Presets 11 and 12** are especially inefficient and inconsistent, and to be avoided at all costs. If possible, forget they even exist, as it's probably better to use a comparably fast (or faster) competing codec. They could still be of use in a convex-hull scenario, but in the case of realtime transcoding, you will be better off with some hardware solution like the ones found in RTX 4000 or Arc GPUs. ### TLDR The same conclusions as the previous blog posts can be made: **clear quality gains can be observed as we decrease presets, until __preset 2__, however the effectiveness of dropping presets is noticeably less and less important as quality is increased.** ## SVT-AV1 v2.1.x vs v2.2.x presets comparisons: In this section, we’ll examine the efficiency and speed differences across presets when upgrading from SVT-AV1 2.1.x to 2.2.x. This comparison should bring a new level of nuance to our results, highlighting both incremental improvements and any notable shifts in performance. SVT-AV1 __v2.1.x__ brought some nice improvements over __v2.0.0__, but does __v2.2.x__ bring appreciable improvements in the presets trade-offs this time around as well? Let's find out! ### `preset -1`: v2.1.x vs v2.2.x - **Let's start things off with the battle of the placebos, with the Compression efficiency & consistency at "high to medium-ish quality" (CRF10 -> 30):** > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Along with the Compression efficiency & consistency at "medium-ish to low quality" (CRF34 -> 50): > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Basically no changes at all, except a slight regression on Minecraft. - What about their speeds though?: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Well **preset -1** basically became 15 to 25% faster, not bad at all! - Preset -1 visual comparisons: [Avatar (full)](https://slow.pics/c/LdIsYSua), [Avatar (HQ)](https://slow.pics/c/2a4aILcQ), [Avatar (LQ)](https://slow.pics/c/zT82h9rX) [Ducks (full)](https://slow.pics/c/BpxD7oZd), [Ducks (HQ)](https://slow.pics/c/vQfwqoid), [Ducks (LQ)](https://slow.pics/c/1hSeoXlp) [Fallout (full)](https://slow.pics/c/EOIoOoAQ), [Fallout (HQ)](https://slow.pics/c/GAfLlWLl), [Fallout (LQ)](https://slow.pics/c/nXGUeEyI) [Minecraft (full)](https://slow.pics/c/XRHmsQOw), [Minecraft (HQ)](https://slow.pics/c/UpCRq43R), [Minecraft (LQ)](https://slow.pics/c/835PVO4r) [Sol Levante (full)](https://slow.pics/c/AQC1xzLS), [Sol Levante (HQ)](https://slow.pics/c/tLv4zf68), [Sol Levante (LQ)](https://slow.pics/c/x8r8YUvF) [Suzume (full)](https://slow.pics/c/Ab6bkcpS), [Suzume (HQ)](https://slow.pics/c/lsGgf2pL), [Suzume (LQ)](https://slow.pics/c/KPNiucZk) [The Mandalorian (full)](https://slow.pics/c/BAF89tjA), [The Mandalorian (HQ)](https://slow.pics/c/tI50zGYS), [The Mandalorian (LQ)](https://slow.pics/c/V2q6DHYT) ### `preset 0`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Efficiency wise, this new **preset 0** is close to unchanged from the old **preset 0**, but its consistency improved slightly in a few clips at high quality and decreased in one clip at low quality. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 0**'s speed sees an improvement of about 20% at best. Overall, **preset 0** got a proper upgrade! - Preset 0 visual comparisons: [Avatar (full)](https://slow.pics/c/7n5bAB2d), [Avatar (HQ)](https://slow.pics/c/AHmNT8Pw), [Avatar (MQ)](https://slow.pics/c/VC6lZr6D), [Avatar (LQ)](https://slow.pics/c/97SAuLaO) [Ducks (full)](https://slow.pics/c/m9iPnqqW), [Ducks (HQ)](https://slow.pics/c/gBXawmpH), [Ducks (MQ)](https://slow.pics/c/7dRVJzXD), [Ducks (LQ)](https://slow.pics/c/qshRSgnZ) [Fallout (full)](https://slow.pics/c/aq9eTZf5), [Fallout (HQ)](https://slow.pics/c/xu99HM0j), [Fallout (MQ)](https://slow.pics/c/eC0gzO7y), [Fallout (LQ)](https://slow.pics/c/pB9Ps3a6) [Minecraft (full)](https://slow.pics/c/aWbIxmgr), [Minecraft (HQ)](https://slow.pics/c/3jVGagWZ), [Minecraft (MQ)](https://slow.pics/c/MlYl7i4e), [Minecraft (LQ)](https://slow.pics/c/7MImyYc0) [Sol Levante (full)](https://slow.pics/c/SmDGEAPx), [Sol Levante (HQ)](https://slow.pics/c/ZiCgjlxE), [Sol Levante (MQ)](https://slow.pics/c/4GbaSzv6), [Sol Levante (LQ)](https://slow.pics/c/EPTFWHQI) [Suzume (full)](https://slow.pics/c/oVZqCWsl), [Suzume (HQ)](https://slow.pics/c/LqadqVGG), [Suzume (MQ)](https://slow.pics/c/TnD6CAyF), [Suzume (LQ)](https://slow.pics/c/CDuu3SOK) [The Mandalorian (full)](https://slow.pics/c/x8Sot0wg), [The Mandalorian (HQ)](https://slow.pics/c/idbPz3uh), [The Mandalorian (MQ)](https://slow.pics/c/OpFzXtjr), [The Mandalorian (LQ)](https://slow.pics/c/3d8gpuoo) ### `preset 1`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 1** is mostly unchanged but sees another slight regression in Minecraft at high quality. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Depending on the clip, speed is mostly unchanged or ever so slighty improved. **Preset 1** is a bit stagnant this release. - Preset 1 visual comparisons: [Avatar (full)](https://slow.pics/c/BnOX37oJ), [Avatar (HQ)](https://slow.pics/c/bWjBtTyR), [Avatar (MQ)](https://slow.pics/c/6UQKyNac), [Avatar (LQ)](https://slow.pics/c/NpyxU3uF) [Ducks (full)](https://slow.pics/c/KVX4piAj), [Ducks (HQ)](https://slow.pics/c/EsEpbKVY), [Ducks (MQ)](https://slow.pics/c/8AxTTvKw), [Ducks (LQ)](https://slow.pics/c/ZgW8715d) [Fallout (full)](https://slow.pics/c/xQprIMbB), [Fallout (HQ)](https://slow.pics/c/qRvBez7k), [Fallout (MQ)](https://slow.pics/c/taXnNGUx), [Fallout (LQ)](https://slow.pics/c/s6fbIqvr) [Minecraft (full)](https://slow.pics/c/cE86n2Gb), [Minecraft (HQ)](https://slow.pics/c/eOyUxHhq), [Minecraft (MQ)](https://slow.pics/c/tKPYsd9K), [Minecraft (LQ)](https://slow.pics/c/evfzJDjc) [Sol Levante (full)](https://slow.pics/c/BSzecR9N), [Sol Levante (HQ)](https://slow.pics/c/mWIoU45b), [Sol Levante (MQ)](https://slow.pics/c/geYXLoIY), [Sol Levante (LQ)](https://slow.pics/c/QzSjaYXF) [Suzume (full)](https://slow.pics/c/pBVidgiB), [Suzume (HQ)](https://slow.pics/c/5KtTXd3q), [Suzume (MQ)](https://slow.pics/c/lb0yMDhZ), [Suzume (LQ)](https://slow.pics/c/TipcRRJr) [The Mandalorian (full)](https://slow.pics/c/4ouGFV9s), [The Mandalorian (HQ)](https://slow.pics/c/oxPNsHdU), [The Mandalorian (MQ)](https://slow.pics/c/EPju5ULC), [The Mandalorian (LQ)](https://slow.pics/c/e58LmgFm) ### `preset 2`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 2**'s efficiency has regressed at high quality on some clips, improved in some and stayed the same in others. Except in one clip, consistency seems to have improved all around. At low to medium quality targets, efficiency is mostly unchanged, same for consistency. Not exactly noteworthy. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Speed was improved by about 10-20%. Not a bad showcase, for sure. - Preset 2 visual comparisons: [Avatar (full)](https://slow.pics/c/CJg3dsFg), [Avatar (HQ)](https://slow.pics/c/71wpH5eB), [Avatar (MQ)](https://slow.pics/c/AN0syhNv), [Avatar (LQ)](https://slow.pics/c/NRocyEay) [Ducks (full)](https://slow.pics/c/NyGfMRpQ), [Ducks (HQ)](https://slow.pics/c/OAqs5b8k), [Ducks (MQ)](https://slow.pics/c/tQSF50Ro), [Ducks (LQ)](https://slow.pics/c/gJjm3BGl) [Fallout (full)](https://slow.pics/c/snQuTi1f), [Fallout (HQ)](https://slow.pics/c/0rLG3TUT), [Fallout (MQ)](https://slow.pics/c/MMX6DgNz), [Fallout (LQ)](https://slow.pics/c/lwoIkLfm) [Minecraft (full)](https://slow.pics/c/5OYsrhGY), [Minecraft (HQ)](https://slow.pics/c/kCwbA7Ir), [Minecraft (MQ)](https://slow.pics/c/1nnyrvjD), [Minecraft (LQ)](https://slow.pics/c/F1RTfQf7) [Sol Levante (full)](https://slow.pics/c/xduJGoTU), [Sol Levante (HQ)](https://slow.pics/c/8SdKLegQ), [Sol Levante (MQ)](https://slow.pics/c/uHNR7DNz), [Sol Levante (LQ)](https://slow.pics/c/wgZgH7w6) [Suzume (full)](https://slow.pics/c/zC4AEMLt), [Suzume (HQ)](https://slow.pics/c/kGip9wuY), [Suzume (MQ)](https://slow.pics/c/aoQ0m9ms), [Suzume (LQ)](https://slow.pics/c/7ZMT3inc) [The Mandalorian (full)](https://slow.pics/c/8BKkJ4du), [The Mandalorian (HQ)](https://slow.pics/c/XQeD7vfY), [The Mandalorian (MQ)](https://slow.pics/c/6rQMASQR), [The Mandalorian (LQ)](https://slow.pics/c/Jl2Fh5wE) ### `preset 3`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Practically, it's a wash. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Still, **preset 3** got slightly faster, I'm happy to report this is a speedup! - Preset 3 visual comparisons: [Avatar (full)](https://slow.pics/c/Vql4MJWQ), [Avatar (HQ)](https://slow.pics/c/Iuqp0Zbz), [Avatar (MQ)](https://slow.pics/c/MuTOEHE9), [Avatar (LQ)](https://slow.pics/c/poiDDJRQ) [Ducks (full)](https://slow.pics/c/mqWOgsYF), [Ducks (HQ)](https://slow.pics/c/16L6HyaH), [Ducks (MQ)](https://slow.pics/c/KaoPy7ae), [Ducks (LQ)](https://slow.pics/c/0L84OWnI) [Fallout (full)](https://slow.pics/c/JuXAjSY6), [Fallout (HQ)](https://slow.pics/c/mR5rJfsU), [Fallout (MQ)](https://slow.pics/c/Zpy7Hzds), [Fallout (LQ)](https://slow.pics/c/GeQrpfhp) [Minecraft (full)](https://slow.pics/c/rBCbaJkx), Minecraft (HQ comparison unavailable), [Minecraft (MQ)](https://slow.pics/c/MYLA8KPH), [Minecraft (LQ)](https://slow.pics/c/aEMnRk7D) [Sol Levante (full)](https://slow.pics/c/F7rmPiaW), [Sol Levante (HQ)](https://slow.pics/c/PYO4K4sy), [Sol Levante (MQ)](https://slow.pics/c/viWYsgX7), [Sol Levante (LQ)](https://slow.pics/c/4NBaUoEX) [Suzume (full)](https://slow.pics/c/d15XbowQ), [Suzume (HQ)](https://slow.pics/c/eS6EQKng), [Suzume (MQ)](https://slow.pics/c/IJtLW35Z), [Suzume (LQ)](https://slow.pics/c/zf352iox) [The Mandalorian (full)](https://slow.pics/c/UrAekqg6), [The Mandalorian (HQ)](https://slow.pics/c/WWgygRS4), [The Mandalorian (MQ)](https://slow.pics/c/ry5sPqmB), [The Mandalorian (LQ)](https://slow.pics/c/6wI4wexZ) ### `preset 4`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 4** sees a consistent though small improvement in average scores and standard deviation across the entire quality range on basically all clips. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Unfortunately, it got slower as a result. If you remember, **v2.1.0** did the exact contrary over **v2.0.0**, I wonder if **preset 4** simply took back the place it previously had... - Preset 4 visual comparisons: [Avatar (full)](https://slow.pics/c/ArdO4K7R), [Avatar (HQ)](https://slow.pics/c/88ikqaZn), [Avatar (MQ)](https://slow.pics/c/rM6LRBg0), [Avatar (LQ)](https://slow.pics/c/ofmHWCkv) [Ducks (full)](https://slow.pics/c/HVkqo5G3), [Ducks (HQ)](https://slow.pics/c/w44DNiqe), [Ducks (MQ)](https://slow.pics/c/GlRLCaBt), [Ducks (LQ)](https://slow.pics/c/Lkzi1VBU) [Fallout (full)](https://slow.pics/c/44jtBEqw), [Fallout (HQ)](https://slow.pics/c/G9YPifBb), [Fallout (MQ)](https://slow.pics/c/86m4exaw), [Fallout (LQ)](https://slow.pics/c/m4KHBihZ) [Minecraft (full)](https://slow.pics/c/n9OP2sht), [Minecraft (HQ)](https://slow.pics/c/JohJ3MzJ), [Minecraft (MQ)](https://slow.pics/c/e6b8R24h), [Minecraft (LQ)](https://slow.pics/c/xNwNlmAG) [Sol Levante (full)](https://slow.pics/c/vB7ndGZI), [Sol Levante (HQ)](https://slow.pics/c/R5uENbD5), [Sol Levante (MQ)](https://slow.pics/c/i7eV72sl), [Sol Levante (LQ)](https://slow.pics/c/zUMYarNf) [Suzume (full)](https://slow.pics/c/rgmuB5xB), [Suzume (HQ)](https://slow.pics/c/y9rxWVre), [Suzume (MQ)](https://slow.pics/c/4GhRL8gZ), [Suzume (LQ)](https://slow.pics/c/F3Hq8V9z) [The Mandalorian (full)](https://slow.pics/c/3BAR5P80), [The Mandalorian (HQ)](https://slow.pics/c/s2xzApJd), [The Mandalorian (MQ)](https://slow.pics/c/Ihjks1JO), [The Mandalorian (LQ)](https://slow.pics/c/JjGKUO1r) ### `preset 5`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Both metrics say **preset 5** regressed slightly to moderately, though surprisingly its consistency is basically unchanged. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). The result of this regression is an impressive speedup of up to 25%. - Preset 5 visual comparisons: [Avatar (full)](https://slow.pics/c/Lc8HjJBK), [Avatar (HQ)](https://slow.pics/c/r3gJkCv3), [Avatar (MQ)](https://slow.pics/c/k2TwNLHN), [Avatar (LQ)](https://slow.pics/c/cMlNo17a) [Ducks (full)](https://slow.pics/c/X4EZFJcM), [Ducks (HQ)](https://slow.pics/c/zKLphp19), [Ducks (MQ)](https://slow.pics/c/dYs1snV3), [Ducks (LQ)](https://slow.pics/c/JFtne5WV) [Fallout (full)](https://slow.pics/c/9ye1q9Rm), [Fallout (HQ)](https://slow.pics/c/zhEuOjRc), [Fallout (MQ)](https://slow.pics/c/YmZ3aRLO), [Fallout (LQ)](https://slow.pics/c/9ye1q9Rm) [Minecraft (full)](https://slow.pics/c/kyjcJWxt), [Minecraft (HQ)](https://slow.pics/c/UTWgYc3P), [Minecraft (MQ)](https://slow.pics/c/FtOxX3RH), [Minecraft (LQ)](https://slow.pics/c/jNyqLPgP) [Sol Levante (full)](https://slow.pics/c/ipyfREjA), [Sol Levante (HQ)](https://slow.pics/c/73V8CGKk), [Sol Levante (MQ)](https://slow.pics/c/z8P9UokS), [Sol Levante (LQ)](https://slow.pics/c/OPwerF86) [Suzume (full)](https://slow.pics/c/xfsYGM5C), [Suzume (HQ)](https://slow.pics/c/dL2J923D), [Suzume (MQ)](https://slow.pics/c/OhZNMzjx), [Suzume (LQ)](https://slow.pics/c/XFCnEebJ) [The Mandalorian (full)](https://slow.pics/c/xrO227rQ), [The Mandalorian (HQ)](https://slow.pics/c/m7Yky6EK), [The Mandalorian (MQ)](https://slow.pics/c/pN0cH6Yn), [The Mandalorian (LQ)](https://slow.pics/c/vB9zK9iz) ### ~~`preset 6`: v2.1.x vs v2.2.x~~ **Preset 6** is mapped to **preset 7** in **v2.2.x**. ### `preset 7`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 7** is close to unchanged in **v2.2.x**. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). It still got some slight to moderate speedups though, which can be appreciated. - Preset 7 visual comparisons: [Avatar (full)](https://slow.pics/c/YtvMG9X6), [Avatar (HQ)](https://slow.pics/c/P0gF7nbQ), [Avatar (MQ)](https://slow.pics/c/syT6uiVt), [Avatar (LQ)](https://slow.pics/c/zkkv1ePp) [Ducks (full)](https://slow.pics/c/pyVQmAWj), [Ducks (HQ)](https://slow.pics/c/eXSNnqV7), [Ducks (MQ)](https://slow.pics/c/ja7d0sFU), [Ducks (LQ)](https://slow.pics/c/JCTFfcMu) [Fallout (full)](https://slow.pics/c/S6SbCW8x), [Fallout (HQ)](https://slow.pics/c/bWCLQWwV), [Fallout (MQ)](https://slow.pics/c/hvauusst), [Fallout (LQ)](https://slow.pics/c/04LVSbGD) [Minecraft (full)](https://slow.pics/c/mh8VUfrO), [Minecraft (HQ)](https://slow.pics/c/P0pIx6XP), [Minecraft (MQ)](https://slow.pics/c/vnrXVajs), [Minecraft (LQ)](https://slow.pics/c/8WKQdw2N) [Sol Levante (full)](https://slow.pics/c/kH9LcEzG), [Sol Levante (HQ)](https://slow.pics/c/1DWCNq2B), [Sol Levante (MQ)](https://slow.pics/c/clxjB7Uw), [Sol Levante (LQ)](https://slow.pics/c/52wLF7NR) [Suzume (full)](https://slow.pics/c/3VlVk3O9), [Suzume (HQ)](https://slow.pics/c/FKel5bhI), [Suzume (MQ)](https://slow.pics/c/XXiyyCNN), [Suzume (LQ)](https://slow.pics/c/rrBoua68) [The Mandalorian (full)](https://slow.pics/c/unxuKHEK), [The Mandalorian (HQ)](https://slow.pics/c/GMXnxSdA), [The Mandalorian (MQ)](https://slow.pics/c/v8wSN90W), [The Mandalorian (LQ)](https://slow.pics/c/fswh5kCT) ### `preset 8`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). The efficiency and consistency of **preset 8** has improved at high quality. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). And we can observe a speed increase of around 10%. Some crazy speed deviations can be noticed in Sol Levante. - Preset 8 visual comparisons: [Avatar (full)](https://slow.pics/c/8tKOAvGN), [Avatar (HQ)](https://slow.pics/c/YLWgqmxU), [Avatar (MQ)](https://slow.pics/c/pDg4gLry), [Avatar (LQ)](https://slow.pics/c/ADFSsXmP) [Ducks (full)](https://slow.pics/c/ug0TVcoF), [Ducks (HQ)](https://slow.pics/c/WiDmtcVZ), [Ducks (MQ)](https://slow.pics/c/ERVfuYHH), [Ducks (LQ)](https://slow.pics/c/I0N97rQw) [Fallout (full)](https://slow.pics/c/MqVzmFIi), [Fallout (HQ)](https://slow.pics/c/jUoB0PGo), [Fallout (MQ)](https://slow.pics/c/hXGIfwVj), [Fallout (LQ)](https://slow.pics/c/QUy3iX2H) [Minecraft (full)](https://slow.pics/c/frkuJR97), [Minecraft (HQ)](https://slow.pics/c/GayEkyrn), [Minecraft (MQ)](https://slow.pics/c/S36Ha8LP), [Minecraft (LQ)](https://slow.pics/c/lxoYsv8k) [Sol Levante (full)](https://slow.pics/c/YupMFGeu), [Sol Levante (HQ)](https://slow.pics/c/evfWqxZ9), [Sol Levante (MQ)](https://slow.pics/c/Juh027Eq), [Sol Levante (LQ)](https://slow.pics/c/ETPf9ppU) [Suzume (full)](https://slow.pics/c/oeFUWMHd), [Suzume (HQ)](https://slow.pics/c/01vzYfnq), [Suzume (MQ)](https://slow.pics/c/BIL4g7zk), [Suzume (LQ)](https://slow.pics/c/boxyyeZN) [The Mandalorian (full)](https://slow.pics/c/57M6RHM0), [The Mandalorian (HQ)](https://slow.pics/c/8WJcnZfU), [The Mandalorian (MQ)](https://slow.pics/c/M7q5PYz8), [The Mandalorian (LQ)](https://slow.pics/c/ctoke6p1) ### `preset 9`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Efficiency and consistency stayed mostly the same. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Speed improved by a few percents at most. **Preset 9** has stagnated over **v2.1.x**. - Preset 9 visual comparisons: [Avatar (full)](https://slow.pics/c/P3zZKVF0), [Avatar (HQ)](https://slow.pics/c/sL4HPZLa), [Avatar (MQ)](https://slow.pics/c/QOxyI9h0), [Avatar (LQ)](https://slow.pics/c/RfLKIRjj) [Ducks (full)](https://slow.pics/c/aj6SVcIZ), [Ducks (HQ)](https://slow.pics/c/UpnejngN), [Ducks (MQ)](https://slow.pics/c/KJRzl3FY), [Ducks (LQ)](https://slow.pics/c/4mMq9sz3) [Fallout (full)](https://slow.pics/c/vHqlIJBK), [Fallout (HQ)](https://slow.pics/c/XHoe9Udp), [Fallout (MQ)](https://slow.pics/c/tukibDud), [Fallout (LQ)](https://slow.pics/c/cV57jRF9) [Minecraft (full)](https://slow.pics/c/XRT1OYnX), [Minecraft (HQ)](https://slow.pics/c/HqtoRlvV), [Minecraft (MQ)](https://slow.pics/c/Ml2rLjvO), [Minecraft (LQ)](https://slow.pics/c/EGCsqcSl) [Sol Levante (full)](https://slow.pics/c/8URvlnba), [Sol Levante (HQ)](https://slow.pics/c/jPFT5hMG), [Sol Levante (MQ)](https://slow.pics/c/92EZO3U3), [Sol Levante (LQ)](https://slow.pics/c/bWgcvUtj) [Suzume (full)](https://slow.pics/c/yA2y0v92), [Suzume (HQ)](https://slow.pics/c/pAPBbl7r), [Suzume (MQ)](https://slow.pics/c/7hTvbMy1), [Suzume (LQ)](https://slow.pics/c/arrDGAd9) [The Mandalorian (full)](https://slow.pics/c/KQHqkAVZ), [The Mandalorian (HQ)](https://slow.pics/c/UzdN0rSC), [The Mandalorian (MQ)](https://slow.pics/c/dxKpfcvs), [The Mandalorian (LQ)](https://slow.pics/c/FPJAf74J) ### `preset 10`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 10** received the most efficiency and consistency improvements out of all presets in this release. What will be the cost of such drastic change though? - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Well, not that much all things considered! **Preset 10**'s speed did decrease from barely anything to 20% in the most extreme situation, but its improvements well outshine its speed regression. Overall, **preset 11** went from borderline unusable to becoming an interesting new fast real-time preset. This is pretty huge in my opinion, as it offers a new kind of trade-off no other AV1 encoder or prior SVT-AV1 versions did. - Preset 10 visual comparisons: [Avatar (full)](https://slow.pics/c/8GjtQQAm), [Avatar (HQ)](https://slow.pics/c/FriwG6rw), [Avatar (MQ)](https://slow.pics/c/vqau9jk1), [Avatar (LQ)](https://slow.pics/c/WQTbcWvH) [Ducks (full)](https://slow.pics/c/xYotraaP), [Ducks (HQ)](https://slow.pics/c/HHlwCvwc), [Ducks (MQ)](https://slow.pics/c/bgV6MoQz), [Ducks (LQ)](https://slow.pics/c/bOAyMVUk) [Fallout (full)](https://slow.pics/c/MrYTMHoQ), [Fallout (HQ)](https://slow.pics/c/qL9xmCfI), [Fallout (MQ)](https://slow.pics/c/dwdE0pWt), [Fallout (LQ)](https://slow.pics/c/uo7Ztb5f) [Minecraft (full)](https://slow.pics/c/fICToF3E), [Minecraft (HQ)](https://slow.pics/c/cGBhBsaS), [Minecraft (MQ)](https://slow.pics/c/w1P4LAu5), [Minecraft (LQ)](https://slow.pics/c/rNqRWd61) [Sol Levante (full)](https://slow.pics/c/UyMpDhx0), [Sol Levante (HQ)](https://slow.pics/c/b0B7ndsZ), [Sol Levante (MQ)](https://slow.pics/c/3H4ewpMI), [Sol Levante (LQ)](https://slow.pics/c/wpCnz72u) [Suzume (full)](https://slow.pics/c/ZYoFdtdS), [Suzume (HQ)](https://slow.pics/c/L2z8yEd5), [Suzume (MQ)](https://slow.pics/c/DEphKnbY), [Suzume (LQ)](https://slow.pics/c/Dm3OP4xO) [The Mandalorian (full)](https://slow.pics/c/NhYFWBzc), [The Mandalorian (HQ)](https://slow.pics/c/MFMQjVRD), [The Mandalorian (MQ)](https://slow.pics/c/pTt74iCi), [The Mandalorian (LQ)](https://slow.pics/c/e6VwtFfU) ### `preset 11`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). The new **preset 11** places itself between the old **preset 11** and the old **preset 12** efficiency and consistency wise. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Unsurprisingly, its speed is also in-between the old **preset 11** and the old **preset 12**. I'm unsure this new trade-off helps in anything. ### `preset 12`: v2.1.x vs v2.2.x - Compression efficiency & consistency graphs, high quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, medium quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). - Compression efficiency & consistency graphs, low quality range: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). **Preset 12** is almost unchanged from the previous release. - Speed graphs: > **Interactive comparison:** [Open the complete media manifest](/media/articles/svt-av1-third-deep-dive.json). Same speed wise. So, no improvements at all for the two fastest presets. They remain all around very bad performers. In hindsight, it doesn't matter that I couldn't upload the **preset 11 & 12** visual comparisons, because there is literally nothing to see. ### ~~`preset 13`: v2.1.x vs v2.2.x~~ **Preset 13** is mapped to **preset 12** in **v2.2.x**. ### TLDR With **v2.2.x**, we observed new efficiency/speed trade-offs for a good amount of presets. Some presets, like **-1** and **0**, received significant speed improvements at no efficiency cost. **Presets 3 and 7** received more reasonable speedups. **Presets 2, 8 and 11** have seen new trade-offs that are mostly beneficial. **Preset 10** was deeply revamped and replaces **preset 9** in my book as the fastest, still viable, real-time preset. **Preset 4** seems to have returned to the state it was in **v2.0.0**. On the other hand, **preset 5** seemed to have regressed slightly, and **presets 1, 9 and 12** are basically unchanged from **v2.1.x**. ## Conclusion The release of SVT-AV1 **v2.2.x** brings some welcome speed improvements. **Presets 2 through 4** continue to lead in efficiency for AV1 encoding, delivering top-tier quality and compression. Meanwhile, **presets 5 through 10** offer solid alternatives for those who find **presets 2 through 4** too slow, balancing quality with noticeably faster encoding times. Hopefully, this comprehensive third deep dive has given you a helpful starting point for choosing settings when encoding with ~~the latest~~ SVT-AV1(-PSY) v2.2.x. ## Future Once more, this testing focused on establishing the new presets dynamics, however I haven't revisited the different SVT-AV1 parameters since **v1.8.0**. A few meaningful features have been added since, like variance boost, and with this overhauled methodology, the conclusion made in that blog post may be different now. I think it will be worth to revisit this in the future, maybe in the next blog post for **v2.3.0**? Yes, I'm fully aware I'm late because **v2.3.0** has already been out for two whole weeks. Even if the frontend of this blog post doesn't seem to have radically changed, my entire workflow has tremendously evolved since last time. It may have taken me since mid-August to manage to complete this blog post, but my efforts should allow me to produce a follow-up faster. I am conscious of this blog post's limitations. First of all, I observed some odd behaviors from XPSNR on certain clips which I haven't been able to pinpoint yet. It would also give me nonsensical standard deviation results, which is the reason why only the standard deviation in SSIMU2 scores was given. Second of all, SVT-AV1's own behavior starts to get messy when you approach the SSIMULACRA2 0 score, rendering all the data in that region pretty much useless. Plus, aggregating these metrics scores take forever, so I'm looking into ways to accelerate the process, for instance by offloading the work to my GPU using [turbo-metrics](https://github.com/Gui-Yom/turbo-metrics). Also, I'm stuck between wanting to increase the amount of data points in each graphs to get more detailed results and having to keep everything readable. This time around I had to seperate each graph into three quality levels, but that's already too much for my liking. I will look into improving myself on the data presentation front. I feel like I have to streamline this formula to make it more digestible for everyone. Please, I'm open to **your** remarks and suggestions to improve on this blog post formula. That said, here are my plans for future blog posts: - a follow-up v2.3.0 article that also revisits the useful SVT-AV1 parameters. *__I'm planning for this to release before 2025.__* - an article focused on giving you encoding tips and explaining common AV1 encoding knowlegde is still planned for *someday*. - a *future* article focused on observing the evolution of all software AV1 encoders since the beginning of their development, as well as comparisons with vpxenc, AVM (development ground for AV2) and VVenC. - and many more... Thanks for reading! *Support me by making a donation on [my Ko-Fi page](https://ko-fi.com/nekotrix), as a reward for my efforts and to compensate for the electricity bills of two whole weeks of non-stop encoding.* {} --- # Fast Computation of SSIMULACRA2 on GPUs: A Performance Evaluation Canonical URL: https://encode.wiki/articles/turbo-metrics-performance/ Collection: articles Status: needs-review License: CC BY-SA 4.0 Checksum: b7eda39e9d93ca0b3582827329747457f95528ca069cfc4cf6b3d390c6d0b351 # Introduction In this blog entry, we will evaluate the performance of [TurboMetrics](https://github.com/Gui-Yom/turbo-metrics), a software which provides a GPU-accelerated implementation of SSIMULACRA2, and promises to save a lot of time in its computation. Video encoding is a time-consuming task by itself. When it comes to assessing the resulting encodes, objective quality metrics are often used, providing advantages over subjective measurements, e.g., faster evaluation processes. Nevertheless, the computation of some of these metrics is still somewhat slow. This is the case for [SSIMULACRA2](/reference/metrics/ssimulacra2/), one of the most popular objective metrics for image and video assessment. ## Basics Before we start with the evaluation, we will first cover some fundamental (and not-so-fundamental) concepts. ### SSIMULACRA2 SSIMULACRA2, which is sometimes abbreviated as "ssimu2", is a perceptual metric for images, based on the concept of the multi-scale structural similarity index measure ([MS-SSIM](https://en.wikipedia.org/wiki/Structural_similarity_index_measure)). In the words of its developer, it is "[based on MS-SSIM], computed in a perceptually relevant color space, adding two other (asymmetric) error maps, and aggregating using two different norms". A total of 54 error maps are computed over multiple downscalings (from 1:1 to 1:32) of the image to assess (_distorted_) and its source image (_reference_). These error maps are then added using a weighted sum to produce the final SSIMULACRA2 score. The weight of each error map is tuned based on a large set of subjective scores from different image benchmarks. SSIMULACRA2 scores are in the range [-inf, 100]. They are reported to correlate to subjective visual quality scores as follows: * 30 = low quality * 50 = medium quality * 70 = high quality * 90 = very high quality; likely impossible to distinguish from the original when viewed at 1:1 from a normal viewing distance One special consideration when using SSIMULACRA2 to assess video quality is that, even though the developers state that a score of 90 corresponds to a "visually lossless" image, for videos, this threshold is considered to be lower, around 80. This is because it is harder to notice artifacts on individual frames, given the short period of time the viewer perceives them (usually, between 1/24 and 1/60 of a second, 0.04167 and 0.0167 seconds, respectively). The reference implementation for SSIMULACRA2 can be found [here](https://github.com/cloudinary/ssimulacra2). Other popular implementations include [the Rust implementation](https://docs.rs/crate/ssimulacra2_rs/latest), and [the Zig implementation](https://github.com/dnjulek/vapoursynth-zip). Although technically speaking SSIMULACRA2 is an image-focused metric, it has gained popularity as a video assessment metric due to its reliability in correlating to subjective measurements. It is considered to provide better results than other historically more popular metrics, such as [PSNR](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio), [(MS-)SSIM](https://en.wikipedia.org/wiki/Structural_similarity_index_measure), and even [VMAF](https://github.com/Netflix/vmaf). In that context, each video frame is treated as a separate image. SSIMULACRA2 scores are computed for each of the video's frames independently, and the average is taken as the video's score. Other useful statistics include the standard deviation (which correlates to the consistency of the quality of the video), the median, the 5th percentile, and the 95th percentile. Although SSIMULACRA2 is considered to be quite reliable for video assessment, it is worth noting that, being a purely image-based metric, it disregards any temporal information present in videos, which might be seen as a disadvantage. Other metrics, such as [XPSNR](https://github.com/fraunhoferhhi/xpsnr), were developed with video assessment as their main objective, and do consider temporal information. As stated previously, SSIMULACRA2 computes 54 error maps per image assessed. This entails a high amount of computation, especially for videos, which usually contain tens of thousands of frames (images). That explains its rather slow execution time. Speeding up these computations would be very much desired, if possible. --- #### In-depth: SSIMULACRA2 computation The summary of the algorithm for SSIMULACRA2 is the following, as [reported here](https://github.com/Gui-Yom/turbo-metrics/blob/master/crates/ssimulacra2-cuda/README.md#computing-ssimulacra2): 1. Get the frame pair (reference and distorted). 2. Convert the frames to linear RGB (using 32-bit floating-point). 3. For each scale (1:1 to 1:32, downsampling by 2 every step: 6 different scales): 1. Downscale by the scale number (if needed). 2. Convert the frames to XYB color space. 3. Blur the following pictures, using a recursive Gaussian blur: (reference * reference), (distorted * distorted), (reference * distorted), (reference), (distorted). 4. Compute 1-SSIM, artifact and detail\_loss error scores from the 5 blurred images and the original frame pair; this yields 3 error maps. 5. The error maps are reduced to a single number using the 1-norm and 4-norm. 4. From this, we get 6 scores for each scale (6) and color component (3), totaling 108 scores. 5. The scores are added using a weighted sum. 6. The final value is processed through a non-linear function and clipped to render a score between 100 and -infinite. #### SSIMULACRA2 subsampling Given that the metric is quite slow to compute, it is often computed over a subset (subsample) of the video frames, instead of over all of them. For example, a _stride_ of 3 could be used, to compute SSIMULACRA2 scores for 1 frame out of every 3 in the video, skipping the computation for the other 2 frames. This is expected to speedup the computation by a factor of ×3. However, this evidently induces an _error_ in the score, as the average of the scores of a subsample of the video frames is not guaranteed to be equal to the average of the scores of all the frames. The smaller the subsample (i.e., the higher the computation stride), the higher this error will be. Nevertheless, it has been reported that small strides usually produce averages significantly close to the real average of all the frames. Thus, it is pretty safe to do this. To minimize the error of this sampling method, it has been stated that "as long as you don't sample in a power of two ([i.e., every] 2, 4, 8... [frames]), you'll be fine, as you want to pick frames from every temporal layer [of the video]". Other sources further recommend picking strides that are prime numbers (e.g., 1 every 3, 5, 7, 11... frames), since "prime numbers reduce the likelihood of sampling bias due to constant frame rate". ### TurboMetrics [TurboMetrics](https://github.com/Gui-Yom/turbo-metrics) is a WIP software suite developed by the very talented [Gui-Yom/LimelioN](https://github.com/Gui-Yom). It is "a collection of video related libraries and tools oriented at performance and hardware acceleration". It comes as a collection of Rust libraries/crates, and some include command-line applications to interact with them. There are currently no public releases of the suite in the dedicated "Releases" page of its GitHub repository, but you can download the code and compile it yourself. The latest tagged version of the software, at the moment of writing, is v0.2.2. TurboMetrics focuses mainly on **NVIDIA GPUs** using CUDA. However, the developer is actively working on adding support for other hardware (e.g., AMD and Intel GPUs). Nevertheless, I only have experience with NVIDIA GPUs, so I cannot tell how the progress is going on that front. Among the tools, there is a working SSIMULACRA2 implementation that uses CUDA to leverage GPU acceleration, ssimulacra2-cuda. We will focus exclusively on it from now on. We will use this tool to compute SSIMULACRA2 scores for certain video samples. However, it is worth noting that the tool is also able to compute other metrics, such as SSIM and PSNR. It can even compute multiple metrics on the same rum. ssimulacra2-cuda is [stated to be](https://github.com/Gui-Yom/turbo-metrics/tree/master/crates/ssimulacra2-cuda) "close to the original implementation, and with close results". Also, the developer has said that "[r]ight now, [it] can compute SSIMULACRA2 orders of magnitude faster than the ssimulacra2\_rs implementation". Therefore, we can expect the tool to compute SSIMULACRA2 scores for videos much faster than any CPU implementation out there, but provide scores that are not quite the same as the ones those CPU implementations would give, including the reference one. In this blog entry, we will test that second claim, concerning the speed of TurboMetrics in computing SSIMULACRA2. --- #### In-depth: Implementation differences from the reference As [reported here](https://github.com/Gui-Yom/turbo-metrics/blob/master/crates/ssimulacra2-cuda/README.md#differences-from-the-reference-implementation), the GPU implementation of SSIMULACRA2 provided in TurboMetrics differs from the reference implementation in the following ways that can affect the accuracy of the reported scores: * The conversion from YUV to linear RGB might not be yielding the same results as other tools. * Explicit Fused Multiply-Add (FMA) operations are used when possible, as the GPU can leverage them for higher performance. * The order of operations might not be the same, as some calculations have been rearranged. * Floating-point operations might not yield the same results, especially with the approximated math functions used in the GPU (like `powf`). Additionally, the differences in scores are amplified by the following features of the SSIMULACRA2 computation: * The original weights were computed by fitting the error scores to Mean Opinion Score (MOS), and deviation in the error scores is amplified by them. * The final non-linear function is rather steep (it is a cubic function, x³), and, thus, score variations are amplified again. ## Performance evaluation ### Setup In this experiment, we will evaluate TurboMetrics in terms of performance; that is, the amount of ssimu2 scores computed per unit of time. For that, an episode of an animated series (~25 minutes), encoded with H.264, has been chosen as sample video. However, the kind of video content used, and most of its technical characteristics (e.g., codec, encoding parameters, etc.) except for resolution, should be irrelevant, as the same computations are performed for every video of specific frame dimensions. In this case, the video has a resolution of 1080p (1920×1080 pixels per frame), with 8 bits per pixel, and YUV color space with 4:2:0 chroma subsampling. The video source and example distorted encode have been compared using **ssimulacra2\_rs**, **vszip**, and **TurboMetrics**. ssimulacra2\_rs and vszip are both CPU implementations of SSIMULACRA2, the former written in Rust and the latter in Zig (but invoked from VapourSynth/Python scripts). It has been reported that the Zig implementation is considerably faster than the Rust one, and we can check that out as well! The CPU implementations have been executed on an Intel i5-11400 CPU (base frequency of 2.60 GHz, turbo frequency of 4.40 GHz), using 6 CPU cores (`-f 6` parameter in ssimulacra2\_rs, `-t 6` parameter in ssimulacravszip.py). TurboMetrics has been executed both on an NVIDIA GTX 1060 GPU and an NVIDIA RTX 4060 GPU. The three programs have been executed for the same video pair with **varying frame strides: 1, 3, 5, 7, 11, 13, 17, 19, and 23** (not skipping frames, plus every prime number lower than 24 frames / 1 second, excluding 2). For the CPU and RTX 4060 executions, the distorted video was a custom AV1 encode of the reference video. For the GTX 1060, as it does not have AV1 decoding hardware support, the H.264 reference video was compared to itself (the computations being performed should be equivalent anyway). ### Results and discussion The exact results of this experiment are shown in the following table: | Stride | ssimulacra2\_rs FPS | vszip FPS | GTX 1060 GPU FPS | RTX 4060 GPU FPS | |:-:|:-:|:-:|:-:|:-:| | 1 | 3.37 | 14.63 | 46.70 | 85.64 | | 3 | 9.55 | 37.67 | 133.86 | 245.73 | | 5 | 15.15 | 49.34 | 164.34 | 350.01 | | 7 | 20.45 | 56.37 | 197.17 | 400.94 | | 11 | 30.23 | 64.46 | 222.98 | 458.61 | | 13 | 34.54 | 66.65 | 239.17 | 488.83 | | 17 | 40.51 | 69.23 | 249.58 | 520.30 | | 19 | 42.82 | 69.99 | 258.71 | 525.44 | | 23 | 45.43 | 88.60 | 264.85 | 546.14 | Here they are in graph form, for better visualization: ![CPU vs GPUs decoded FPS](/img/blog_turbo-metrics_11-2024/ssimu2_cpu-gpu_dec.svg) On a superficial level, we can already make some observations: * SSIMULACRA2 computation with ssimulacra2\_rs is quite slow: 3.37 frames per second (without skipping frames) when using 6 threads. That is like 0.56 frames per second per thread. I know the CPU used is no beast by any measure, but beast CPUs are quite expensive and not that common (that is something to keep in mind). Those processing speeds are comparable to some of the slower-ish presets of SVT-AV1. - vszip is, in fact, considerably faster: Between ×4.34 (full video computation) and ×1.63 (stride of 19) with the same amount of threads. However, it seems that vszip does not scale as well as the other implementations (except for that weird performance boost with a stride of 23). * GPU computation of SSIMULACRA2 scores is impressive, considering the CPU performances: - Compared to ssimulacra2\_rs: Speedups for the GTX 1060 (an 8-year-old GPU) are between ×14.01 (stride of 3) and ×5.83 (stride of 23). Speedups for the RTX 4060 are between ×25.73 (stride of 3) and ×12.02 (stride of 23). - Compared to vszip: Speedups for the GTX 1060 are between ×3.70 (stride of 19) and ×2.99 (stride of 23). Speedups for the RTX 4060 are between ×7.51 (stride of 17) and ×5.85 (stride of 1 / full video computation). - It is amazing seeing how GPUs not only achieve real-time SSIMULACRA2 computation of entire videos, but more than ×1.94 and ×3.56 real-time processing, even (GTX and RTX, respectively). Nevertheless, the most interesting observation is made when we use computation strides bigger than 1: The performance does _not_ scale linearly with the stride, as it would be expected. This happens with both the CPU and GPUs, although it is more noticeable in the case of the GPUs due to their higher performance. Why is that? Well, there is actually a good explanation: The reported FPS are _decoded_ FPS. That is, the speed in which the software is decoding and processing all the video frames. All the frames in the video bitstream are decoded and reconstructed as images, but the SSIMULACRA2 score is only computed for some of them. SSIMULACRA2 computation affects decoding speed in that the decoding of each frame only starts after the processing of the previous has finished (kind of, this is not be entirely true for multithreaded CPU implementations, but we can ignore that). The faster the SSIMULACRA2 processing is done, the sooner the next frame starts decoding. And there is no faster processing than not doing any processing at all, that is, skipping the frame! However, **the decoding itself is a process that has to be done regardless**, and the time it takes should not be disregarded. Although decoding as a process is considerably faster than SSIMULACRA2 computation, it starts becoming the bottleneck when you both compute SSIMULACRA2 so fast, and skip computing frames. ### Considerations about video decoding performance [NVIDIA's Video Codec SDK 13.0 application note](https://docs.nvidia.com/video-technologies/video-codec-sdk/13.0/nvdec-application-note/index.html#nvdec-performance) gives these indicative per-NVDEC figures for 1080p YUV 4:2:0 decoding: * Pascal architecture decodes H.264 at up to 694 frames per second. * Ada architecture decodes H.264 at up to 903 frames per second. * Ada architecture decodes AV1 at up to 1,018 frames per second. Being proprietary NVIDIA technology, served as a black box through their API, we cannot be sure how the NVDEC engines are implemented and what exactly are their limitations under scenarios such as these (i.e., concurrent decoding of multiple video streams of same or different codecs). We can only trust the information NVIDIA provides. NVIDIA states that all GeForce products consist of a single NVDEC; however, we cannot be certain how its resources are managed or shared when working with multiple simultaneous decoding contexts. In our case, considering two videos are being decoded at the same time: * The GTX 1060 decodes up to 529.7 H.264 (2×264.85) frames per second when using a stride of 23 frames, which corresponds to 75% of its reported peak decoding performance for H.264. * The RTX 4060 decodes up to 546.14 H.264 + 546.14 AV1 (1092.28 total) frames per second when using a stride of 23 frames, which corresponds to 61.85% and 54.34% of its reported peak decoding performance for H.264 and AV1 respectively. Notice how, even though the NVDEC hardware resources are reportedly being shared between the two decodes, both are able to reach more than 50% peak performance **at the same time**. We can just focus on _processed_ FPS, that is, only not-skipped frames processed per second, or SSIMULACRA2 scores computed per second, to see how the processing performance degrades with increasing strides. | Stride | ssimulacra2\_rs FPS | vszip FPS | GTX 1060 GPU FPS | RTX 4060 GPU FPS | |:-:|:-:|:-:|:-:|:-:| | 1 | 3.37 | 14.63 | 46.70 | 85.64 | | 3 | 3.18 | 12.56 | 44.62 | 81.91 | | 5 | 3.03 | 9.87 | 32.87 | 70.00 | | 7 | 2.92 | 8.05 | 28.17 | 57.28 | | 11 | 2.75 | 5.86 | 20.27 | 41.69 | | 13 | 2.66 | 5.13 | 18.40 | 37.60 | | 17 | 2.38 | 4.07 | 14.68 | 30.61 | | 19 | 2.25 | 3.68 | 13.62 | 27.65 | | 23 | 1.98 | 3.85 | 11.52 | 23.75 | Here they are in graph form, for better visualization: ![CPU vs GPUs processed FPS](/img/blog_turbo-metrics_11-2024/ssimu2_cpu-gpu_comp.svg) It can be seen how the actual SSIMULACRA2 processing performance quickly decreases, as more and more relative time gets dedicated to decoding the videos. One should also note that the NVDEC hardware is the same for all GPUs in the same generation (e.g., all the RTX 40 GPUs). Therefore, even if the computation of the SSIMULACRA2 scores of a video could still be accelerated further by using more powerful GPUs, e.g., RTX 4070/4080/4090, that would only be significant when using small strides. As the stride increases, the video decoding will become the bottleneck for all those GPUs sooner or later, and all of them would perform similarly to the RTX 4060. In other words: **rather than spending more money on a more powerful GPU to compute SSIMULACRA2 faster, it is much more worthwhile to just increase the stride**; especially since the error incurred from using a stride greater than 1 usually is statistically almost insignificant. ### Conclusions As a quick summary of the experiment, we can highlight: * **GPU processing greatly accelerates SSIMULACRA2 computation**, with modern GPUs enabling speeds **faster than real-time** for most 1080p content (~85 fps), including gaming; and old GPUs still providing speeds faster than real-time (~46 fps) for popular, lower-fps 1080p video, such as movies and series. * As SSIMULACRA2 computations accelerate, **video decoding speed becomes a relevant factor, to the point of being the main bottleneck when the computation stride is large enough**. It is not worthwhile to increase the computation stride too much, as you will quickly get diminishing returns in performance. --- ### Further discussion: Possible improvements One may think that a possible solution to the decoding speed bottleneck, which does not require waiting until the next-generation GPUs implement faster decoding hardware, would be to **make the software decode only the frames that are going to be processed**. Nevertheless, that approach comes with significant caveats: In modern video coding, frames cannot be decoded independently of each other, and a full group-of-pictures (GOP) would need to be decoded to reconstruct any of its frames. Usually, GOPs are from 1 to 10 seconds long (24 to 240 frames in our test case), depending on the encoding settings. Using any computation stride smaller than the GOP size would not see any benefit from this technique; and greater computation strides **would only see limited benefits**. (Aside from the possible accuracy/representativeness concerns associated to using such big strides.) Additionally, when leveraging GPU programming, the actual hardware is often treated like a black box, and the available APIs may limit considerably the control the user has over it. Another, more useful approach one may think of, would be to exploit **computation overlapping**: To keep decoding frames while the SSIMULACRA2 scores are being computed, storing the decoded frames that are to be processed in a buffer. This seems to be partially implemented in TurboMetrics already, but only to some extent. Without any modification to the SSIMULACRA2 computations, the overlap in computation and decoding would still have limited effect for low strides (where SSIMULACRA2 computation is still the bottleneck), as every time the decoded frame buffer fills completely, the decoder would have to wait for it to have free space again. In theory, the SSIMULACRA2 computations could be sped up further (maybe requiring faster but more imprecise approximations). And, ideally, TurboMetrics should aim to approach the theoretical decode FPS limit in the computation of the SSIMULACRA2 scores, thus perfectly overlapping computing and decoding. In such scenario, skipping the computation of any frames would not make sense anymore, as no speed advantage would be obtained from it. However, this is still far from being a reality, as a speedup of at least ×10.31 is needed for an RTX 4060 to match its decoding speeds with a computation stride of 1. ## Short discussion on scoring error My original idea for this blog entry was to evaluate TurboMetrics both in terms of performance and scoring error. By "scoring error" I mean "the difference between the scores a CPU reference provides, and the ones TurboMetrics provides". However, once the first draft for that original version of this blog entry was completed, I realized I was not satisfied with the methodology followed and conclusions extracted from that second experiment. Therefore, I am putting off that evaluation for a second blog entry, that I will write once I make a new, bigger, and better, evaluation of the error. The main concern about the error the GPU computation incurs on SSIMULACRA2 scores is that SSIMULACRA2 scores have been tuned mathematically to correlate with visual quality, while the GPU implementation's scores _try to mimic_ those scores, but have not undergone any kind of mathematical tuning or assessment themselves. The GPU scores could be seen as "SSIMULACRA2 + random error", and that "random error" might break the desired correlation, thus reducing the reliability of the metric. Nevertheless, it is worth noting that SSIMULACRA2 is not by any means a kind of "perfect metric" or "absolute truth" about video quality assessment. **SSIMULACRA2 still is unreliable in some situations, and it does not always present a perfect correlation with visual observations**. You may find here and there specific examples of SSIMULACRA2 scores that do not accurately reflect the (perceived) quality of a compressed image or video. Besides, we should keep in mind that quality is a subjective matter. In that regard, and as an example, people who do not care much about film grain might give denoised videos higher quality scores than those that SSIMULACRA2 gives. (Other metrics, such as VMAF, are more tolerant to denoising.) With all this in mind, the results provided by TurboMetrics are not necessarily "incorrect", but rather provide a kind of "**alternative metric**". Just to give a preview on the error evaluation thus far, I will share the following: The preliminary results I obtained from that now-scraped evaluation seem to indicate that: * The scores provided by the GPU implementation of SSIMULACRA2 are **statistically different** to the real scores (provided by the CPU implementations). * **No linear correlation** between the real scores and the GPU scores has been found. * TurboMetrics **might tend to under-score**, compared to the real scores (i.e., the results for the frames tend to be lower than those a CPU implementation gives). This has yet to be assessed over a larger sample size to prove if it is really true, though. I am looking for help with the error evaluation for the follow-up blog entry. Mainly, I need a considerably larger sample of SSIMULACRA2 scores for videos, computed both in CPU and GPU. I myself do not have enough material (encoded videos and/or the SSIMULACRA2 scores for all their frames) yet to do a sufficiently good evaluation, and it would take me quite some time to get it. Thus, I am looking for anyone who has saved the SSIMULACRA2 scores for all the frames of any video they had assessed, and is able to re-assess those same videos using TurboMetrics. If you think you can help, and are willing to, please contact me through the [AV1 for Dummies Discord server](https://discord.gg/bbQD5MjDr3). ## Overall conclusions and final thoughts In this blog entry, we have tested the GPU implementation of SSIMULACRA2 available through TurboMetrics, in terms of performance (i.e., scores computed per unit of time). The main conclusions are the following: * All in all, TurboMetrics provides **impressive acceleration** of the SSIMULACRA2 computation. * **The performance of TurboMetrics does not scale linearly with the computational stride/frame skipping**. When increasing the stride, a point of diminishing returns is reached rather soon. - This is caused by the time taken to decode the video frames, which is non-negligible, and becomes the bottleneck when computing SSIMULACRA2 so fast and only for some of the frames. Hardware acceleration of computationally-intensive tasks is an important topic nowadays in several fields, especially considering how powerful current-day GPUs are; and I would hope for the teams developing these quality assessment metrics to consider it as an existing option. So far, only [XPSNR](https://github.com/fraunhoferhhi/xpsnr) has seemed to be concerned about computational complexity from the beginning, and the developers' response was not to consider hardware acceleration, but rather to simplify the design of their metric. After developing this blog entry, **my own recommendation** for assessing videos using SSIMULACRA2 would be to **use TurboMetrics, with a stride of 3 or 5 frames**. If you are willing to tolerate the error incurred by the GPU implementation, you should tolerate the error incurred by skipping frames, which probably is an order of magnitude lower. Of course, you should not take my recommendations blindly, and you should try it out yourself to see what works for you, and what you can tolerate. ## Future TurboMetrics' development is still ongoing, and I know Gui-Yom has tons of ideas for it. To begin with, I know the following ideas are being worked on: * Support for GPUs of other vendors (AMD, Intel). * Support for FFmpeg piping, to enable working with videos of any codec. * GPU acceleration of XPSNR. All these features would improve TurboMetrics significantly, while also opening the door for more tests to be performed. Additionally, as it has been mentioned in this blog entry, the SSIMULACRA2 computation can still be optimized further, including overlapping computation and decoding. Gui-Yom knows this, and plans to tackle that issue eventually. A list of features to improve TurboMetrics' computation of SSIMULACRA2, written by the main developer, can be found [here](https://github.com/Gui-Yom/turbo-metrics/blob/master/crates/ssimulacra2-cuda/README.md#how-to-do-better-). As I said earlier, I also plan to write a follow-up blog entry evaluating the scoring error incurred in the GPU implementation, to test the reliability of the scores provided by TurboMetrics. Additionally, it may be insightful to evaluate the error for GPU implementations of other video metrics, such as [VMAF CUDA](https://github.com/Netflix/vmaf/blob/master/libvmaf/README.md), and see if we find any similarities. I do not recall reading any analysis on the matter for other metrics, either. Nevertheless, I cannot give a date yet for when that follow-up blog entry might come out. I reckon it will take quite some time to perform the required experimentation, as well as processing and analyzing the results. Also, this blog entry is heavily missing information on 4K video, which I believe to be of high interest. The main reason for it is that most 4K content is served using HEVC, and TurboMetrics still does not support that codec. When support for it gets added, experiments with 4K video should be conducted. If you like TurboMetrics and have knowledge about programming, please consider [contributing to it](https://github.com/Gui-Yom/turbo-metrics) to the best of your ability. Together, we can make it even more awesome than it already is! And, lastly, thank you very much for reading! #### Special thanks * **Gui-Yom**, for developing TurboMetrics, and also for proofreading and green-lighting this blog entry. * **Trix**, for linking me to the Python script to easily compute SSIMULACRA2 using vszip (ssimulacravszip.py), and for helping me to get it to work the way I needed. {} --- # Welcome to Encode Wiki Canonical URL: https://encode.wiki/articles/welcome/ Collection: articles Status: current License: CC BY-SA 4.0 Checksum: 68e71c4247dec11538d72fa96b5b2e6303c5c1e8b792dab4cce25649e5973bab Encode Wiki is a community-maintained home for practical information about codecs and encoding tools. It combines durable reference documentation with reproducible guides, experiments, and benchmark articles. The initial corpus comes from the AV1 Community Contributors' [Codec Wiki](https://github.com/av1-community-contributors/codec-wiki). Its pages will be migrated with source provenance and modification notices under CC BY-SA 4.0. The application and deployment code written for Encode Wiki is separately licensed under MIT. The public site is designed to remain useful without accounts, analytics, or an AI service. Clean machine-readable editions will also make the reviewed corpus easier to use from other tools. --- # Codecs Canonical URL: https://encode.wiki/codecs/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: cdfe46bf0890572d92107769e68a64fb6cb1bff4cb234ac2830fd3e53e2d94de Codec pages describe how formats work, where implementations differ, and what interoperability or quality constraints matter in practice. The migrated collection will cover video, audio, image, subtitle, and general-purpose compression. --- # AAC Canonical URL: https://encode.wiki/codecs/audio/aac/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: da3674f44708d4b8f4aa30dcf157fd4017465d9f6a5bf572614070088c948bd3 AAC, or Advanced Audio Coding, is an umbrella for a number of different codecs. When people refer to AAC, they are often referring to the commonly used AAC-LC profile developed as part of the original AAC standard in 1997 (although there is a distinction between this version of AAC-LC, called MPEG-2 AAC, & MPEG-4 AAC which is newer). However, there are a number of other variants that have been created over time. These include: - AAC-LC (low-complexity AAC) - AAC-LD (low delay AAC) - AAC-ELD (enhanced low delay AAC) - HE-AAC (high efficiency AAC, uses Spectral Band Replication) - HE-AACv2 (high efficiency AAC v2, uses Spectral Band Replication + Parametric Stereo) - xHE-AAC (extended high efficiency AAC (kinda), uses USAC (Unified Speech & Audio Coding)) AAC is even used as a Bluetooth audio codec for encoding audio streams & sending them to a Bluetooth audio device. Encoding & even decoding some of the above formats can prove to be difficult, so it is worth exploring each codec individually. ## Format Breakdown Let's explore each codec individually. ### AAC-LC While AAC-LC was introduced alongside two higher complexity profiles (AAC Main & AAC-SSR), AAC-LC has seen much more widespread adoption through various distribution mediums for video (often paired with AVC video) & audio alike. AAC-LC is ubiquitous within the Apple ecosystem & was (& still is) used on YouTube before their switch to primarily using Opus. Encoding AAC-LC can be done with relative ease, accessible through nearly every encoder in the "Encoders" section. ### AAC-LD & AAC-ELD AAC-LD & AAC-ELD are both designed to transmit audio in instances where latency is very important. Both are far more efficient than previous low-latency audio coding offerings. Over AAC-LD, AAC-ELD offers better audio quality through Spectral Band Replication, lower latency, & a greater quality range with a lower bitrate minimum & higher maximum. ### HE-AAC High Efficiency AAC introduces Spectral Band Replication (SBR) to the AAC specification for the purpose of higher quality audio at lower bitrates. SBR is an encoding technique that allows the decoder to reconstruct higher frequencies from an audio signal given lower frequencies & data that informs the decoder about information in the higher frequencies, allowing them to be effectively restored from this helper data. In short, lower frequencies are encoded with extra detail incorporated to allow the reconstruction of higher frequency information in an audio signal. This feature alone allows HE-AAC to be much more efficient than AAC-LC at lower bitrates; the gap closes substantially at higher bitrates, however. ### HE-AACv2 High Efficiency AAC v2 introduces Parametric Stereo (PS), which further increases audio quality with an emphasis on lower bitrates. PS uses a mono signal downmixed from a multichannel stereo input alongside information about the spatial properties of the stereo input to allow the decoder to reconstruct a left & right channel using salient spatial data from the mono signal. HE-AACv2 combine PS with SBR for greater efficiency gains over HE-AAC & AAC-LC, although again the gap closes at higher bitrates. ### xHE-AAC Extended High Efficiency AAC is actually slightly different from xHE-AAC. While Extended High Efficiency AAC specifically includes all of HE-AACv2's coding techniques as well as compression techniques from the USAC specification, the Extended High Efficiency AAC profile was designed in such a manner that building an encoder for the format would allow it to be perfectly backwards compatible with past variations of AAC. xHE-AAC is a codec that combines the Extended High Efficiency AAC profile with further USAC coding techniques, specifically within the *MPEG-D DRC Loudness Control Profile*. For all intents and purposes, when someone refers to "USAC audio," they probably mean xHE-AAC. xHE-AAC further extends performance at lower bitrates compared to past variants of AAC. USAC's specialty & purpose was to create an audio codec that didn't compromise music performance for speech & vice versa, but rather automatically tuned its coding technique selection to perform the best on any given source. xHE-AAC is not widely supported. On a Mac, playback is only possible through QuickTime Player & other utilities which use CoreAudio for decoding. Windows 11 & Android support xHE-AAC natively, but Windows 10 & Linux will not allow you to play it back without some grief. The most effective way currently is to use an AAC plugin with foobar2000, although this plugin is 32-bit only, so you'll need 32-bit foobar2000. This works on Linux under WINE. ## Encoders There are a number of ways to encode each format, which I'll outline below. ### Fraunhofer FDK AAC Fraunhofer FDK AAC is a high quality, open-source AAC encoder by [Fraunhofer IIS](https://en.wikipedia.org/wiki/Fraunhofer_Society). It was introduced with the release of Android 4.1 and has since been forked to a [seperate repository](https://github.com/mstorsjo/fdk-aac). Issues regarding the legality of its distribution have arisen since the Free Software Foundation declared the license incompatible with the GPL. The license does not grant the user rights to the patented technologies used in the source code, and therefore restricts its use. However, this topic is still debated as Debian does not consider FDK AAC free software while Red Hat does. Another similar tool, FhG-AAC, performs similarly to FDK-AAC but with greater precision as it performs floating point operations. It can be used through the proprietary Windows media player Winamp. FDK-AAC can encode the following formats: `AAC-LC AAC-LD AAC-ELD HE-AAC HE-AACv2` ### Core Audio Apple's proprietary Core Audio Toolbox encoder is a popular, high-quality choice for encoding AAC. This encoder can be used in macOS via Apple's own `afconvert` utility, or in FFmpeg by specifying `aac_at` as the audio encoder. Windows users can also take advantage of Core Audio by using the free [qaac](https://github.com/nu774/qaac) command-line utility, but users will need to either have [iTunes](https://support.apple.com/en-ca/HT210384) installed, or extract the libraries from said installer, using the [makeportable](https://github.com/nu774/makeportable) script. Core Audio can encode the following formats: `AAC-LC, HE-AAC, HE-AACv2`* *`HE-AACv2` encoding is only supported on macOS. ### FFmpeg AAC FFmpeg's native AAC encoder is known for being completely free & open source, licensed under the GNU General Public License (or LGPL, depending on how FFmpeg is configured). If you want a completely FOSS solution for encoding AAC, FFmpeg AAC is one of the only places you'll find this. It is not known for being particularly feature rich or high quality & doesn't support the entire profile of the AAC variants it can encode, but it gets the job done in most scenarios. FFmpeg's AAC encoder can be called with a simple `-c:a aac` parameter, but there are more granular options available. By default, the profile is `aac_low` which encodes MPEG-4 AAC-LC. The `mpeg2_aac_low` profile encodes simpler, lower-quality MPEG-2 AAC from 1997, while `aac_main` encodes the more obscure main AAC profile from the 1997 MPEG-2 specification. Finally, `aac-ltp` encodes AAC long-term prediction introduced in MPEG-4. Using FFmpeg AAC, we can encode the following formats: `AAC-LC` ### FAAC FAAC is an old AAC encoder, name meaning Freeware Advanced Audio Coder. It is one of the lower quality options, & isn't recommended for general use. It only supports `AAC-LC`. ### Nero AAC The discontinued Nero AAC audio coder had its last release in 2010. The developer of FAAC worked on Nero AAC, & the Nero AAC encoder & decoder are proprietary. Nero AAC was known to produce decent quality output while it was maintained, although development has since stalled. Nero AAC can encode the following formats: `AAC-LC, HE-AAC, HE-AACv2` ### Exhale Exhale is the only encoder on this list capable of encoding xHE-AAC audio. While it does not excel at this task compared to proprietary competition, there isn't really a straightforward way to encode xHE-AAC outside of using exhale on most platforms right now. Given that fact, it is the only xHE-AAC encoder many can consider using due to the lack of real competition. It can encode Extended HE-AAC audio with or without eSBR, although encoding using eSBR produces higher quality results. `exhale` can encode the following formats: `xHE-AAC` ## Conclusion Alternatives to AAC include Opus, Vorbis, MP3, & FLAC, among others. - Vorbis & MP3 are considered to be worse, although Vorbis has its moments & is entirely royalty-free unlike AAC. - FLAC is a lossless audio codec with an entirely different use case. It is important to reinforce that lossy audio codecs are not replacements for lossless, as good as they might get. - Opus is competitive with HE-AACv2 & xHE-AAC moreso than the older, worse variants, & is widely supported as well as royalty free. The open source reference encoder `opusenc` is an excellent implementation of the format & the tooling is much easier to use compared to AAC's frankly nightmarish assortment of various options with numerous individual downsides. xHE-AAC is generally better than Opus, especially at lower bitrates, which is a fact that merits consideration. --- # ALAC Canonical URL: https://encode.wiki/codecs/audio/alac/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Dolby Digital Canonical URL: https://encode.wiki/codecs/audio/dolby/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 2f0c7fc0ca622bf9ec9f6dcf207c444fd69d915efccabf76af917e224edd08e0 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Dolby Digital is a family of both lossless and lossy audio compression algorithms and technologies. ## Format Overview ### AC-3 Originally known as Dolby Digital, AC-3 was first released in 1991 to provide digital 5.1 sound in cinemas from 35mm film reels. AC-3 is notable for being the first audio codec to make use of the “Modified Discrete Cosine Transform” algorithm. The codec has seen widespread use an adoption, due to its prevalence in DVDs, TV, and Blu-rays as a surround codec. #### Dolby Digital Surround EX Like Dolby’s earlier Pro Logic technology, Dolby Digital Surround EX matrixes a sixth, centre back surround channel into the left and right surround channels of a 5.1 stream, allowing for a 6.1 mix to be unfolded when played on a 6.1 or 7.1 system with EX decoding. This technology is fully backwards compatible with existing AC-3 decoders, producing the standard 5.1 stream. Surround EX was first introduced in 1999 with the release of “Star Wars: Episode I – The Phantom Menace”. ### E-AC-3 Often referred to as “Dolby Digital Plus”, E-AC-3 is the successor to Dolby’s earlier AC-3 codec, featuring support for higher bitrates (6,144kbps vs 640kbps), more channels (15 vs 5), and additional coding tools allowing for more efficient encoding. E-AC-3 can be found in the short-lived HD-DVD format, Blu-ray discs, and as the main surround codec for most streaming services, particularly if Dolby Atmos is used. Contrary to popular belief, E-AC-3 is not backwards compatible with AC-3, rather Dolby mandates that all E-AC-3 decoders can also decode standard AC-3 content. As E-AC-3 is an optional codec on Blu-ray, all discs encoded with E-AC-3 encode the first 5.1 channels as AC-3, with the additional rear channels/Atmos content being encoded as E-AC-3. ### TrueHD Dolby’s TrueHD is a lossless multi-channel audio codec based on Meridian’s Lossless Packing (MLP) codec, although the two aren’t compatible with each other. TrueHD is mainly used on Blu-ray and supports Dolby Atmos’s spatial audio data. The TrueHD specification supports up to 16 audio channels (although the Blu-ray specification limits this to 7.1) with a sample rate of 192KHz and a bit depth of 24 bits. As TrueHD is an optional codec on Blu-ray, each TrueHD steam includes a backup AC-3 stream encoded alongside it for compatibility purposes. Since 2010, Dolby TrueHD has seen a decline in usage in favour of DTS-HD Master Audio on Blu-ray discs, but has seen a slight resurgence as the codec used for Dolby Atmos audio, but DTS-HD MA is still more common on non-Atmos titles. ### AC-4 To be added. ### Atmos To be added. --- # FLAC Canonical URL: https://encode.wiki/codecs/audio/flac/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: f3b3078582a001f31b6f701e3d80f040a7c7b039b7979b27e15740d7e1f2d52e > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! FLAC (Free Lossless Audio Coding) is an open-source lossless audio codec with widespread support & compatibility released in 2001. It represents the most efficent lossless audio format in common use today. FLAC is commonly contained in a ogg container with either a `.flac` or `.ogg` extension. It can less commonly be used within a matroska container (`.mkv` or `.mka`) for mixing with a video stream. > **Caution — It is not recommended to transcode a lossily encoded file to FLAC as the file size will grow tremendously while any quality loss from lossy encoding will remain. FLAC is best if you need to preserve existing lossless audio..** > > ## Software support FLAC is supported by the majority of web browsers and media players in common use as of 2024. ### WAV to FLAC using [FFmpeg](/tools/utilities/ffmpeg/): ```bash ffmpeg -i example.wav -c:a flac example.flac ``` ### WAV to FLAC using FLAC command-line tool: You can include an argument of a number 0-8 to specify the compression effort, 0 being fastest and 8 having the highest compression. ```bash flac example.wav -8 -o example.flac ``` --- # Intro Canonical URL: https://encode.wiki/codecs/audio/intro/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 48745b9ccc30ff0e86255e3a9bc225a1a0d7e9b08408f19dc41409e44b515206 # Introduction to Lossy & Lossless Audio Compression > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Digital audio is the representation of sound recorded in, or converted into, digital form. To understand digital audio, it's crucial to grasp some fundamental concepts, including **sampling**, **Nyquist Frequency**, and the **Nyquist-Shannon Sampling Theorem**. ## Sampling & the Nyquist Frequency **Sampling** is the process of converting a continuous, analog audio signal into a discrete digital signal by measuring the amplitude of the audio signal at uniform intervals. The frequency of this measurement is known as the *sampling rate*, typically measured in samples per second, or Hertz (Hz). For example, audio CDs use a sampling rate of 44,100 Hz, which means the audio signal is sampled 44,100 times every second. To accurately represent a wave, you need at least two measurements per cycle; one to capture the peak of the wave, and one to capture the trough. If you sample less than twice per cycle, you can't distinguish between different frequencies; this is where the **Nyquist frequency** comes from. Named after Harry Nyquist, the **Nyquist frequency** is *half of the sampling rate of a discrete signal processing system*. For a given sampling rate, the Nyquist frequency represents the highest frequency that can be accurately sampled without introducing errors such as aliasing. For example, with a sampling rate of 48,000 Hz, the Nyquist frequency is 24,000 Hz. Capturing frequencies above the Nyquist frequency for a given system can bring about **aliasing** artifacts. Aliasing occurs where high-frequency components appear as lower frequencies in the sampled signal, distorting the information. The **Nyquist-Shannon sampling theorem** states that to avoid aliasing, the sampling rate must be at least twice the highest frequency present in the signal. This theorem is crucial for ensuring that the digital representation of the audio signal retains all the information from the original analog signal without distortion. ## Lossless Audio Compression The main benefit of lossless compression is the preservation of audio quality, making it ideal for professional audio production, archiving, and situations where high fidelity is required. However, lossless files are significantly larger than their lossy counterparts, which can be a drawback for storage and (especially) streaming. > **Note — Lossless Compression.** > > If you would like to dive more deeply into the topic of lossless compression, you can check out the [Lossless Compression](/start-here/lossless/) entry in the Introduction section of the wiki. [FLAC](/codecs/audio/flac/), [WavPack](/codecs/audio/wavpack/), & [ALAC](/codecs/audio/alac/) are examples of popular lossless audio codecs that you are likely to encounter in the wild. ## Lossy Audio Compression The primary advantage of lossy compression is the significant reduction in file size, making it ideal for streaming, portable devices, and situations where storage space is limited. However, the trade-off is a potential loss in audio quality, which may be noticeable in critical listening environments. > **Note — Lossy Compression.** > > If you would like to dive more deeply into the topic of lossy compression, you can check out the [Lossy Compression](/start-here/lossy/) entry in the Introduction section of the wiki. [MP3](/codecs/audio/mp3/), [AAC](/codecs/audio/aac/), [Vorbis](/codecs/audio/vorbis/), & [Opus](/codecs/audio/opus/) are some examples of popular lossy audio codecs that you are likely to encounter in the wild. ## Conclusion Understanding these baseline principles of digital audio processing is essential for informed reading when it comes to the rest of the Audio section of this wiki. We hope this page helped you grasp some of the fundamental concepts that underpin digital audio processing. --- # MP3 Canonical URL: https://encode.wiki/codecs/audio/mp3/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 049a42856fd9de487f3e823aeef0d8e268a8f2b505302770d4b0d8fb9312d322 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! MP3, formally known as MPEG-1 Audio Layer III or MPEG-2 Audio Layer III, is a coding format for digital audio. It was developed largely by the Fraunhofer Society in Germany under the lead of Karlheinz Brandenburg, with support from other digital scientists in other countries. MP3 is defined in two ISO/IEC specification families: MPEG-1: 11172-3 and MPEG-2: 13818-32. It uses lossy compression, which often allows for large reductions in file size compared to uncompressed audio. Lossy MP3 compression works by attempting to reduce (or approximate) the accuracy of certain components of sound that could be considered (by some [psychoacoustic](/start-here/terminology/#perceputal--psychovisual--psychoacoustic) analysis) to be beyond the hearing capabilities of most humans and storing the coefficients corresponding to these more salient frequency bands. Compared to CD-quality digital audio, MP3 compression can commonly achieve a 75 to 95% reduction in size. For example, an MP3 encoded at a constant bit rate of 128 kbit/s would result in a file approximately 9% of the size of the original CD audio. MP3 audio is considered transparent at 320kb/s. It is still very common to see MP3 files in the wild today, despite the fact that the format was finalized in 1993 (with modifications in 1995 to support lower sample rates and bit rates). This is due to the fact that MP3 could be considered the first widely adopted audio format that allowed for high quality audio to be compressed to a relatively small file size. Compared to more modern formats like [Opus](/codecs/audio/opus/) and [AAC](/codecs/audio/aac/), MP3 may not seem as impressive, but it is still widely supported by many devices and pieces software and has certainly left a powerful legacy to live up to for modern codecs. --- # Opus Canonical URL: https://encode.wiki/codecs/audio/opus/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: c4d6e95893650c36cd2696063d441209f546bb0be22f9c8f766922296b5f6728 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Opus is an open-source audio codec that has largely replaced [Vorbis](/codecs/audio/vorbis/) as the standard open audio codec. It is the recommended codec for usage in WebM video containers in tandem with the [VP9](/codecs/video/vp9/) or [AV1](/codecs/video/av1/) video codecs. Opus is known for its incredible coding efficiency and unique multi-channel optimizations. Stereo Opus audio reaches [transparency](https://en.wikipedia.org/wiki/Transparency_(data_compression)) (psychoacoustically lossless audio quality) at 128kb/s, compared to [AAC](/codecs/audio/aac/)'s generally agreed upon 256kb/s and [MP3](/codecs/audio/mp3/)'s 320kb/s. Transparency varies based on the type of content & the encoding implementation used, especially for codecs other than Opus, and the values provided above may be debated to a degree. Opus is described on [opus-codec.org](https://opus-codec.org/) as a "totally open, royalty-free, highly versatile audio codec. Opus is unmatched for interactive speech and music transmission over the Internet, but is also intended for storage and streaming applications. It is standardized by the Internet Engineering Task Force (IETF) as [RFC 6716](https://datatracker.ietf.org/doc/html/rfc6716) which incorporated technology from Skype’s SILK codec and Xiph.Org’s CELT codec." Opus supports the following features: - Bitrates from 6 kb/s to 510 kb/s (with a maximum of around 255 kb/s per channel on non stereo layouts) - Sampling rates from 8 kHz (narrowband) to 48 kHz (fullband) - Frame sizes from 2.5 ms to 60 ms - Support for both constant bitrate (CBR) and variable bitrate (VBR) - Audio bandwidth from narrowband to fullband - Support for speech and music - Support for mono and stereo - Support for up to 255 channels (multistream frames) - Dynamically adjustable bitrate, audio bandwidth, and frame size - Good loss robustness and packet loss concealment (PLC) - Floating point and fixed-point implementation *via opus-codec.org and wiki.hydrogenaud.io*. ## Format Breakdown Opus is a hybrid audio codec, composed of two codecs as mentioned above. These are Skype's SILK codec for voice & Xiph.Org's CELT codec. Opus's initial name, Harmony, may have been because of the "harmony" of these two codecs and the musical connotation of harmony. ### SILK SILK, initially from Skype, was designed to be used for voice calls on Microsoft products like Skype. The first stable release of the codec was in 2009, and since then it has been freely licensed under the BSD 2-Clause license which has allowed for its adoption into Opus. The version of SILK used in Opus is substantially modified from - and not compatible with - the standalone SILK codec previously described here. SILK is optimized for speech, and so has limited sample rates as follows: > Narrowband: 3-4000hz > Mediumband: 3-6000hz > Wideband: 3-8000hz SILK's latency is 10 to 60ms based on the desired framesize + 5ms lookahead to estimate noise shaping + (potentially) 1.5ms sampling rate conversion overhead if the input audio needs to be resampled. ### CELT Much like SILK, CELT is under the BSD 2-Clause license. The preview release came out in 2011. CELT stands for "Code-Excited Lapped Transform" and was designed to be the true successor to Vorbis, even being dubbed as "Vorbis II" during its initial development as part og Xiph.Org's "Ghost" project in 2005. CELT was designed to be a full-band general purpose codec without a particular specialization for a certain kind of audio, making it distinctly different from Xiph's [Speex](/codecs/audio/speex/) codec & more similar to Vorbis. It is computationally simple relative to competing codec technologies like [AAC](/codecs/audio/aac/) & even Vorbis, enabling extremely low latency that is competitive with [AAC-LD](/codecs/audio/aac/#aac-ld--aac-eld). CELT can work with the following sample rates: > Narrowband: 3-4000hz > Mediumband: 3-6000hz > Wideband: 3-8000hz > SuperWideband: 3-12000hz > Fullband: 3-20000hz ## Encoders ### Opusenc Opus's reference encoder is [opusenc](https://github.com/xiph/opus), which is known for its fantastic performance and versatility. It is licensed under the BSD 3-clause license as part of the reference libopus library. There are a myriad of options that may be used to encode with opusenc, but the utility is considered to have sane encoding defaults for local storage & playback. The best options will be outlined below. Usage: `opusenc [options] input_file output_file.opus` - `--bitrate #.###` Sets the overall target bitrate in kbit/s. Most encoders use *bits* per second, meaning you have to specify "128K" for 128kbit/s for example. Opus doesn't follow this, so you'd just have to type "128" though keep in mind using efficient VBR encoding means the final bitrate may be different than the target. Opus supports bitrates from 6 kb/s to 510 kb/s. - `--vbr` Tells the encoder to encode using a variable bit rate, allocating more or less bits when necessary to preserve overall fidelity per bit. This is the best option for local storage & playback, and is *enabled by default.* - `--cvbr` Tells the encoder that it is allowed to vary the bitrate like with VBR, but it must constrain the maximum bitrate at any given moment to the value provided. - `--hard-cbr` Tells the encoder to use a constant bitrate the whole time. - `--music` & `--speech` Forces the AI content-detector built into opusenc to treat the input as either speech or music. The bitrate range where this is relevant is around 12-40kb/s. - `--comp #` Sets the encoder complexity to a value from 0 to 10, 0 being the least complex & 10 being the most. *The default is 10.* - `--framesize #` Sets the maximum encoder frame size in milliseconds. Lowering this is useful for improving latency at the expense of audio quality per bit. It is worth noting that 40 & 60ms framesizes are just multiple 20ms frames stitched together via opusenc's default behavior, and are not considered useful as they just lower the encoder's adaptability which can worsen both latency & coding efficiency. *The default value is 20.* - `--expect-loss #` Percentage value for expected packet loss. Not useful for local encoding & playback, but useful for real-time applications. *Default value is 0.* - `--downmix-mono` Downmixes multiple channels into a single channel. - `--downmix-stereo` Downmixes multiple channels into two channels, left & right, given more than two channels are provided to the encoder. - `--no-phase-inv` Disables phase inversion. Helpful when downmixing stereo to mono, although this is the default behavior in that scenario since libopus 1.3. Slightly decreases stereo audio quality. - `--max-delay #` Sets maximum container delay in milliseconds, from 0-1000. *Default is 1000.* Looking at the default values for the encoder flags, opusenc almost always follows the best practices for every default value. This makes it very easy to use, and it is as simple as plugging in a source of some kind and using only the most basic commands to encode with opus. An example opusenc command: ```bash opusenc "input.wav" "output.opus" --bitrate 96 ``` [FFmpeg](/tools/utilities/ffmpeg/) using libopus: ```bash ffmpeg -i "input.flac" -c:a libopus -b:a 128K "output.ogg" ``` If you'd like to learn more about opusenc & its recommended default behavior, read this article on [Opus Recommended Settings](https://wiki.xiph.org/Opus_Recommended_Settings#Bandwidth_Transition_Thresholds). > **Info — Existing bug in ffmpeg.** > > Due to a bug in ffmpeg [(#5718)](https://trac.ffmpeg.org/ticket/5718), ffmpeg won't automatically remap `5.1(side)` to `5.1` when using libopus. > To remap the channel layout explicitly, try this: > ```bash > ffmpeg -i "input.flac" -c:a libopus -af aformat=channel_layouts=5.1 "output.ogg" > ``` > > > **Tip — You can handle arbitrary audio stream mappings with this:.** > > > > ```bash > > -af aformat=channel_layouts=7.1|5.1|stereo -mapping_family 1 > > ``` ### FFopus FFopus is an experimental native opus encoder from FFmpeg. It is not widely regarded as providing any decent uplift in coding efficiency compared to libopus, and is usually considered worse; its only merit is being able to handle 5.1(side) streams while libopus in FFmpeg cannot. It only implements the CELT part of the Opus codec. FFopus usage: ```bash ffmpeg -i "input.wma" -c:a opus -b:a 128K -strict -2 "output.opus" ``` ### vac-enc [VAC](https://github.com/gianni-rosato/vac-enc), or Value Added Codec, is a libopus encoder that uses SoX to resample inputs & supports output to `.ogg` rather than exclusively `.opus`. Better resampling theoretically leads to better coding efficiency, but vac-enc hasn't been thoroughly tested. Encoding a 16-bit signed little endian `pcm_s16le` WAV to 128kbit/s Opus in an OGG container: ```bash vac-enc input.wav output.ogg 128 ``` --- # Speex Canonical URL: https://encode.wiki/codecs/audio/speex/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: b1fbea3c71e31ae2568850b9fca8d528a6dd9ae95098ae71fb2a8bc6bb2fb20f > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Speex is an open-source audio codec designed for speech. It has largely been replaced by [Opus](/codecs/audio/opus/). --- # Vorbis Canonical URL: https://encode.wiki/codecs/audio/vorbis/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 6200f4ffbe8c1165d809169170b6607b434c428ef111d860209537516dc41f7c > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Vorbis is an open-source audio codec first released in 2000, maintainted by the Xiph.org Foundation. It has seen great success in its usage by Spotify, among others. It is the default audio codec for Minecraft's sounds & music. It has largely been replaced by [Opus](/codecs/audio/opus/). ## Usage Vorbis is supported in [ffmpeg](/tools/utilities/ffmpeg/). ### Encoder ```bash ffmpeg -i input.wav -c:a libvorbis output.ogg ``` ### Decoder ```bash ffmpeg -i input.ogg output.wav ``` See [ffmpeg](/tools/utilities/ffmpeg/) for more options. --- # WavPack Canonical URL: https://encode.wiki/codecs/audio/wavpack/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: b5239f5b8163b63642e0b984c5e72f96a2d16af91419a5760047ee05fdae20e6 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! WavPack is an open-source lossless audio codec with support for lossless & lossy compression with a unique hybrid compression mode for compressing a lossy audio stream alongside a lossless reference. Created by David Bryant in 1998, it gained a lot of software support, although not as much as [FLAC](/codecs/audio/flac/). Compressed file size is somewhat between [FLAC](/codecs/audio/flac/) and heavier state-of-art lossless audio compressors like TAK, OptimFrog or SAC. Compared to [FLAC](/codecs/audio/flac/), WavPack usually gives a lower bitrate at the expense of slightly more resource usage. ## Features WavPack is one of the most robust and feature-rich lossless audio codecs. Some notable features include: - [Hybrid mode](#hybrid-mode) - Support for 1-32 bit integer/floating point audio streams - Muxable into Matroska `.mkv` container - Multichannel with up to 4096 channels - APEv2/ID3v1 tagging format - RIFF chunks support - Multithreaded encoding/decoding - Error detection using CRC32 checksums and optionally also MD5 hash of original audio data ## Hybrid Mode Hybrid Mode is not to be confused with hybrid codecs like [Opus](/codecs/audio/opus/). WavPack uses the same algorithm for both lossy and lossless mode. When using lossy mode, the encoder transmits only the unary magnitude and the sign bit of Recursive Golomb encoded residuals. During decoding, those data points can be further enhanced if the correction file is provided. WavPack can produce 2 output files when using Hybrid Mode. The main `.wv` file with truncated (lossy) residuals and a `.wvc` correction file containing the enhancement layer. When both files are provided to the decoder, it should be able to recreate original audio data. Otherwise, if only the `.wv` file is available, the decoder will decode lossy audio stream. ## Format Breakdown Like in many lossless audio codecs, WavPack only encodes the prediction error value. In the default `fast` mode, prediction is just extrapolation of the previous two samples. More sophisticated predictors are used with higher encoding modes. Due to poor performance, unpredictability, and other problems with floating-point arithmetic in CPUs of its time, WavPack only uses integer arithmetic even when operating on IEEE float data. Nowadays, many of those issues were addressed, however it could still make porting WavPack to chips with no FPU support much easier. The encoding process consists of 3 main steps: - Joint stereo processing - Converts the stereo channels to the standard difference and average, removing inter-channel correlations. - Multipass decorrelation - Includes multiple prediction passes where the number of passes and predictor type depend on the selected encoding mode, removing intra-channel correlations between neighboring audio samples. - Entropy coding the residuals with Recursive Golomb Coding - Instead of Rice Coding, the author proposed a new technique that combines Golomb and Elias gamma code to better address the nature of audio data. ## Encoders ### wavpack ```bash title="Default options" wavpack input.wav -o out.wv ``` ```bash title="Fast, lowest compression, md5 hash" wavpack input.wav -f -m -o out.wv ``` ```bash title="Very slow, highest compression, 8 threads" wavpack input.wav -hh -x6 --threads=8 -o out.wv ``` ```bash title="Lossy, slow, 240kbps" wavpack input.wav -b240 -h -x3 -o out.lsy.wv ``` ```bash title="Highest hybrid compression, very slow, 4bps" wavpack input.wav -b4 -cc -hh -x6 -o out.hyb.wv ``` `wvunpack` can be used to decode resulting `.wv` files, however most major [media players](/tools/video-players/) like MPV or VLC already have (limited) WavPack support. Options: - `-f` Faster encode/decode at the expense of larger file size - `-h` Slower encode/decode with higher compression - `-hh` Slowest encode/decode with highest compression - `-x0` Disable extra filters - `-x3` Try all predefined filters, slow, higher compression - `-x6` Generate custom filters, very slow, best compression - `-b240` Enable lossy mode, set bitrate to `240kbps` (acceptable range is `24`-`9600` but it won't get lower than `2 bits per sample`) - `-b4` Enable lossy mode, set `bits per sample` to `4` (acceptable range is `2`-`23.9`) - `-c` Enable hybrid mode (will produce `.wv` and `.wvc` file) - `-cc` Enable and optimize for hybrid mode, might lower decoding speed and hurt quality - `-m` Include MD5 hash of original audio data in the output file - `--threads=8` Use `8` threads (acceptable range is `1`-`12`) For more detailed description of all available options, see the [manual](https://www.wavpack.com/wavpack_doc.html). ### FFmpeg [FFmpeg](/tools/utilities/ffmpeg/) has its own native WavPack encoder and decoder. It used to also support `libwavpack` with `--enable-libwavpack`, however it was [removed](https://hydrogenaud.io/index.php/topic,120038.0.html) due to interface incompleteness. The native encoder is single-threaded and doesn't support neither Lossy or [Hybrid](#hybrid-mode) Mode. It uses the `-compression_level` parameter to control speed to compression ratio. ```bash title="Fastest, lowest compression" ffmpeg -i input.wav -compression_level 0 out.wv ``` ```bash title="Slowest, highest compression" ffmpeg -i input.wav -compression_level 8 out.wv ``` For all possible parameters, consult the [FFmpeg documentation](https://www.ffmpeg.org/ffmpeg-codecs.html#wavpack). ## Adoption issues As of 2024, WavPack has been largely superseded by [FLAC](/codecs/audio/flac/), which became the de facto standard for lossless audio on the Web and in Hardware. The implementation of WavPack in media software is often incomplete. FFmpeg doesn't support [Hybrid Mode](#hybrid-mode), and other media players usually don't support it either. There are also [issues](https://www.reddit.com/r/ffmpeg/comments/tpptoi/using_hybrid_wavpack_in_video/) with its support in the `.mkv` container. Without this feature, WavPack doesn't provide much benefit over already widespread [FLAC](/codecs/audio/flac/). The compressed file might be slightly smaller, however music streaming companies tend to choose well-standardized FLAC which also has the benefit of `DRM` support in the `.mp4` container (apparently very important thing on the modern web). Even if [Hybrid Mode](#hybrid-mode) had better software support, the minimum lossy setting is `2 bits per sample`. That translates to around `200kbps` with stereo audio track which is quite high. The quality of WavPack lossy mode is also somewhat lacking compared to modern lossy codecs such as [Opus](/codecs/audio/opus/) or [AAC](/codecs/audio/aac/) because it doesn't utilize any psychoacoustic model. ## Notes - WavPack was one of the first compressed lossless audio codecs preceding Monkey's Audio (2000), [FLAC](/codecs/audio/flac/) (2001), OptimFROG (2002) and [ALAC](/codecs/audio/alac/) (2004). - OptimFROG Dualstream is a feature of OptimFROG codec serving a similar purpose to WavPack Hybrid, however it was introduced much later. - Apparently there are some devices with WavPack [Hardware Support](https://hydrogenaud.io/index.php/topic,119143.0.html). **References:** - [WavPack Compression Techniques](https://www.wavpack.com/WavPack.pdf) - [WavPack Technical Document](https://www.wavpack.com/technical.htm) - [Old WavPack File Format Specification](https://web.archive.org/web/20160705230053/http://www.wavpack.com/file_format.txt) - [Hydrogenaudio Wiki](https://wiki.hydrogenaud.io/index.php?title=WavPack) - [Multimedia.cx Wiki](https://wiki.multimedia.cx/index.php/WavPack) - [Wikipedia](https://en.wikipedia.org/wiki/WavPack) - [FFmpeg WavPack Source](https://ffmpeg.org/doxygen/3.3/wavpackenc_8c_source.html) --- # 7z Canonical URL: https://encode.wiki/codecs/data/7z/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: d743e0c53e92958e45aff500c1f636637baa61400ab719115baad11f8e3375c1 # 7-zip (7z) > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! 7-zip (7z) is a file format that supports several different data compression, encryption, & pre-processing algorithms. It was introduced by the 7-Zip archiver, which is free and open-source software for dealing with various data compression formats including formats similar to 7z like [XZ](/codecs/data/xz/). The 7-zip format has some noteworthy advantages over the popular [ZIP](/codecs/data/zip/) format. - The 7-zip utility can compress files to the 7z format "30-70% better" than to ZIP format despite having a highly efficient ZIP encoder. It mainly uses the LZMA & LZMA2 algorithms, which are more modern than DEFLATE and usually compress better. - 7-zip can encrypt files with AES-256 using a user provided password. AES-256 is more secure than the ZipCrypto encryption often used by ZIP. - 7-zip can support files up to 16 exabytes in size, while traditional ZIP has a 4 GB limit (ZIP64, which is commonly used, does not suffer from this 4 GB limitation so this is less relevant now). 7-zip also supports various pre-processing filters, which can improve compression for certain types of data like executables and binaries. However, 7-zip also has some drawbacks and limitations. - 7-zip is not as widely supported as ZIP by other software and platforms. Some users may need to install additional programs or plugins to open or extract 7z files. - Slower speed: 7-zip archives may take longer to compress or decompress compared to ZIP. This is somewhat mitigated by the 7-zip utility's effective parallelization when decoding, but this only affects real time as opposed to user time meaning it is still likely going to be more expensive to decompress than ZIP. - 7-zip does not have any built-in mechanism to repair corrupted or damaged archives. Users may need to use third-party tools or backup copies to recover their data1 7z archives are supported natively by macOS & many Linux distributions. --- # brotli Canonical URL: https://encode.wiki/codecs/data/brotli/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: fcb9390f97afe5d73823e983d3530254de9f20f8c193865873135c302e62ebc0 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Brotli was released by Google in late 2013, & it is commonly used on the Web for content delivery. It is a core part of the `.woff2` Web Open Font Format, allowing web fonts to be smaller when sent to users as part of a website. It is not very common to pass around `.tar.br` Brotli archives like you would with [gzip](/codecs/data/gzip/) or [xz](/codecs/data/xz/), so it is perfectly acceptable that such files aren't really compatible anywhere. Brotli is almost universally compatible across the Web, supported by as much as 96% of the World Wide Web's users. Brotli is based on LZ77 & Huffman coding, much like ZIP. It also uses context modeling to allow the use of multiple Huffman trees for the same alphabet in the same block; this essentially means that based on the context of the data being compressed, it can be compressed more efficiently especially if it contains multiple different kinds of data. Brotli was co-authored & partially developed by Jyrki Alakuijala, who also worked on [JPEG-XL](/codecs/images/jxl/) & the efficient [JPEG](/codecs/images/jpeg/) encoder jpegli. JPEG-XL's metadata information is usually Brotli-compressed. --- # bzip2 Canonical URL: https://encode.wiki/codecs/data/bzip2/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 12cc185b829d740759a2ce290392077cb24aea9568b9a637ad23a9012574dc74 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! bzip2 is a open source file compression format and utility. It's efficency is slightly better than [zip](/codecs/data/zip/), but worse than lzma based formats like [xz](/codecs/data/xz/) and [7z](/codecs/data/7z/). bzip2 cannot be used to compress mutliple files at once, you should collate files together into a [tarball](/codecs/data/tar/) to compress mutliple files using bzip2. --- # gzip Canonical URL: https://encode.wiki/codecs/data/gzip/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e53130191d171e6ca5ea317679be73b950f34e8282666d06fa01087011c8c313 Gzip is a DEFLATE implementation for use with individual files. It is popular on Unix-like systems such as Linux & macOS, and is often seen paired with [`tar`](/codecs/data/tar/) to create `.tar.gz` archives. Formats like [ZIP](/codecs/data/zip/) & [PNG](/codecs/images/png/) also use Deflate to different effects. ## Format Breakdown While ZIP is a multi-file archiving format that can compress multiple files into a single compressed file, Gzip is a single-file format that compresses a single file into a single compressed file. Both use DEFLATE for compression. ZIP supports encryption, while Gzip does not. ZIP also stores more extensive header information. ## History In order to properly understand the gzip format, we must first talk about ZIP. A lot of similar or identical information is covered in our ZIP entry. The ZIP format was developed by Phil Katz as an open format with an open specification, where his implementation PKZIP was shareware. A [restricted ZIP format](http://www.digitalpreservation.gov/formats/fdd/fdd000361.shtml) exists and is used in other filetypes such as Java .jar archives, a slew of Microsoft Office file formats, Office Document Format files (.odt, .ods, .odp), and EPUB files for e-readers. In around 1990, Info-ZIP came onto the scene. "Info-ZIP's purpose is to provide free, portable, high-quality versions of the Zip and UnZip compressor-archiver utilities that are compatible with the DOS-based PKZIP by PKWARE, Inc." (https://infozip.sourceforge.net/). They did this successfully, leading to increased adoption of the ZIP format. In the early 1990s the [gzip](/codecs/data/gzip/) format was developed, derived from the Deflate code in the Info-ZIP utilities. It was designed to replace the Unix `compress` utility, which used the (at the time) patented LZW compression algorithm which threatened its free use. Though some specific implementations of Deflate were patented by Phil Katz, the format was not, so a Deflate implementation that did not infringe on any patents was written. As a `compress` replacement, the Unix gzip utility can decompress data that was compressed using `compress`. Gzip compresses quite a bit better than Unix compress due to its use of DEFLATE, and it has very fast decompression. It also adds a CRC-32 checksum as an integrity check for the archived data. The header format permits the storage of more information than the compress format allowed, such as the original file name & the file modification time. The popular [`tar`](/codecs/data/tar/) utility, which creates an archive of files, has an option to compress directly to the `.tar.gz` format and is a very popular use caze for gzip. Since the compression of a `.tar` can take advantage of redundancy across files, ZIP often compresses less effectively than the marriage of tar & gz. `.tar.gz` is the most common archive format in use on Unix due to its very high portability, but there are better compression methods available. Some of these include [XZ](/codecs/data/xz/), [bzip2](/codecs/data/bzip2/), [brotli](/codecs/data/brotli/), [7-zip](/codecs/data/7z/), & [Zstandard](/codecs/data/zstd/). ## Encoding ### Linux & macOS Chances are, you have gzip already available on your system. You can encode gzip archives using the `gzip` command. 1. Open a terminal window. 2. Navigate to the directory where you want to create the gzip archive. 3. Use the `gzip` command followed by the name of the file you want to compress. For example: ```bash gzip -7 myfile.txt ``` This will create a compressed file called `myfile.txt.gz` in the current directory using compression level 7. Compression levels span from 1 through 9 (`-1 .. -9`; shortcuts are `--fast` for `-1`, `--best` for `-9`). 4. If you want to compress multiple files at once, you can use the `-a` option followed by the names of the files you want to compress. For example: ```bash gzip -a myfile1.txt myfile2.txt ``` This will create compressed files called `myfile1.txt.gz` & `myfile2.txt.gz` in the current directory. 5. If you want to compress a directory and all its contents, you can use the `-r` option followed by the name of the directory. For example: ```bash gzip -r mydirectory/ ``` This will create compressed versions of each file in the specified directory. 6. If you want to encode the gzip archive with a different extension, you can use the `-S` option followed by the suffix `.suf`. For example: ```bash gzip -S .suf myfile.txt ``` This will create a gzip-compressed file called `myfile.txt.suf` in the current directory. Also, you can use other options like `-v` for verbose mode, `-f` to force overwriting & compress links, `-l` for listing the files and `-d` for decompressing the files. You can find more information about the `gzip` command & its options by running `man gzip` in a terminal. ### Windows To be filled. *References: Mark Adler is an American software engineer best known for his work in the field of data compression as the author of the Adler-32 checksum function, and a co-author of the zlib compression library and gzip. He has contributed to Info-ZIP, and has participated in developing the Portable Network Graphics (PNG) image format. Much of this post is based on his writing in [this StackOverflow answer](https://stackoverflow.com/questions/20762094/how-are-zlib-gzip-and-zip-related-what-do-they-have-in-common-and-how-are-they)* --- # tar Canonical URL: https://encode.wiki/codecs/data/tar/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: a27f14787fedcaabbce67e952c272864d15755317e00d63fe03f4295614f7f13 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! `tar`, or Tape ARchive, is a archiving format and utility first developed for Version 7 Unix in 1977. It's original purpose was to collate files into one that can be stored on tape. Similarly, today it is used to bring many files together into a "tarball", which can be compressed with any general data compression algorithm. ## Usage > **Note — This guide has been written for GNU tar on linux, however it should be applicable to BSD tar, macOS tar, and the tar command in powershell on Windows..** > > ### Create a tar archive ```bash tar -cf {archive name} {files listed here} ``` You can use `tar` to compress your archive, for example into a `.tar.gz` or `.tar.xz` archive. To do this, you either can either use a flag such as `-z`, `-j`, or `-J` ([gzip](/codecs/data/gzip/), [bzip2](/codecs/data/bzip2/), [xz](/codecs/data/xz/)), or you can use `-a` ('automatic'), which allows it to intuit what algorithm you want from the file extension, such as `archive.tar.xz` for an xz compressed tarball. GNU tar can use these compression algorithms * gzip (.gz) * bzip2 (.bz) * xz (.xz) * lzip (.lz) * lzma (.lzma) * lzop (.lzo) * zstd (.zstd) ### Extract a tar archive ```bash tar -xf {tarball}.tar -C {directory to extract to} ``` tar can extract from it's supported compressed formats, such as `archive.tar.xz` automatically, with no extra flags. --- # xz Canonical URL: https://encode.wiki/codecs/data/xz/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 3fc95183c231f92788784b04c6ff9912e15549b2224258cfafd45a013ca5df8e > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! XZ is a data compression format and utility based on the Lempel-Ziv-Markov Chain Algorithm (LZMA). The XZ format itself is an improvement on LZMA, allowing for preprocessing filters similar to [7-zip](/codecs/data/7z/) to increase the resulting archive's compression ratio. XZ can only compress one file at a time, so making a [tar](/codecs/data/tar/) archive of the files you'd like to compress (if there are multiple) is necessary when using XZ. XZ is more widely supported when compared to other data compression formats, seeing support across iOS, macOS, and many Linux distributions by default. To decompress & compress XZ on Windows, you will likely need the 7-Zip archive utility. ## Usage This usage is for the `xz` utility on linux, but is applicable to other platforms where xz can be used. It should be noted that `xz`'s default behavior is to delete the original file after it has completed the relevant compression or decompression operation, but this can be stopped with the flag below. An arbitary number of files may be passed to xz and it will individually complete the specified operation on each given file. ### Compression ```bash xz {file} ``` This will result in a file named `{file}.xz` being created in the current working directory. A more advanced variant is listed here: ```bash xz -# --extreme -M 800Mib -T 2 -k {file} ``` - `-#` is a number between 0 and 9 specifying speed presets, 0 being the fastest and 9 slowest. - `--extreme` is an option allowing xz to use more time than the standard preset level. - `-M {size}` is an option restricting the memory usage of `xz` either as a percentage of system memory or an absolute amount. - `-T {threads}` is an option restricting the number of threads used by `xz`. - `-k` prevents xz from deleting the input file. ### Decompression ```bash xz -d {file}.xz ``` This decompresses the xz archive to it's original file. - `-M {size}` is an option restricting the memory usage of `xz` either as a percentage of system memory or an absolute amount. - `-T {threads}` is an option restricting the number of threads used by `xz`. - `-k` prevents xz from deleting the input file. --- # ZIP Canonical URL: https://encode.wiki/codecs/data/zip/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 2a6d2c340ad4c5ae7d63dbd042ef842451aa59233af22eb7d8c9b064f708cac5 > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. ZIP is an archive file format that supports lossless data compression. A ZIP file may contain one or more files or directories that may have been compressed using any one of a number of different algorithms present in the ZIP specification. The most common algorithm used in ZIP is DEFLATE, which is also used in [gzip](/codecs/data/gzip/) & [PNG](/codecs/images/png/). Deflate acts as a combination of LZ77 lossless coding & Huffman coding, where it can first use LZ77 to find patterns in the data & reduce redundancy. This is followed by using Huffman coding to assign smaller bit values to patterns found more frequently by LZ77. Additionally, files in a ZIP archive are compressed individually so it is possible to extract existing files or add new ones without applying compression or decompression to the entire archive. ZIP is noteworthy for its nearly universal compatibility. "Traditional ZIP" (compression method 8 in the ZIP specification) limits the size of compressed archives to 4 GB, though most ZIP compressors use Deflate64(tm) (compression level 9 in the ZIP specification) to bypass this limitation. ZIP is competitive with [gzip](/codecs/data/gzip/) and has been succeeded many times by formats & algorithms such as [bzip2](/codecs/data/bzip2/), [XZ](/codecs/data/xz/), [7-zip](/codecs/data/7z/), [brotli](/codecs/data/brotli/) (to a degree), and [Zstandard](/codecs/data/zstd/). ## Format Breakdown DEFLATE is an LZ77-based compressor that finds repeated sequences of bytes in the input data and replaces them with shorter references to previous occurrences. It also uses Huffman coding to encode the symbols with variable-length codes based on how frequently they occur. DEFLATE has two modes for each block of compressed data: These are specified as either "static" or "dynamic" Huffman compressed blocks. In static mode, the Huffman codes are fixed and predefined. In dynamic mode, the Huffman codes are generated dynamically & transmitted along with the compressed data. ZIP files have a specific structure that consists of four main file header components: local file headers, central directory file headers, end of central directory record, and data descriptors. The local file headers store information about each compressed file, such as its name, size, CRC-32 checksum, compression method, and optional extra fields. The central directory file headers store similar information as the local file headers, but also include the offset of each local file header in the ZIP file. The end of central directory record marks the end of the ZIP file and contains information about the number and size of the central directory file headers. The data descriptors are optional and store additional information about the compressed data, such as its CRC-32 checksum, uncompressed size, & compressed size. ZIP files can also support other compression methods, such as Deflate64(tm), BZIP2, LZMA, & [Zstandard](/codecs/data/zstd/). These methods are not widely supported by most ZIP utilities and may cause compatibility issues. ZIP files can also contain uncompressed data. The format also supports encryption to protect the data from unauthorized access. There are two types of encryption supported by ZIP: traditional ZipCrypto encryption and strong encryption. ZipCrypto encryption is considered insecure, while stronger encryption in ZIP uses more resilient algorithms albiet spread across a number of standards. Because of this, strong encryption is not standardized and may cause compatibility issues. ZIP files can reduce the size of files and folders by more efficiently representing redundant data. They can also combine multiple files and folders into a single archive that can be easily transferred or stored. You will not find a more popular implementation than ZIP for general data compression purposes like these. ZIP files can also preserve the metadata of the original files, such as their names, paths, dates, and attributes. However, ZIP files also have some limitations and concerns. For example, traditional ZIP files (that aren't ZIP64) have a maximum size of 4 gigabytes for each compressed file and 65,535 entries for each archive. Most ZIP implementations do not support symbolic links or hard links within the archive. Additionally, ZIP can be encoded in a number of different ways. Apple has a default "Compress" option in Finder that compresses selected files into a ZIP file, and many Linux desktops offer GUI functionality for creating ZIP files easily as well. It is common to compress to ZIP on Windows using the 7-zip data compression & decompression utility (not to be confused with the [7-zip compression format](/codecs/data/7z/), though the two are related). ## History The ZIP format was developed by Phil Katz as an open format with an open specification, where his implementation, PKZIP, was shareware. A [restricted ZIP format](http://www.digitalpreservation.gov/formats/fdd/fdd000361.shtml) exists and is used in other filetypes such as Java .jar archives, a slew of Microsoft Office file formats, Office Document Format files (.odt, .ods, .odp), and EPUB files for e-readers. In around 1990, Info-ZIP came onto the scene. "Info-ZIP's purpose is to provide free, portable, high-quality versions of the Zip and UnZip compressor-archiver utilities that are compatible with the DOS-based PKZIP by PKWARE, Inc." (https://infozip.sourceforge.net/). They did this successfully, leading to increased adoption of the ZIP format. In the early 1990s the [gzip](/codecs/data/gzip/) format was developed, derived from the Deflate code in the Info-ZIP utilities. It was designed to replace the Unix `compress` utility, which used the (at the time) patented LZW compression algorithm which threatened its free use. Though some specific implementations of Deflate were patented by Phil Katz, the format was not, so a Deflate implementation that did not infringe on any patents was written. Unlike `.tar`, `.zip` has a central directory at the end, which provides a list of the contents. That and the separate compression provides random access to the individual entries in a `.zip` file. A `.tar` file would have to be decompressed and scanned from start to end in order to build a directory. The popular `tar` utility, which creates an archive of files, has an option to compress directly to the `.tar.gz` format and is a very popular use caze for gzip. Since the compression of a `.tar` can take advantage of redundancy across files, ZIP often compresses less effectively than the marriage of tar & gz. `.tar.gz` is the most common archive format in use on Unix due to its very high portability, but there are better compression methods available. Some of these include [XZ](/codecs/data/xz/), [bzip2](/codecs/data/bzip2/), [brotli](/codecs/data/brotli/), [7-zip](/codecs/data/7z/), & [Zstandard](/codecs/data/zstd/). In this case, the benefit of ZIP is that because it compresses files separately and builds a central directory at the end of the archive which provides a list of the contents, ZIP provides random access to the individual entries in a `.zip` file. A `.tar` file would have to be decompressed and scanned from start to end in order to build a directory. ## Encoding #### Linux & macOS To encode to a ZIP file most efficiently on Linux or macOS, it is worth using the 7-zip implementation of DEFLATE for ZIP compression. The 7zip website's homepage claims: "For ZIP and GZIP formats, 7-Zip provides a compression ratio that is 2-10 % better than the ratio provided by PKZip and WinZip." You can use the highly flexible 7-zip CLI utility through binaries available on [7-zip's Download page](https://www.7-zip.org/download.html). Here are some direct download links: [Linux x86_64](https://www.7-zip.org/a/7z2301-linux-x64.tar.xz) | [macOS Universal](https://www.7-zip.org/a/7z2301-mac.tar.xz) Once you've downloaded the utility, remember whether you are choosing to use the `7zz` binary or the static `7zzs` binary. Commands run using `7zz` should run using `7zzs` as well, so please replace `7zz` in our examples as appropriate if you choose not to use it. Additionally, please copy your choice of binary to your `/usr/local/bin` if you want to be able to use it everywhere. - To encode a ZIP file at the lowest effort setting using one thread: ```bash 7zz a -bso0 -tzip -mmt1 -mx1 "Output.zip" "Input" ``` - To encode a ZIP file at the highest effort setting using eight threads: ```bash 7zz a -bso0 -tzip -mmt8 -mx9 "Output.zip" "Input" ``` #### Windows To be filled. ## Conclusion The only real benefit of using ZIP over more modern formats currently is compatibility. It may be viable when compared to 7z & XZ due to a reduction in complexity that improves encode & decode speed, but Zstandard is incredibly performant in both of these areas and generally outperforms ZIP. When it comes to content delivery on the Web, Brotli has been adopted across all modern web browsers and offers a better alternative to older compression technologies used on the Web that resemble ZIP. --- # zpaq Canonical URL: https://encode.wiki/codecs/data/zpaq/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 1fddb9c762ceb6fd17ce34f85d625a91db77afd2a0cd012c9a147a8b10a33a05 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! ZPAQ is a lossless data compression algorithm that combines several techniques to achieve high compression ratios. It was developed by Matt Mahoney. ZPAQ uses a multitude of different compression algorithms to try to achieve the best size-to-compression-time ratio possible while producing the smallest possible archives without much concern given to decompression performance. On the official ZPAQ website, it looks like it is designed for "realistic backups that have a lot of duplicate files and a lot of already compressed files." ZPAQ is also considered an "incremental journaling archiver" meaning you can add files to an existing archive based on if they were changed or not. This reduces the time needed to wait for a new backup to finish, if that is your use case. Since ZPAQ is so focused on compression ratio, this kind of feature may reduce the burden imposed by long compression times in practical use cases where it makes sense. Windows & macOS do not handle ZPAQ archives properly by default, and it is unlikely many Linux distros do either. --- # zstd Canonical URL: https://encode.wiki/codecs/data/zstd/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: aef79b471162a91813e9ece78ff794adb8fa842ea49393fc2b06e4d2971f7b82 # Zstandard > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Zstandard is a compression algorithm developed by Facebook known for its extremely fast decompression speeds. It was released in early 2015 and is used in a variety of different contexts. It was designed to perform similarly to older Deflate-based compression algorithms like [ZIP](/codecs/data/zip/) or [gzip](/codecs/data/gzip/) while being faster overall. In practice, it is said to compress similarly to pure LZMA (part of [XZ](/codecs/data/xz/) & [7-zip](/codecs/data/7z/)) while being much faster. While `.tar.zstd` archives aren't as popular as `.tar.xz` or `.tar.gz`, Zstandard is already a very popular tool for compression in the world of open-source software. It has been integrated into both the FreeBSD kernel & the Linux kernel and is available as a filesystem compression method for the btrfs, squashfs, bcachefs, & OpenZFS filesystems. Filesystem compression refers to a compression scheme that transparently compresses files stored on a filesystem at all times, leading to an overall reduction in storage used across the filesystem. The command line `zstd` utility can compress to Zstandard at compression levels 1 through 19 by default. The upper bound is raised to 22 when passing the `--ultra` flag. All Arch Linux packages are compressed at zstd level 20, allowing Arch packages to be decompressed 14 times faster compared to XZ at the cost of an average 0.8% filesize increase across all packages. It is popular in the game emulation scene as well, as many game file formats for emulating console games support zstd compression. The ZIP file format standard actually supports Zstandard in compression level 93 since version 6.3.8, published in 2020. Content encoding using zstd is supported in chromium since Chromium 118 behind an experimental flag, meaning it might compete with [Brotli](/codecs/data/brotli/) on the web in the future. Apple's LZFSE algorithm is purportedly similar to Zstandard compression level 6. Zstandard has the potential to effectively compete with nearly every modern compression method available across most modern use cases. In certain scenarios, if it takes off as a content delivery format, it could replace Brotli if the benefits of super-fast & super-light decode improve the responsiveness of web pages & are worth sacrificing a bit of compression ratio. When using the much higher effort settings, it often outcompetes Brotli for the archive size as well. In the future, `.tar.zst` could replace 7-zip, ZIP, or other archiving formats, making speedy decode a reality on systems featuring varying levels of compute horsepower. ## Usage > **Note — This guide has been written for the `zstd` command-line utility, however GUI archivers such as peazip and 7zip have growing support for zstd..** > > ## Compress a file Like many other compressing utilities, in order to compress mutliple files, one should probably archive them with [tar](/codecs/data/tar/). ```bash zstd -# {file} -o {file}.zstd ``` `-#` is actually a number that represents the desired compression level, for example `-3`, `-6`. By default you can specify 1-19. By also passing `-ultra`, you can go up to compression level 22. ## Decompress a file ```bash zstd -d {file}.zstd -o file ``` --- # AVIF Canonical URL: https://encode.wiki/codecs/images/avif/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 117145e6b39f6ed4ff51dfa654e93c14bacedd13be09884ebcc776e239754302 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. AVIF, which stands for AV1 Image File Format, is a newer image codec that is based on the [AV1](/codecs/video/av1/) video codec. AVIF supersedes [HEIC](/codecs/images/heic/), & uses the same HEIF container as HEIC. AVIF is designed to have a better featureset & better general lossy compression than older image codecs, including [WebP](/codecs/images/webp/), HEIC, & [JPEG](/codecs/images/jpeg/). AVIF is often compared to [JPEG-XL](/codecs/images/jxl/), though in practice, the two have very different strengths. There are two AVIF profiles available for encoding: Baseline & Advanced, which are based on AV1's Main & High profiles respectively. The AVIF Baseline profile supports up to 8,192\*4,352 resolution specified by the requirement of using AV1 Level 5.1 or lower. Using tiling, it is possible to increase the maximum resolution of the AVIF Baseline profile to 65536\*65536, although this hurts coding efficiency as visual anomalies may be encountered along the edges of the tile boundaries. AVIF is also limited to 10 bit color precision in its Baseline profile. In the AVIF Advanced profile, the maximum image dimensions extend to 16,384*8,704. Tiling may be used in the Advanced profile to create larger images, but the same limitations regarding visual artifacts apply. The AVIF Advanced profile extends the allowed AV1 Level to 6.0 or lower, & the highest bit depth offered by this profile is 12 BPC. It is worth noting that while it is currently a near certainty that AVIF implementations will support both the Baseline & Advanced profiles, this may not always be the case. This is a problem that affects HEIC currently, & is a known potential weakness of video-based image codecs. ## Performance Checklist Lossless? *Poorly* Lossy? *Yes* Supported Bit Depths: *8 BPC, 10 BPC, 12 BPC* HDR/Wide Gamut? *Yes* Animation? *Yes* Transparency? *Yes* Progressive Decode? *No* Royalty Free? *Yes* **Compatible Browsers** (full support) - [Google Chrome](https://www.google.com/chrome/) 85+ - [Safari](https://www.apple.com/safari/) 16.4+ - [Firefox](https://www.mozilla.org/en-US/firefox/new/) 113+ - [Opera](https://www.opera.com/) 71+ - [GNOME Web](https://apps.gnome.org/app/org.gnome.Epiphany/) - [Thorium](https://thorium.rocks/) - [Mercury](https://thorium.rocks/mercury) ## Format Breakdown ### Advantages AVIF is known for its extremely strong lossy compression performance for non-photographic images as well as photographic images. AVIF is consistently better than JPEG visually. Using [libaom](/tools/software-encoders/aomenc/)'s `--tune iq` or [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/)'s Tune 4, AVIF is generally the most efficient image codec on the Web for quality per bit. AVIF's quality per bit is generally better than [JPEG XL](/codecs/images/jxl/), though JPEG XL is considerably more feature-rich. AVIF compatibility has grown rapidly since its adoption in Google Chrome in 2020. For a relatively new image format, its level of penetration has been stellar, especially in the browser market; AVIF is considered [Baseline](https://web-platform-dx.github.io/web-features/) as of 2024. AVIF's wider featureset enables new experiences through images, including HDR. AVIF also presents astonishing animation prowess, as it is capable of using AV1's video coding techniques which make it easily the best animated image format for most use cases. ### Limitations AVIF encoding implementations are difficult to use, and images require much longer encoding times for what can be considered competitive quality. Making encoding more difficult, AVIF's use of intra-frame coding techniques that share data between blocks reduces parallelization capability & worsens generation loss. Theoretically, this improves coding efficiency, though. Via the AVIF Encoding section of the aomenc page:
AVIF Encoding with aomenc through avifenc

Using aomenc through avifenc is widely considered to be the best way to encode AVIF images, as SVT-AV1 only supports 4:2:0 chroma subsampling, rav1e isn't fast enough for still images, & the libaom team have put more effort into intra coding than the teams responsible for producing the other prominent open source AV1 encoders.

A sample command for encoding AVIF looks like this:

avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=16 -a tune=ssim \[input\] output.avif

Where:

AVIF does not have progressive decode. This is a common weakness of video-based image codecs. While there is a hacky way to do progressive AVIF by encoding a low fidelity frame & then a high fidelity frame in an animated AVIF at a high framerate so the low fidelity frame is loaded & plays first, this is far from ideal for the average user & adds to an already burdensome encoding process. Additionally, this has issues with Firefox. Finally, AVIF's lossless mode is underwhelming, often producing larger files than PNG. When compressing losslessly, it can generally be advised that you avoid AVIF. ### Encoders #### AV1 Encoders In practice, any usable AV1 encoder should be able to produce AVIF images. This makes our list of feature-complete, open-source AVIF encoders end up looking quite familiar: - [libaom](/tools/software-encoders/aomenc/) - [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/) - [SVT-AV1](/tools/software-encoders/svt-av1/) - [rav1e](/tools/software-encoders/rav1e/) In the proprietary space, [Aurora1](/tools/software-encoders/aurora1/) was used at Cloudinary (a major multimedia CDN) for AVIF compression for a while, but has since been replaced by libaom. #### tinyavif Rachel Barker's tinyavif (whose original public repository is no longer available) is a barebones AVIF encoder that is designed to be easy to understand. It is not feature-complete, but it is a good choice for those who want to learn more about how AVIF works (and by extension, how AV1 works). It is written in Rust, and is described as "the world's most minimal AV1 encoder." ## Conclusion AVIF is a strong image format that is worth considering for use on the web. It is a great choice for images that are not high fidelity, as it will compress better than JPEG in most cases. AVIF is also a great choice for animated images, as it is the best format for this use case. However, AVIF is not a great choice for lossless images, as it tends to produce larger files than PNG. Overall, AVIF is a great format to use for images that are not high fidelity, as it will compress better than JPEG in most cases. --- # GIF Canonical URL: https://encode.wiki/codecs/images/gif/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 3cd4fd84ba71e085004647dc0260c589a6a6cc247677c98be2038f94d2a86689 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Graphics Interchange Format (GIF) is an image file format first released by CompuServe in 1987. It remains popular due to it's widespread support for animated images despite its obsolete efficency. Other animated image formats like Animated [AVIF](/codecs/images/avif/) & Animated [WebP](/codecs/images/webp/) have since surpassed GIF in functionality, as has the animated [PNG](/codecs/images/png/) variant APNG. ## Performance Checklist Lossless? *Yes* Lossy? *No* Supported Bit Depth: *256 colors* HDR/Wide Gamut? *No* Animation? *Yes* Transparency? *Yes* Progressive Decode? *No* Royalty Free? *Yes* --- # HEIC Canonical URL: https://encode.wiki/codecs/images/heic/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: bfa56692c52a1769c9607a82455a2aba0f42e69d1b51924c7fc434b85aaf279b > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! The HEIC image format, also known as the High Efficiency Image Format, is a newer image codec that was developed to provide improved compression and better performance compared to traditional image formats like [JPEG](/codecs/images/jpeg/). HEIC files use [HEVC](/codecs/video/hevc/) internally, meaning the format is not royalty free. While this has limited its adoption across the Web, this format is supported by many modern devices including the entire Apple ecosystem. iPhones shoot HDR HEIC photos by default by utilizing the iPhone's HEVC hardware video encoder to capture these images. Some Android phones are capable of shooting HEIC as well, but these are often transcoded from JPEG. HEIC has largely been surpassed by [AVIF](/codecs/images/avif/), which uses the same container to store [AV1](/codecs/video/av1/)-compressed images. ## Performance Checklist Lossless? *No* Lossy? *Yes* Supported Bit Depths: *8 BPC, 10 BPC* > *Higher bit depths not widely supported* HDR/Wide Gamut? *Yes* Animation? *Yes* Transparency? *Yes* Progressive Decode? *No* Royalty Free? *No* --- # JPEG Canonical URL: https://encode.wiki/codecs/images/jpeg/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 304f0c7f59b1640cb994cd246c73089de9db35663190e815e8eaa1fbfab8965f JPEG (Joint Photographic Experts Group) compression is a widely used method for reducing the size of digital images while preserving visual quality. It's based on the principles of lossy compression, which means that some image data is discarded to achieve a smaller filesize. ## Performance Checklist Lossless? *No* Lossy? *Yes* Supported Bit Depth: *8 BPC* HDR/Wide Gamut? *Kinda* Animation? *No* Transparency? *No* Progressive Decode? *Yes* Royalty Free? *Yes* ## Compression Learning how JPEG compresses images is immensely helpful for understanding how other compression methods work in other codecs. It is definitely worth reading to get a useful background in understanding concepts like entropy coding, the DCT, and color spaces other than RGB. Here's a step-by-step explanation of how JPEG compression works: #### Color Space Conversion Most digital images are originally in the RGB (Red, Green, Blue) color space. The first step in JPEG compression is to convert the image to the YCbCr color space. Y represents the luminance (brightness), while Cb and Cr represent the chrominance (color information). The Cb & Cr components are subsampled to a quarter of the resolution of the original image, meaning the resulting color space is chroma subsampled with *4:2:0* subsampling. #### Image Tiling The image is divided into smaller blocks or tiles, typically 8x8 pixels each. Each of these blocks will be processed separately. #### Discrete Cosine Transform (DCT) For each 8x8 block, a mathematical transformation called the [Discrete Cosine Transform](/start-here/terminology/#discrete-cosine-transform-dct) is applied. This transformation converts the pixel values into a set of frequency components, taking spatial data and transforming it to the frequency domain. The DCT is applied to each color channel in the YCbCr color space. This algorithm is a particularly good choice for image (and music/speech) compression because it has high energy compaction relative to our understanding of images & their perceptual quality. High energy compaction means the DCT is able to represent a signal with a small number of significant coefficients, in this case mainly in the lower frequencies. #### Quantization After the DCT, the frequencies are quantized in a table representing frequency coefficients & their corresponding frequencies. Less perceptually important details can be omitted to reduce filesize by discarding coefficients in the table that correspond to less visually salient frequencies. This is "lossy" compression, and is the key step in achieving a high compression ratio while still maintaining an image that looks reasonable. The quantization table used in this step can vary in the number of frequencies it attempts to retain, affecting the trade-off between compression & image quality. #### Zigzag Scanning The quantized coefficients are then reordered using a zigzag pattern. This is done to prepare the data for the next step. #### Run-Length Encoding The zigzag-ordered coefficients are run-length encoded. This means that sequences of zeroes are compressed into a shorter representation. For example, if there are many consecutive zeroes in the data, they can be represented as (0, 10) instead of listing ten individual zeroes. #### Entropy Encoding The run-length encoded data is further compressed using entropy encoding. JPEG uses Huffman coding, which assigns shorter codes to more frequently occurring values in the table of DCT coefficients, reducing the overall file size. #### Saving the File The compressed luminance and chrominance data, along with information about color space conversion, quantization tables, and EXIF data, are saved in the JPEG file format. #### Decoding When you open a JPEG image, the reverse process occurs. The file is decoded, and the DCT coefficients are dequantized, the inverse DCT is applied, and the image is converted back to the RGB color space to be displayed on a screen. It's important to note that JPEG compression is ***lossy***, meaning that some image quality is discarded in the pursuit of smaller file sizes. This makes it different than codecs designed for lossless compression like [PNG](/codecs/images/png/), [WebP](/codecs/images/webp/)'s lossless mode, and [JPEG-XL](/codecs/images/jxl/)'s lossless mode. The degree of compression and the quality of the compressed image can be adjusted through settings when saving a JPEG, allowing for a trade-off between file size & image fidelity. While JPEG is certainly not the most state of the art lossy image codec compared to its newer and (usually) better successors like [JPEG-XL](/codecs/images/jxl/) (an actual direct successor) & [AVIF](/codecs/images/avif/), it enjoys near universal compatibility with (likely) most utilities you would work with in your everyday life that have anything to do with images. --- # JPEG 2000 Canonical URL: https://encode.wiki/codecs/images/jpeg2000/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 7a3a2453eaeea4e7b6a173f025d4fd489c1fbf1b590a949bc66edf0d0016d2da > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! JPEG-2000 is an older image compression format that uses wavelet technology to achieve high compression ratios while maintaining image quality. It supports both lossy and lossless compression, and is commonly used in applications such as digital photography, medical imaging, and video surveillance. JPEG-2000 files can be transparently compressed and decompressed using a variety of software tools and libraries, making it a flexible and widely-supported format for image storage & transmission. JPEG-2000 never effectively took off on the Web, but digital cinema distribution is often done with JPEG-2000. A "DCP" is a "Digital Cinema Package," which is a format used to distribute and play back digital movies in theaters. These DCPs are often compressed losslessly with JPEG-2000. --- # JPEG XL Canonical URL: https://encode.wiki/codecs/images/jxl/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: d1394f7f8b15ee47b68e97d8cc217034b3c1aecaf1175501df979ccbe51d43c2 # JPEG XL JPEG XL (JXL) is a compression format for images that was developed by the Joint Photographic Experts Group (JPEG) in 2020. It is designed to provide improved compression efficiency compared to the traditional [JPEG](/codecs/images/jpeg/) format, while still maintaining image quality. JPEG XL uses a combination of techniques such as perceptual color encoding & advanced entropy coding to achieve its improved compression performance. It also has a lossless JPEG recompression mode, where an existing JPEG file can be turned into a JXL that can be decoded for a bit-for-bit exact replica of the original JPEG. ## Performance Checklist Lossless? *Yes* Lossy? *Yes* Supported Bit Depths: *Up to 32 BPC* HDR/Wide Gamut? *Yes* Animation? *Yes* Transparency? *Yes* Progressive Decode? *Yes* Royalty Free? *Yes* ## Format Breakdown JPEG XL has a number of standout features that make it an appealing image codec to work with for many use cases. From the [JPEG XL Info page](https://jpegxl.info), JXL has the following features: - **Best lossless image compression**: It offers about 35% smaller file sizes than PNG (50% smaller for HDR). - **High-fidelity lossy image compression**: JPEG XL provides about 60% smaller file sizes than JPEG for the same visual quality. - **Progressive decoding**: This allows an image to be displayed in lower quality before the entire file has been downloaded, improving user experience on slow connections. - **Lossless JPEG transcoding**: JPEG images can be converted to JPEG XL without any mathematical loss, and the resulting file is about 20% smaller. - **Designed for both photographic and synthetic images**: JPEG XL works well with a wide range of image types, including photos, graphics, and illustrations. - **Fast software encoding and decoding**: The codec is designed to be efficient and fast, enabling quick image loading and saving. - **Full support for wide gamut and HDR**: JPEG XL supports a wide range of colors and high dynamic range, making it suitable for modern displays. - **Perceptually optimizing reference encoder**: The encoder is designed to optimize image quality based on how humans perceive images. ### Lossless Compression JPEG XL offers excellent lossless compression capabilities. While lossless WebP was an improvement over PNG for 8-bit lossless image encoding, JPEG XL manages not only to outdo lossless WebP in encoding efficiency but also be more versatile for bit depths greater than 8-bit (a category PNG previously dominated). 16-bit lossless imagery, especially HDR images that are becoming more popular & rarely utilize 8-bit color depth, are where JPEG XL shines, and it is the only codec to compete with PNG in that regard while providing better coding efficiency. Example: JPEG XL compresses [this 16-bit AdobeRGB PNG](https://imgsaver.com/images/2023/10/03/16bit.png) better than PNG. Using: `cjxl 16bit.png 16bit.jxl -d 0.0 -e 9 -I 100 -g 3 -E 11` 16-bit PNG: `1533373` bytes. 16-bit JXL: `1211029` bytes. ### Lossy Compression JPEG XL is also adept at lossy compression, especially at quality levels that we as humans care about. It promises to be around 60% better than JPEG. While video-based codecs like AVIF are often better in terms of quality per bit, JPEG XL is both fast and efficient for medium and high fidelity photographic image compression. ### Supported Bit Depth(s) JPEG XL supports up to 32 bits per channel of bit depth, making it future proof for the increasingly popular HDR photos coming out of smartphones. There is essentially zero downside to encoding high bit depth with JXL relative to the resulting encode's size. Considering many smartphones take HDR photos now, JXL offers a compelling pipeline for these photos to make their way to the Web in the future especially as companies like Adobe & Apple have already embraced the new codec. ### Progressive Decode JPEG XL provides actual progressive decode support that you can experiment with here on a supported browser like Safari, Waterfox, Thorium, Mercury, or any browser on iOS. Progressive decode is a feature only JPEG is able to offer a real implementation of, rendering low frequency transform coefficients before the rest of the image arrives to allow an image to display before the entire thing has been sent over the network. Blurhashes do not replace this technology, but rather compliment it, allowing another layer of progressive decode that can be used even before the image begins to load progressively. This is an important feature to improve the user experience on websites featuring large images, or on any website if your Internet connection isn't strong. ### Lossless JPEG Re-Compression An incredibly unique JPEG XL feature is lossless JPEG re-compression, or the ability to take a JPEG input and provide an output with a smaller filesize (on average, 20% smaller) that is pixel-for-pixel identical. This is why companies like Meta have endorsed JPEG XL, as it offers a path forward for the existing JPEGs on the Internet. ### Industry Support From the JPEG XL Wikipedia page: > Besides Cloudinary and Google originally, throughout JPEG XL's preliminary implementation in web browsers, various representatives of well-known industry brand names have publicly voiced support for JPEG XL as their preferred choice, including Facebook, Adobe, Intel and the Video Electronics Standards Association, The Guardian, Flickr and SmugMug, Shopify, the Krita Foundation, and Serif Ltd. Apple also features ecosystem-wide JPEG XL support as of iOS 17 & macOS Sonoma. ### Other Features JPEG XL has the potential to replace popular formats like TIFF for authoring workflows due to its broad feature set. From the JXL Wikipedia, some additional features include: - Image dimensions of over a billion (2^30-1) pixels on each side. - Up to 4099 channels, including support for alpha transparency - There can be multiple frames with zero duration, allowing support for layers in graphics software - Animation support, allowing JXL to rival GIF - Images can be stored in tiles to reduce the time needed to decode them. - Graceful quality degradation across a large range of bitrates means quality loss isn't as abrupt as with older formats. - Perceptually optimized reference encoder which uses a perceptual color space, adaptive quantization, and conservative default settings. - Support for wide color gamut and HDR - Efficient encoding and decoding without requiring specialized hardware: JPEG XL is about as fast to encode and decode as old JPEG using libjpeg-turbo and an order of magnitude faster to encode and decode compared to HEIC with x265. It is also parallelizable. - Royalty-free format with an open-source reference implementation available on GitHub. ## Encoders JPEG XL has a couple of noteworthy encoders currently available to work with. Because JPEG XL is so new, most encoders aren't yet intelligent enough to take advantage of the whole format yet. Here's a quote from Jon Sneyers in the JPEG XL discord that sums it up nicely: > Encode side: 80% or so of the coding tools are used in one way or another by the encoder (the 20% is splines and super large VarDCT blocks, and also the things that are not used by default without using special experimental options, such as delta palette and noise). But the coding tools that are used, are typically used in a specific, limited way that doesn't come anywhere close to exhausting the bitstream expressivity. Sneyers is talking about libjxl's `cjxl` encoder, which will be discussed further below. ### libjxl The reference [libjxl](https://github.com/libjxl/libjxl) implementation has the capability to both decode and encode JPEG XL image files. Both are discussed below. #### Encoding libjxl's encoder `cjxl` has more options to play around with. It takes a few primary arguments, distance (`-d`), quality (`-q`), and effort (`-e`). **Distance and quality** Distance and quality are two ways of specifying *how much loss* you are willing to tolerate, and as such, they are mutually exclusive, as they pull the same levers under the hood. * Distance is designed to map to how 'close' one must be to the source to notice any loss. It is represented as a scale between 0.0 & 25.0. 0.0 is **mathematically lossless**, every pixel will have the exact same value as the source. 1.0 is designed to be **visually lossless**, look the same at a normal viewing distance, and higher values have more loss. * Quality is designed to roughly map to [JPEG](/codecs/images/jpeg/)'s quality argument. A range 0-100, where 100 is **mathematically lossless**, 90 is intended to be **visually lossless**, and 0 is almost unrecognizable as the original image. **Effort** Effort is similar to `cpu-used` in video encoding. It specifies the amount of effort the encoder will make in order to get the smallest file size it can. It takes the form of a range 1-9, where higher numbers will spend more resources to get diminishing returns in terms of smaller size, while lower values do the opposite, leaving file size on the table for faster encoding. ```bash title="Encoding with effort 9 and distance 1.0" cjxl -e 9 -d 1.0 example.png example.jxl ``` ```bash title="This, by default uses lossless JPEG compression." cjxl example.jpg example.jxl ``` #### Decoding Decoding a `.jxl` image is straightforward with libjxl's decoder, `djxl`: ```bash djxl example.jxl example.png ``` `djxl` can decode to pixels via pipes, png, apng for animated jxl, jpg, ppm, and pfm. By default, if the `.jxl` file was encoded with lossless jpeg recompression, `djxl` will rebuild the exact jpeg file that was originally compressed. To avoid this, and create a new jpeg file: ```bash djxl -j example.jxl example.jpg ``` **Keep in mind this is now a lossy process as `djxl` will decode to pixels, then encode a new `.jpg` with those pixels.** #### Building A full build guide is provided in the [libjxl build instructions](https://github.com/libjxl/libjxl/blob/main/BUILDING.md) in the GitHub repo. This guide is simplified, and is only focused on building a working efficient encoder & decoder. These instructions should work for macOS and Linux, although macOS support isn't guaranteed. ```bash title="1. Clone the repo" git clone https://github.com/libjxl/libjxl.git --recursive --shallow-submodules ``` ```bash title="2. Install dependencies. May have to run these commands with root" apt install cmake pkg-config libbrotli-dev clang # Debian Linux pacman -Syu cmake pkgconf brotli clang # Arch Linux brew install cmake pkg-config brotli # macOS ``` ```bash title="3. Set CC & CXX variables before building (Recommended)" export CC=clang CXX=clang++ ``` ```bash title="4. cjxl & djxl will be available in the build/tools directory." cd libjxl && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-O3 -march=native" -DCMAKE_C_FLAGS="-O3 -march=native" -DBUILD_TESTING=OFF -DJPEGXL_WARNINGS_AS_ERRORS=OFF -DJPEGXL_ENABLE_SJPEG=OFF .. cmake --build . -- -j$(nproc) ``` This will build `cjxl` and `djxl` with O3 optimization for your CPU architecture on Linux or macOS. Again, be aware that macOS support is not a priority. Via the libjxl OS X build guide: > OSX builds have "best effort" support, i.e. build might not work at all, some tests may fail and some sub-projects are excluded from build. ### libjxl-tiny [libjxl-tiny](https://github.com/libjxl/libjxl-tiny) contains a simpler encoder implementation of JPEG XL, aimed at photographic images without an alpha channel. The goal is to guide hardware implementations of the encoder where support for the full set of encoding tools is not feasible. The color management is outside the scope of this library, the encoder input is given as a portable float map (PFM) in the linear sRGB colorspace, where individual sample values can be outside the \[0.0, 1.0\] range for out-of-gammut colors. For more details, see the [overview of the coding tools](https://github.com/libjxl/libjxl-tiny/blob/main/doc/coding_tools.md). The last commit was ten months ago, so it is uncertain whether libjxl-tiny could be considered active. ### Hydrium [Hydrium](https://github.com/Traneptora/hydrium) is a fast, ultra-low-memory, streaming JPEG XL encoder written in portable C. It is maintained by Traneptora. ### zune-jpegxl [zune-jpegxl](https://github.com/etemesi254/zune-image/tree/dev/crates/zune-jpegxl) is a simple, fast and fully safe modular JXL encoder written in Rust. It is maintained by etemesi254. zune-jpegxl has the following features: - Lossless encoding - 8 bit and 16 bit support - Grayscale and RGBA encoding - Threading capabilities ## Decoders ### jxl-oxide [jxl-oxide](https://github.com/tirr-c/jxl-oxide) is a spec-conforming JPEG XL decoder written in pure Rust. It is maintained by Wonwoo Choi. {} Sources: - [JXL Wikipedia](https://en.wikipedia.org/wiki/JPEG_XL) - [JPEGXL.info: Why JXL](https://jpegxl.info/why-jxl.html) - [Apple JXL Announcement](https://webkit.org/blog/14205/news-from-wwdc23-webkit-features-in-safari-17-beta/#images) - [JPEG XL: How It Started, How It's Going](https://cloudinary.com/blog/jpeg-xl-how-it-started-how-its-going) - [The Case for JPEG XL](https://cloudinary.com/blog/the-case-for-jpeg-xl) - [Time for Next-Gen Codecs to Dethrone JPEG](https://cloudinary.com/blog/time_for_next_gen_codecs_to_dethrone_jpeg) - [Image Codec Comparison](https://giannirosato.com/blog/post/image-comparison/) --- # PNG Canonical URL: https://encode.wiki/codecs/images/png/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: b3138a72184698d7c2161dc17635b0c5090f5e16b427d6d0c97064f26ffcab4b > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Portable Network Graphics (PNG) is a free lossless image file format released in 1996. It was ceated as an alternative to [GIF](/codecs/images/gif/), which was at the time a proprietary format. It gained animation support similar to GIF with the release of APNG in 2008, which is now supported by all popular web browsers. ## Performance Checklist Lossless? *Yes* Lossy? *No* Supported Bit Depths: *8 BPC, 16 BPC* HDR/Wide Gamut? *Yes* Animation? *Yes* Transparency? *Yes* Progressive Decode? *Kinda* Royalty Free? *Yes* --- # QOI Canonical URL: https://encode.wiki/codecs/images/qoi/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 1e46b7dc6fc58fb94496e5fbf8d0222fb8d115f19c60747dbde51b798ddf0768 QOI (Quite OK Image Format) is an image compression format that aims to provide a simple, fast, and efficient way to compress and decompress images losslessly. It was designed to be easy to implement while offering better compression ratios than the widely used but more complex [PNG](/codecs/images/png/) format while achieving much faster encoding & decoding speeds. ## Performance Checklist Lossless? *Yes* Lossy? *No* Supported Bit Depths: *8 BPC* HDR/Wide Gamut? *No* Animation? *No* Transparency? *Yes* Progressive Decode? *No* Royalty Free? *Yes* ## Format Breakdown QOI compression is based on a simple and fast algorithm that exploits spatial redundancy in images. The algorithm uses a combination of run-length encoding (RLE), a small lookup table, delta encoding, and full-color pixel storage to achieve efficient compression. Depending on the algorithm's decision, a chunk (pixel) can take up one to five bytes. The QOI format supports images with 3 or 4 channels (RGB or RGBA) and 8 bits per channel. The format supports two colorspaces: Linear RGB & sRGB with linear alpha. These do not affect the way pixels are encoded. Here is a breakdown of the various chunk types in QOI: 1. **`QOI_OP_RGB`**: Full RGB pixel value using 8 bits (1 byte) for each of the red, green, and blue channels. The alpha channel is 255 in RGB images, and always remains unchanged. 2. **`QOI_OP_RGBA`**: Full RGBA pixel value using 8 bits for each of the red, green, blue, & alpha channels. 3. **`QOI_OP_DIFF`**: The difference between the current pixel and the previous pixel for the red, green, and blue channels are stored using 2 bits for each channel. The differences are stored with a bias of 2 and wrap (so 1 minus 2 would be 255). The alpha channel remains unchanged. 4. **`QOI_OP_LUMA`**: These pixels encode the green channel difference from the previous pixel using 6 bits, and then encode the red and blue channel differences relative to the green channel difference using 4 bits each. This allows for more efficient encoding of small color changes. The alpha channel remains unchanged. 5. **`QOI_OP_RUN`**: These are the simplest, encoding a run-length of pixels that are identical to the previous pixel. The run length is stored using 6 bits with a bias of -1, allowing for runs of 1 to 62 pixels. 6. **`QOI_OP_INDEX`**: These are stored by referencing a previously seen pixel value from a rolling array of 64 recent pixel values by using a simple hash on each pixel as it is identified. If another pixel matches a previously seen hash value in the array, the index of the referenced pixel is stored. The QOI format also includes a simple 14-byte header that stores the image dimensions, color space, and channel depth information. The end of file is signaled by an 8-byte end marker. ### Benchmarks The creator of QOI benchmarked the format against libpng & `stbi_image_write` using the C implementation in QOI via [`qoibench.c`](https://github.com/phoboslab/qoi/blob/master/qoibench.c) on a collection of 2,879 screenshots, icons, photos, & textures ([source](https://qoiformat.org/benchmark/qoi_benchmark_suite.tar)). The results are as follows: | **Library** | **Decode (ms)** | **Encode (ms)** | **Decode MP/s** | **Encode MP/s** | **Size (kb)** | **Compression Rate** | |---------|-----------|-----------|--------------|--------------|---------|-------:| | `libpng`| 7.0 | 83.8 | 66.56 | 5.54 | 398 | 24.2% | | `stbi` | 7.0 | 60.5 | 66.63 | 7.67 | 561 | 34.2% | | `qoi` | 2.1 | 2.9 | 226.03 | 161.99 | 463 | 28.2% | The results show that QOI is significantly faster than libpng and `stb_image_write`, and it also achieves better compression ratios than libpng on average. ### Advantages Some of the key advantages of QOI include: - Super simple: [the spec](https://qoiformat.org/qoi-specification.pdf) is only one page - Extremely fast encoding & decoding speeds - Data chunks are byte-aligned, so data can be streamed to a decoder one byte at a time - Better compression ratios compared to PNG for many types of images - Supports transparency - Royalty-free, open-source (CC0), & easy to integrate into any application ### Limitations - Limited to 8 bits per channel (no support for higher bit depths) - Not suitable for images with high-frequency noise or very little spatial redundancy - Lacks advanced features like progressive loading, interlacing, or custom metadata Despite its limitations, QOI provides a compelling alternative to PNG for many use cases where simplicity, speed, and good compression ratios are desired. QOI is not especially well supported at present, but adoption is rapidly growing as developers can easily integrate support into their applications due to the format's simplicity. --- # WebP Canonical URL: https://encode.wiki/codecs/images/webp/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 5ae978ebcac1b9a63297ea8571597f90ce683b0a75d8ebf251ab868667acd80e WebP is a free image file format first released by Google in 2010. It consists of 2 primary "modes" of operation. A lossy mode derived from the [VP8](/codecs/video/vp8/) video codec, and a novel lossless mode added in 2011. ## Performance Checklist Lossless? *Yes* Lossy? *Yes* Supported Bit Depth: *8 BPC* HDR/Wide Gamut? *No* Animation? *Yes* Transparency? *Yes* Progressive Decode? *No* Royalty Free? *Yes* ## History Google announced the WebP format on 30 September 2010. It was initially proposed as a new open format for lossy compressed true-color graphics on the web, aiming to create files that were smaller than comparable JPEG files while maintaining similar image quality. The foundation of WebP utilized technology acquired by Google during its purchase of On2 Technologies. WebP is closely related to VP8, serving as a derivative, and is a sister project to the WebM multimedia container format. The reference library (libwebp) is released under a BSD free software license. Additional features were added after the initial release: - Extended File Format (October 2011): An extension enabling WebP to support features like animation, embedding an ICC profile, and including XMP and Exif metadata. This format also initially supported tiling, but that was later removed. - Lossless Compression and Transparency (November 2011): Google announced a new lossless compression mode and the ability to support transparency (alpha channel) in both lossy & lossless modes. This support was enabled by default in libwebp, starting with version 0.2.0 on 16 August 2012. Google's initial benchmarks showed that converting PNG files found on the web to lossless WebP resulted in a 45% reduction in file size. ### Adoption libwebp is developed by Google, and reached version 1.0 in April 2018. In November 2024, WebP was formally specified and published by the IETF as [RFC 9649](https://www.rfc-editor.org/rfc/rfc9649). WebP has seen widespread adoption across the internet to reduce image sizes. [caniuse.com](https://caniuse.com) reports that over [95% of web browsers fully support WebP](https://caniuse.com/?search=webp). Support has also grown across various software programs since 2010. ### Criticism Despite its adoption, WebP has faced criticism: Critics, including [Josh Aas from Mozilla Research](https://research.mozilla.org/2014/07/15/mozilla-advances-jpeg-encoding-with-mozjpeg-2-0/) in 2014, have questioned whether the format offers significant speed benefits, noting that studies were "not able to conclude that WebP outperformed JPEG by any significant margin". Early critiques in 2010 noted that the quality of WebP-encoded results could be poor, particularly due to blurriness, arguing that the encoder attempted to optimize too hard for [PSNR](/reference/metrics/psnr/) rather than [psychovisual quality](/start-here/psychovisual/). There is apparently an attempt to address this issue in the form of Halide Compression's [Iris-WebP](https://halide.cx/iris/) encoder released in 2025, though performance claims cannot be directly validated due to the encoder's proprietary nature. The biggest challenge facing WebP's reputation has been lack of compatibility with older software, and the practice of content delivery networks replacing source JPEG/PNG files with WebP versions have been cited as making the format "user-unfriendly". Users who download images often find they must convert the file format later. ### Vulnerabilities In September 2023, critical vulnerabilities were discovered relating to WebP images in libwebp's decoder. [CVE-2023-4863](https://www.cve.org/CVERecord?id=CVE-2023-4863), was actively exploited and carried a high-risk rating (CVSS 8.8). This flaw could be triggered by a maliciously crafted lossless WebP file, potentially causing an overflow condition that could result in denial of service or remote code execution. The extensive use of libwebp across major browsers posed a patching challenge as well. ### WebP 2 In June 2021, Google began developing WebP 2, intended to achieve better compression ratios while providing faster encoding and decoding speeds than other modern formats. However, in October 2022 Google changed [the development repository](https://chromium.googlesource.com/codecs/libwebp2/)'s README file to state that "WebP 2 would not be released as an image format." ## Technical Details The WebP file format is based on the [Resource Interchange File Format](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format) (RIFF). The file structure is composed of chunks, each identified by a 32-bit FourCC (four-character code) and followed by a 32-bit size field indicating the payload size. The WebP container allows for features beyond a single VP8 key frame. The side length of WebP images is limited to 16,383 pixels squared. WebP uses two primary compression schemes: lossy (based on VP8) and lossless (novel). ### Lossy See the [VP8](/codecs/video/vp8/) page for more information. ### Lossless > **Note — Lossless Compression.** > > If you would like to dive more deeply into the topic of lossless compression, you can check out the [Lossless Compression](/start-here/lossless/) entry in the Introduction section of the wiki. WebP's lossless compression uses a newer algorithm designed by Google software engineer Jyrki Alakuijala, which is unrelated to VP8. Simple lossless WebP files use a RIFF container followed by a "VP8L" chunk containing the VP8L bitstream data. Lossless WebP supports **8-bit RGBA** (red, green, blue, alpha) color space exclusively. The format stores and restores pixel values exactly, even for fully transparent pixels. It relies on a universal algorithm for sequential data compression (LZ77), prefix coding (Huffman coding), and a color cache to compress bulk data. The lossless mode employs dedicated entropy codes for different color channels, exploitation of 2D locality of backward reference distances, A color cache for recently used colors (which allows referencing them with shorter codes), and transforms such as the Predictor Transform, Color Transform, Subtract Green Transform, and Color Indexing Transform are applied before entropy coding to reduce symbolic entropy. A distinctive aspect of the lossless format is its *recursive definition*: control images used for functions like local entropy code selection are encoded using the same methods as the main image itself. ### Extended Format Features The WebP container format (the RIFF container for WebP) enables several advanced features collectively known as the *Extended File Format*. WebP supports animation, which is promoted by Google as an alternative to GIF. Animated WebP supports 24-bit color depth with transparency, allows combining frames using both lossy and lossless compression within the same animation, and supports seeking to specific frames. The animation parameters are managed by: - A 'VP8X' chunk to indicate the extended WebP spec - An 'ANIM' Chunk containing global parameters, such as the background color & loop count (0 = infinite loops) - Multiple 'ANMF' Chunks: Contain information for a single frame, including its position, duration, blending method (alpha-blending or overwrite), and disposal method (leave canvas as is, or dispose to background color). Converting animated GIFs to lossy WebP can reportedly reduce file size by 64%. Additionally, transparency information is supported through an alpha channel. In the extended format, alpha data for lossy images is stored in an optional 'ALPH' chunk. This data can be stored as raw 8-bit transparency values or compressed using the WebP lossless format. The extended format also allows embedding metadata and color profiles. An image may contain an embedded ICC profile, described by the International Color Consortium. If not present, sRGB is assumed. Metadata can be stored in Exif or XMP formats, placed in optional 'EXIF' or 'XMP' chunks. The RIFF container allows for the inclusion of *unknown chunks* (FourCCs not defined in the specification) for future extensions or application-specific data. Readers should ignore these chunks, and writers should preserve them. ## Conclusion WebP is largely succeeded by [AVIF](/codecs/images/avif/), and the libwebp encoder has not received adequate psychovisual attention to compete with more technically advanced encoders like [libaom](/tools/software-encoders/aomenc/) or more well-optimized perceptual encoders like Google's [jpegli](https://github.com/google/jpegli) JPEG encoder. While a lot of WebP's original criticisms aren't as relevant in 2025, the three biggest issues with the format that have historically drawn the most ire (lack of support, vulnerabilities, poor perceptual encoding) have marred WebP's reputation in a way that the community may not reconcile for a long time, if ever. Especially since the Chromium browser engine's removal of the [JPEG XL](/codecs/images/jxl/) image format (effectively killing widespread web adoption), new video-derived web image codecs sit in an uncomfortable place, and WebP is the poster child of this unfortunate discomfort. --- # SRT Canonical URL: https://encode.wiki/codecs/subtitles/srt/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: e56a9fa55c8ef99da8e84a2aa9daf846a69e30c1b7d4b5fb1520beb639ad1b8c SubRip Text (SRT) is a text format for subtitles, described as 'the most basic of all subtitle formats'. SRT files are plain text with the extension `.srt`. ## Format Subtitles are placed into sequentially ordered groups, called cues, with a starting and ending timestamp, encoded `hours:minutes:seconds,milliseconds`. Note the seperator for the millisecond value is a comma. The starting and ending value are seperated by ` --> `. ### Unoffical features Some basic HTML tags are supported by some viewers, such as: * `bold` **bold** * `italics` *italics* * `` underlined * `Blue` colored text. Note these will be displayed verbatim on viewers that don't support these features. ## Example ``` 1 00:00:00,000 --> 00:01:00,000 This subtitle will be visible for the first minute of the stream 2 00:01:00,000 --> 00:01:30,000 and this one for thirty seconds after that. ``` --- # SubStation Alpha Canonical URL: https://encode.wiki/codecs/subtitles/ssa/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: f5cd39efaccab34d10ff93db87ee46210db67b2852309361d01e4fc52781c5f3 SubStation Alpha (SSA), also known as Advanced Substation Alpha (ASS) for v4+, is a subtitle format. It was originally used by the Windows program of the same name, aimed at the karaoke and anime communities. It's advanced styling compared to alternatives made it popular with release groups. # Overview SSA is a "plain" text format, with Unicode support in ASS. It can either be left as a plain text file with the extension of .ssa or .ass, or muxed into a Matroska (.mkv) or AVI (.avi) file. The original SubStation Alpha software is abandonware, however a wide variety of media authoring, muxing, and playing software supports SSA, including [VLC](/tools/video-players/), [MPV](/tools/video-players/), and [FFmpeg](/tools/utilities/ffmpeg/). ## Format SSA uses the word "script" to refer to the subtitles that track a video. The character ';' at the beginning of a line is used to mark comments. ### ASS (SSA v4+) header ``` [Script Info] ; This is an Advanced Sub Station Alpha v4+ script. ; For Sub Station Alpha info and downloads, ; go to http://www.eswat.demon.co.uk/ ; or email kotus@eswat.demon.co.uk ; ; Advanced Sub Station Alpha script format developed by #Anime-Fansubs@EfNET ; http://www.anime-fansubs.org ; ; For additional info and downloads go to http://vobsub.edensrising.com/ ; or email gabest@freemail.hu ; ; Note: This file was saved by Subresync. ; ScriptType: v4.00+ Collisions: Normal PlayResX: 384 PlayResY: 288 Timer: 100.0000 [V4+ Styles] Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding Style: Default,Tahoma,16,&H00000000,&H00ffffff,&H00ffffff,&H00c0c0c0,-1,0,0,0,100,100,0,0.00,1,2,3,2,20,20,20,1 [Events] Format: Layer, Start, End, Style, Actor, MarginL, MarginR, MarginV, Effect, Text Dialogue: 0,0:01:41.70,0:01:46.84,Default,,0000,0000,0000,,Le rugissement des larmes !\NTu es mon ami. Dialogue: 0,0:02:00.99,0:02:02.87,Default,,0000,0000,0000,,Est-ce vraiment Naruto ? ``` ## Further reading: [Multimedia wiki](https://wiki.multimedia.cx/index.php/SubStation_Alpha) [Specification](http://moodub.free.fr/video/ass-specs.doc) [Archived original software release](https://web.archive.org/web/20030603235926/http://www.eswat.demon.co.uk/substation.html) --- # WebVTT Canonical URL: https://encode.wiki/codecs/subtitles/webvtt/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 83c7120e7fd521e7f71cdfe9c0c405f5dc17e05d9b5ece427f3c3995da9bde00 WebVTT, or Web Video Text Tracks, is the format for subtitles on the web. It is used with the [HTML \ element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/track), or embedded into a webm container. ## Structure WebVTT is a simple, text-based format, based on [SRT](/codecs/subtitles/srt/). All files start with the string `WEBVTT`, optionally, some text, then two new lines. That's where the data we're interested in starts. ### Cue A WebVTT file is basically a bunch of cues. They can have a line with an ID, then they have to have a line specifying from where to where the cue should be displayed like this: `STARTTIME -> ENDTIME [optional settings go here]`, then all the text to be displayed goes after it. That text can have some HTML-like formatting in it. To learn about them, see [the documentation](https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API#webvtt_cues). ## Example ``` WEBVTT 00:01.000 --> 00:04.000 - Never drink liquid nitrogen. 00:05.000 --> 00:09.000 - It will perforate your stomach. - You could die. ``` --- # AV1 Canonical URL: https://encode.wiki/codecs/video/av1/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: eda5d3c1e29eb351076340282af66331b947011f3b73aa48bcd48e80079b0678 AV1 is a royalty-free video compression format designed to succeed [VP9](/codecs/video/vp9/). It presently competes with [VP9](/codecs/video/vp9/), [VVC](/codecs/video/vvc/), and [HEVC](/codecs/video/hevc/). AV1 is computationally more complex than VP9, but is fast to decode due to the mature and efficient dav1d AV1 decoder. AV1 hardware accelerated decoding is also available on a variety of different consumer hardware devices, all of which are enumerated [on Wikipedia](https://en.wikipedia.org/wiki/AV1#Hardware). Standout entries include modern Intel, AMD, & Nvidia integrated & discrete GPUs, Google's Tensor SoC powering the Pixel line, Apple's A17 Pro in the iPhone 15 Pro series, and modern Mediatek & Qualcomm chips. YouTube is currently in the process of transitioning their videos to use AV1. There are a number of viable AV1 encoding solutions available today. The three best, most ubiquitous, and free implementations are [aomenc](/tools/software-encoders/aomenc/), [SVT-AV1](/tools/software-encoders/svt-av1/), & [rav1e](/tools/software-encoders/rav1e/). # A Technical Overview of AV1 *This section has been graciously borrowed from Qu Pengfei's amazing [AV1 README.md](https://github.com/QuPengfei/Technical-Overview-Of-AV1-Spec/blob/master/README.md), with some minor grammar, formatting, and spelling corrections. Thank you, Qu Pengfei!* ## Abstract AV1 (AOMedia Video Codec 1.0) evolved on the basis of VP9 (Google), Thor (Cisco) and Daala (Mozila) under the AOM (Alliance for Open Media). It includes a number of enhancement and the new tools that have been added to improve the coding efficiency. The new tools that are added so far include 4 main aspects: prediction, transform, in-loop filter and entropy encoder. This document provides a snapshot of the coding tools in the current finalized version (on March, 2018) of AV1 spec. ## Introduction According to the AOM web page, AV1 is designed with the following feature. - Royally free - Scales to any modern device at any bandwidth - For use in both commercial and non-commercial content, including user-generated content - Developed for the internet and related applications and services-from browsers and streaming to videoconferencing services - Designed with a low computational footprint and optimized for hardware - Bringing features like 4k UHD, HDR, and WCG to real-time video ## Profile & Levels Profiles and levels specify restrictions on the capabilities needed to decode the bitstreams. The profile specifies the bit depth and subsampling formats supported, while the level defines resolution and performance characteristics. By now levels is still under discussion and there is no more details. AV1 support the three named profiles as the table list. | Profile | Bit depth | Monochrome support | Chroma subsampling | Name | |---------|-----------|--------------------|---------------------|--------------| | 0 | 8/10 | Yes | 4:2:0 | Main | | 1 | 8/10 | No | 4:4:4 | High | | 2 | 8/10 | Yes | 4:2:2 | Professional | | 2 | 12 | Yes | 4:2:0, 4:2:2, 4:4:4 | Professional | Table 1. AV1 Profile ## Block Structure ### Basic Coding block AV1 support the larger super block size, which is up to 128x128 super block is allowed. It supports from 128x128 down to 4x4 coding block. Each 4x4 luma block is allowed to independently select inter or intra mode, its reference mode, and interpolation filter type. For Chroma, 2x2 block size is allowed but still 4x4 transform block size is used. ### Basic Prediction Block AV1 support up to 10 partition type. The size of partition unit is allowed down to 4x4 and totally there are 24 types of block size. | Partition index | Type of partition | |-----------------|-------------------| | 0 | PARTITION_NONE | | 1 | PARTITION_HORZ | | 2 | PARTITION_VERT | | 3 | PARTITION_SPLIT | | 4 | PARTITION_HORZ_A | | 5 | PARTITION_HORZ_B | | 6 | PARTITION_VERT_A | | 7 | PARTITION_VERT_B | | 8 | PARTITION_HORZ_4 | | 9 | PARTITION_VERT_4 | Table 2. Type of Block partition | Index | Partition Block size | Index | Partition Block size | |-------|----------------------|-------|----------------------| | 0 | BLOCK_4X4 | 12 | BLOCK_64X64 | | 1 | BLOCK_4X8 | 13 | BLOCK_64X128 | | 2 | BLOCK_8X4 | 14 | BLOCK_128X64 | | 3 | BLOCK_8X8 | 15 | BLOCK_128X128 | | 4 | BLOCK_8X16 | 16 | BLOCK_4X16 | | 5 | BLOCK_16X8 | 17 | BLOCK_16X4 | | 6 | BLOCK_16X16 | 18 | BLOCK_8X32 | | 7 | BLOCK_16X32 | 19 | BLOCK_32X8 | | 8 | BLOCK_32X16 | 20 | BLOCK_16X64 | | 9 | BLOCK_32X32 | 21 | BLOCK_64X16 | | 10 | BLOCK_32X64 | 22 | BLOCK_32X128 | | 11 | BLOCK_64X32 | 23 | BLOCK_128X32 | Table 3. Size of Block Partition ### Basic Transform Block Both square and rectangle transform block size is supported in AV1. There are total 19 transform block size. | Index | TxSize | Index | TxSize | |-------|----------|-------|----------| | 0 | TX_4X4 | 10 | TX_32X16 | | 1 | TX_8X8 | 11 | TX_32X64 | | 2 | TX_16X16 | 12 | TX_64X32 | | 3 | TX_32X32 | 13 | TX_4X16 | | 4 | TX_64X64 | 14 | TX_16X4 | | 5 | TX_4X8 | 15 | TX_8X32 | | 6 | TX_8X4 | 16 | TX_32X8 | | 7 | TX_8X16 | 17 | TX_16X64 | | 8 | TX_16X8 | 18 | TX_64X16 | | 9 | TX_16X32 | | | > Table 4. Size of Transform Block ## Intra Prediction Intra Prediction in AV1 expends largely compared to VP9. Here is snapshot of Intra Mode. | Index | Intra mode | AV1 | VP9 | Comments | |-------|---------------------|-----|-----|-----------------------------------------------| | 0 | DC_PRED | X | X | | | 1 | V_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 2 | H_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 3 | D45_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 4 | D135_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 5 | D113_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 6 | D157_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 7 | D203_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 8 | D67_PRED | X | X | AV1 support 7 kind of mode based on this mode | | 9 | SMOOTH_PRED | X | | | | 10 | SMOOTH_V_PRED | X | | | | 11 | SMOOTH_H_PRED | X | | | | 12 | TM_PRED(PAETH_PRED) | X | X | AV1 replace TM_PRED with PAETH_PRED | | 13 | Palette Mode | X | | | Table 5. Summary of Intra Mode between AV1 and VP9 ### Directional Intra Prediction Mode VP9 only supports 8 directional intra prediction modes: D45_PRED, D63_PRED, H_PRED, D117_PRED, D135_PRED, D153_PRED, V_PRED, D207_PRED. These modes correspond to prediction angles of 45, 63, 90, 117, 135, 153, 180, and 207 degrees, respectively. To improve intra coding efficiency, more prediction angle options are added to AV1. The prediction angle is calculated as the following: Prediction angle = nominal_angle + (angle_delta \* angle_step), | nominal_angle | angle_step | angle_delta | Total number of angles | |-------------------------------------|------------|-------------|------------------------| | 45, 63, 90, 117, 135, 153, 180, 207 | 3 | [-3, +3] | 8\*7=56 | Table 6. Finer of Intra Mode - norminal_angle is determined by the prediction mode, and is the same as VP9; - angle_delta is in a predefined range and angle_step is a predefined value. In current configuration, angle_delta is in the range of [-3, +3] and angle_step is 3. These settings are selected experimentally. - The total number of supported prediction angles is therefore increased from 8 to 8 \* 7 = 56. ### Smooth Mode It is a Non- Directional Intra Prediction mode. VP9 has 2 non-directional intra prediction modes: DC_PRED and TM_PRED. AV1 expands on this by adding 3 new smooth prediction modes: SMOOTH_PRED, SMOOTH_V_PRED and SMOOTH_H_PRED. The new modes work as follows: |Mode|Comments| |-|-| | SMOOTH_PRED | Useful for predicting blocks that have a smooth gradient. It works as follows: estimate the pixels on the rightmost column with the value of the last pixel in top row, and estimate the pixels in the last row of the current block using the last pixel in left column. Then calculate the rest of the pixels by an average of quadratic interpolation in vertical and horizontal directions, based on distance of the pixel from the predicted pixels. | | SMOOTH_V_PRED | Similar to SMOOTH_PRED, but uses quadratic interpolation only in the vertical direction | | SMOOTH_H_PRED | Similar to SMOOTH_PRED, but uses quadratic interpolation only in the horizontal direction | Table 7. Smooth mode of Intra mode ### Paeth Mode It is a Non- Directional Intra Prediction mode. The new prediction mode PAETH_PRED replaces the existing mode TM_PRED. TM_PRED: Predictor(TM) = left + top – top_left PAETH_PRED: Predictor (PAETH) = argmin \|x- Predictor(TM)\| The idea is to find out the One of left, top, top_left closest in value to Predictor(TM). ### Palette Mode Sometimes, given intra block can be approximated by a block with small number of unique colors. This is especially true for artificial videos like screen-capture, games etc. For such cases, AV1 introduces a new intra coding mode called palette mode. This predictor for a block is signaled by storing (i) a color palette, with 2 to 8 colors, and (ii) color indices into the palette for all pixels in the block. The residual pixel values of the block are as usual transformed and quantized before being entropy-coded. Palette mode can be used by both intra-only as well as inter frames. The number of base colors determines the trade-off between fidelity and compactness. The color indices for pixels are obtained by the nearest neighbor method. The color indices are encoded using the neighborhood-based context to be as compact as possible. Palette Mode is not new. We can see the Palette Mode and Intra block copy in the HEVC SCC (Screen Content Coding) extension. ### Filter Intra mode AV1 adopt the new mode to interpolate (intra filter) the reference samples before prediction. This will reduce the impact of quantization noise. Here is the table to specify the type of intra filtering. | Index | Filter intra type | |-------|-------------------| | 0 | INTRA_DC_PRED | | 1 | INTRA_V_PRED | | 2 | INTRA_H_PRED | | 3 | INTRA_D153_PRED | | 4 | INTRA_TM_PRED | Table 8 Type of Intra filter Mode ### Intra Block Copy Mode This tool is very efficient for coding of screen content video in that repeated patterns in text and graphics rich content occur frequently within the same picture. Having a previously reconstructed block with equal or similar pattern as a predictor can effectively reduce the prediction error and therefore improve coding efficiency. In AV1, Intra block copy is only allowed in intra frames. It disables all loop filtering and only integer offsets are allowed in block copy mode. ### Predict Chroma from Luma Chroma from luma (CfL) prediction is a new and promising chroma-only intra predictor that models chroma pixels as a linear function of the coincident reconstructed luma pixels. ## Inter Prediction ### Affine/Warped Motion Compensation Traditional modern codecs, including VP9, use block motion compensation where motion vectors are translational only. This is not sufficient for real video which often contains complex motion. For example, motion due to camera shake, panning and zoom might require transformations that support shearing, scaling, rotation and changes in aspect ratio. In AV1, we introduce warped motion compensation implemented as similarity and affine transformations to better capture the diversity of motion that exists in real video. There are two affine/warped motion compensation. | Affine Motion Compensation | Comments | |----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Global | It is common for videos to contain a global camera motion which is pertinent to an entire inter frame. It is therefore beneficial to transmit a set of motion parameters at the frame level that is applicable to a large number of blocks in the frame. When a frame is encoded, a set of global motion parameters is computed and transmitted between that frame and each reference frame. These parameters may be either translational, similarity or affine motion model. Subsequently, any block in the frame can signal use of the global motion mode with a given reference to create a suitable predictor. | | Local | Affine motion compensation is also useful to describe complex local object motion. Here, we estimate affine parameters for a single block using the translational motion vectors that are typically conveyed for all inter blocks. Specifically, we estimate an affine or similarity model using the motion vectors from the current block and its causal neighbors which share the same reference frame. | Table 9 Affine Motion Compensation ### OBMC (Overlapped Block Motion Compensation) Motions assigned to surrounding blocks will contribute to predicting a current block, via a well-defined overlapping scheme appropriately designed for advanced variable block-size partitioning frameworks. The OBMC will blend multiple predictors from neighbor blocks. It is not new concept and was proposed and implemented back in the era of h.263. The OBMC was proved to largely reduce prediction errors but not adopted by recent codecs due to extra complexity in the scenario of hybrid inter/intra variable block size coding. In AV1, a practical overlapping mechanism based on two-stage 1-D filtering is proposed for the advanced partitioning framework to implement causal overlapped block prediction. ### Sub-pixel Interpolation Filter The motion vector used in modern video codecs is allowed to have a fractional position for a better prediction quality. So, an interpolation filter module is needed to generate the prediction block at a fractional position in the reference frame. VP9 codec uses a separable interpolation filter to perform inter prediction with ⅛ motion vector precision. Three filter types, SHARP, REGULAR and SMOOTH, in descending order of cutoff frequencies, are provided to deal with various types of noise/distortions that can occur in reference frames/blocks. Given a filter type and a motion vector, the interpolation filter is performed by two one-dimensional filters, one for horizontal direction and one for vertical direction. In AV1 codec, dual interpolation filter is introduced on top of the interpolation module inherited from VP9. Dual filter allows each block/frame to use a different interpolation filter type in horizontal and vertical direction. Up to 9 types of filter will be applied to the block. This idea is based on the observation that a reference frame/block’s horizontal and vertical signals may have distinct frequency characteristics; therefore, using different filter types may produce a better prediction. As before, both the filter types are transmitted in the bitstream on a per block or per frame basis. At the same time AV1 use the high intermediate precision between the horizontal and vertical filter. The same high precision before average the predictors with compound mode. ### Dynamic MV reference VP9 has two candidates MV in the ref list and 4 type of mode (NEARESTMV, NEARMV, NEWMV, and ZEROMV) are used. AV1 support 4 candidate MV and more modes. For single ref mode, AV1 is same as VP9. For compound mode, VP9 restricts motion vectors for a compound predictor to share one motion vector referencing mode, even though they may use different reference frames. To add more flexibility, on top of existing four combinations (NEAREST_NEARESTMV, NEAR_NEARMV, NEW_NEWMV, ZERO_ZEROMV) in VP9, AV1 supports four more empirically selected combinations: NEAREST_NEWMV, NEW_NEARESTMV, NEAR_NEWMV, and NEW_NEARMV. | Index | Type | Ref Mode | |-------|------------------------------|-----------------| | 0 | NEARESTMV | single ref mode | | 1 | NEARMV | single ref mode | | 2 | GLOBALMV(ZEROMV) | single ref mode | | 3 | NEWMV | single ref mode | | 4 | NEAREST_NEARESTMV | compound mode | | 5 | NEAR_NEARMV | compound mode | | 6 | NEAREST_NEWMV | compound mode | | 7 | NEW_NEARESTMV | compound mode | | 8 | NEAR_NEWMV | compound mode | | 9 | NEW_NEARMV | compound mode | | 10 | GLOBAL_GLOBALMV(ZERO_ZEROMV) | compound mode | | 11 | NEW_NEWMV | compound mode | Table 10 MV mode ### Extended Compound Modes AV1 Compound mode support both predictors from the same direction and VP9 only support from the different direction (One forward and one backward reference frame). VP9 only support 1/2 weight to blend the two predictor and AV1 support more flexible weight blending. | Index | Compound type | Comments | |-------|-------------------|---------------------------------------------------------------------------------------------------------------------------------------| | 0 | COMPOUND_WEDGE | Inter-Inter Wedge mode Inter-Intra Wedge mode | | 1 | COMPOUND_SEG | Inter-Inter Compound Segment mode | | 2 | COMPOUND_AVERAGE | (1/2,1/2) weight will be applied to blend the predictors | | 3 | COMPOUND_INTRA | Inter-Intra Gradual mode | | 4 | COMPOUND_DISTANCE | This process computes weights to be used for blending predictions together based on the expected output times of the reference frames | Table 11. Compound type Here are more details about the Compound Segment Mode: - Inter-Inter Compound Segment mode In many cases, regions in one predictor will contain useful content that is not present in the other. The two inter predictors have a larger pixel difference generally. - Inter-Inter Wedge mode Boundaries of moving objects in a video often separate two regions with distinct motions. Coding these regions with separate motion vector reference combinations should be beneficial; however, finding exact object boundaries is not only difficult, but expensive to communicate in the bitstream. Our approach is to design a codebook of masks with only a few possible partitioning combinations and signaling the codebook index in the bitstream. The AV1 wedge codebook contains partition orientations that are either horizontal, vertical or oblique with slopes: 2, -2, 0.5 and -0.5. The wedge prediction mode is used for all square and rectangular blocks, using the 16-ary shape codebooks. | Index | Wedge direction | Comments | |-------|------------------|----------| | 0 | WEDGE_HORIZONTAL | | | 1 | WEDGE_VERTICAL | | | 2 | WEDGE_OBLIQUE27 | | | 3 | WEDGE_OBLIQUE63 | | | 4 | WEDGE_OBLIQUE117 | | | 5 | WEDGE_OBLIQUE153 | | Table 12. Wedge direction - Inter-Intra Gradual mode Decay the weight gradually for the intra from the prediction boundary and increase the weight of inter correspondingly. It support four modes, which include horizontal mode, vertical mode, DC_PRED, and SMOOTH_PRED. - Inter-Intra Wedge mode Blocks cannot always perfectly partition moving objects. For example, occlusion can occur in the middle of a block, it is better to apply different prediction techniques to different contents. Contents that are not occluded in reference frame will prefer inter prediction, while newly revealed content could benefit more from intra prediction using local reference. ### Extended Reference frame Number Up to 7 reference frames out of 8 in the frame stored buffer are extended to be used in the inter mode. The reference frames is allowed to come from the same side or different side in the AV1. LAST3_FRAME, LAST2_FRAME and LAST_FRAME are forward references and LAST_FRAME is the near past frame. BWDREF_FRAME is a backward reference, similar to ALTREF_FRAME. Here is the table to show the reference frame type. | Index | Ref frame Name | |-------|----------------| | 0 | INTRA_FRAME | | 1 | LAST_FRAME | | 2 | LAST2_FRAME | | 3 | LAST3_FRAME | | 4 | GOLDEN_FRAME | | 5 | BWDREF_FRAME | | 6 | ALTREF2_FRAME | | 7 | ALTREF_FRAME | Table 13 Reference frame type ## In-loop Filter Several in-loop tools in AV1 are employed. De-blocking, CDEF and loop restoration are cascaded. ### De-blocking filter AV1 support 4 filter levels per frame and VP9 only has one. Two levels are for Luma component (horizontal and vertical levels). The other two levels are for U and V component separately. In AV1, filter level is allowed to change superblock by superblock. ### CDEF (Constrained Directional Enhancement Filter) CDEF is the combination of CLPF (Constrained Low Pass Filter) and Deringing filter. The main goal of the in-loop CEDF is to filter the coding artifacts and ringing while preserving the detail of image. It takes into account the direction of edge and patterns in the image. It is the similar to the SAO of HEVC. The CDEF is based on the following observation. The amount of ringing artifacts in a coded image tends to be roughly proportional to the quantization step size. The amount of detail is a property of the input image, but the smallest detail actually retained in the quantized image tends to also be proportional to the quantization step size. For a given quantization step size, the amplitude of the ringing is generally less than the amplitude of the details. CDEF works as the following steps: - The frame is divided into filter blocks of 64x64 pixels. Some CDEF parameters are signaled at the frame level, and some may be signaled at the filter block level. - To identify the direction of edge or pattern in each filter block. - To adaptively filter along the identified direction and to a lesser degree along directions rotated 45 degrees from the identified direction. The filter strengths are signaled explicitly, which allows a high degree of control over the blurring. The main reason for identifying the direction is to align the filter taps along that direction to reduce ringing while preserving the directional edges or patterns. CDEF defines primary taps and secondary taps filter. The primary taps follow the direction and the secondary taps form a cross, oriented 45 off the direction. Both primary and secondary taps filter have 8 types. ### LR (In-loop Restoration) filter AV1 employ a set of in-loop image restoration tool after de-blocking to generally de-noise and enhance the quality of the edge. In-loop restoration scheme have two types of filter to remove blur artifacts due to block processing. One is Wiener Filter. The other is Dual Self-Guided filter. These tools are integrated into AV1 with a switchable framework, which trigger the different tool in the different image region. ## Multi-Symbol Entropy Coder Multi-symbol adaptive arithmetic coding model is adopted in AV1. Both syntax element and coefficient are coded with this model. Most recent video codecs encode information using binary arithmetic coding, such as CABA or CAVLC in AVC/HEVC, meaning that each symbol can only take two values. The AV1 entropy encoder come from the Daala range coder and supports up to 16 values per symbol, making it possible to encode fewer symbols. This is equivalent to coding up to four binary values in parallel and reduces serial dependencies, allowing hardware implementations to use lower clock rates, and thus less power. ## Transform ### Transform type For AV1, there is a richer set of transforms for coding Inter and Intra prediction residues. Inter prediction residues do not have a well-defined structure as in the Intra case, but using a bank of transforms, each adapted to a specific type of residue profile within the block, is generally helpful. In AV1, four types of transform are used mainly in the horizontal and vertical direction separately. The total 16 different transforms are available. | Transform type | Comments | |----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------| | DCT | Inter and Intra modes continue to make use of DCT. | | ADST | Asymmetric Discrete Sine Transform | | Flip ADST | It applies ADST in reverse order | | IDTX | Identity transform seems to be particularly useful for coding residue with sharp lines and edges. Identity transform is useful for screen content coding | Table 14 The Main Transform Type in each of direction For each small coded block (4x4 or 8x8), it is possible to choose one of up to 16 different transforms as follows(Detail in Table): \{DCT, ADST, FlipADST, IDTX\} horizontal x \{DCT, ADST, FlipADST, IDTX\} vertical As block sizes get larger, some of these transforms begin to act similarly. Thus, a reduced set of transforms is used for 16x16, 32x32 and 64x64 block sizes. In the transform selection process for Inter and Intra modes, the encoder does a search over the entire set of transforms and selects the one that produces the best rate-distortion cost. Once a transform is selected, a transform type symbol from the set of types available at that size is used to indicate the actual transform used in the bitstream. There are 6 types of transform sets in the AV1 spec, which specify the transform type of Intra and Inter blocks. The transform sets determine what subset of transform types can be used, according to the following table. | Inter or not | Set Number | Transform set | |--------------|------------|----------------| | Don't care | 0 | TX_SET_DCTONLY | | 0 | 1 | TX_SET_INTRA_1 | | 0 | 2 | TX_SET_INTRA_2 | | 1 | 1 | TX_SET_INTER_1 | | 1 | 2 | TX_SET_INTER_2 | | 1 | 3 | TX_SET_INTER_3 | Table 15 Transform Set in the AV1 spec | Transform type | TX_SET_DCTONLY | TX_SET_INTRA_1 | TX_SET_INTRA_2 | TX_SET_INTER_1 | TX_SET_INTER_2 | |-------------------|----------------|----------------|----------------|----------------|----------------| | DCT_DCT | X | X | X | X | X | | ADST_DCT | | X | X | X | X | | DCT_ADST | | X | X | X | X | | ADST_ADST | | X | X | X | X | | FLIPADST_DCT | | | | X | X | | DCT_FLIPADST | | | | X | X | | FLIPADST_FLIPADST | | | | X | X | | ADST_FLIPADST | | | | X | X | | FLIPADST_ADST | | | | X | X | | IDTX | | X | X | X | X | | V_DCT | | X | | X | X | | H_DCT | | X | | X | X | | V_ADST | | | | X | | | H_ADST | | | | X | | | V_FLIPADST | | | | X | | | H_FLIPADST | | | | X | | Table 16 Detailed Transform type supported in each transform set. ### Transform Block Shape and Size Both square and rectangle shape block are used in AV1. The transform block size is less than the partition block size. The block size is very flexible and up to 64x64 and down to 4x4. Details see the table in the Block section. ## Tiles AV1 support flexible tiles, which include uniform and non-uniform tile spacing. Tile area is limited to a maximum 4096x2304. Tiles can be grouped into tile group and each group can be decoded independently to achieve error resilience. Loop filter can be enabled or disabled across tiles. ## Segment Same as VP9, AV1 provides a means of segmenting the image and then applying various adjustments at the segment level. Up to 8 segments may be specified for any given frame. For each of these segments it is possible to specify: - A quantizer (absolute value or delta). - A loop filter strength (absolute value or delta). - A prediction reference frame. - A block skip mode that implies both the use of a (0,0) motion vector and that no residual will be coded. ## SVC (Scalable Video Coding) AV1 support temporal and spatial layer coding. Temporal layer support up to 8 layers and spatial layer support up to 3 layers. | Index | Scalability mode | Index | Scalability mode | |-------|------------------|--------|-------------------| | 0 | SCALABILITY_L1T2 | 8 | SCALABILITY_L2T2h | | 1 | SCALABILITY_L1T3 | 9 | SCALABILITY_L2T3h | | 2 | SCALABILITY_L2T1 | 10 | SCALABILITY_S2T1h | | 3 | SCALABILITY_L2T2 | 11 | SCALABILITY_S2T2h | | 4 | SCALABILITY_L2T3 | 12 | SCALABILITY_S2T3h | | 5 | SCALABILITY_S2T1 | 13 | SCALABILITY_SS | | 6 | SCALABILITY_S2T2 | 14-255 | reserved | | 7 | SCALABILITY_S2T3 | | | Table 17. Temporal and Spatial Mode | Scalability mode | Spatial Layers | Resolution Ratio | Temporal Layers | Inter-layer-dependency | |-------------------|----------------|------------------|-----------------|------------------------| | SCALABILITY_L1T2 | 1 | | 2 | | | SCALABILITY_L1T3 | 1 | | 3 | | | SCALABILITY_L2T1 | 2 | 2:1 | 1 | Yes | | SCALABILITY_L2T2 | 2 | 2:1 | 2 | Yes | | SCALABILITY_L2T3 | 2 | 2:1 | 3 | Yes | | SCALABILITY_S2T1 | 2 | 2:1 | 1 | No | | SCALABILITY_S2T2 | 2 | 2:1 | 2 | No | | SCALABILITY_S2T3 | 2 | 2:1 | 3 | No | | SCALABILITY_L2T2h | 2 | 1.5:1 | 2 | Yes | | SCALABILITY_L2T3h | 2 | 1.5:1 | 3 | Yes | | SCALABILITY_S2T1h | 2 | 1.5:1 | 1 | No | Table 17. Details in the Temporal and Spatial Mode ## Other Tools ### Quantization Matrices AV1 support 15 sets of QMs, which are based on the contrast-sensitive functions. QMs are applied to a frame based on selectable scaling of its quantization level, higher level of quantization imply flatter matrices. The matrices become flatter as the quantization index value increases (and the quality decreases). Inter matrices are slightly flatter than intra matrices. ### Superblock Delta-quantization AV1 allow the per-superblock changes in quantization parameter to support sub-frame rate control. At the same time it support the ROI level rate control on the top of segmentation level parameter. ### OBU (Open Bitstream Unit) An AV1 bitstream consists of a number of OBUs that are normally held within a container format alongside audio and timing information. Here the new tool OBU is introduced in AV1 and it is similar to NAL (Network Abstract Layer) in AVC/HEVC spec. The OBU header is similar to the NAL header. In general the total 8 bits are presented. The OBU extra 8 bits of extension header is used if temporal and spatial layer exist in the bitstream. obu_type is the most important syntax to describe the type of OBU . | Index | obu_type | |-------|----------------------------| | 0 | Reserved | | 1 | OBU_SEQUENCE_HEADER | | 2 | OBU_TD | | 3 | OBU_FRAME_HEADER | | 4 | OBU_TILE_GROUP | | 5 | OBU_METADATA | | 6 | OBU_FRAME | | 7 | OBU_REDUNDANT_FRAME_HEADER | | 8-14 | Reserved | | 15 | OBU_PADDING | Table 18. Type of OBU ## References 1. https://aomediacodec.github.io/av1-spec/av1-spec.pdf --- # AVC / H.264 Canonical URL: https://encode.wiki/codecs/video/avc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 08100f0d368de2fd148de49d8298e1d313cacc0475a62a62ff40831f9e3f6cf1 # H.264 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! H.264, also known as AVC (Advanced Video Coding), is a video compression standard that has played a significant role in multimedia codec technology. H.264 revolutionized video encoding by offering vastly more efficient compression than predecessors, and has been the nearly universal defacto video compression standard since. Its history is marked by continuous refinement and widespread adoption, especially as it pertains to the highly optimized [x264](/tools/software-encoders/x264/) video encoder. Despite being over a decade old, H.264 remains relevant today. Newer codecs like [H.265](/codecs/video/hevc/), [VP9](/codecs/video/vp9/), and [AV1](/codecs/video/av1/) aim to provide more efficient compression than H.264 but are currently not as universal. The choice between these codecs largely depends on the specific requirements of the application, compatibility and support, and the balance between compression efficiency and computational complexity. --- # AVS3 Canonical URL: https://encode.wiki/codecs/video/avs3/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 0c175b2914990b6be7180be3e2feff20ad649537fcd0c1ebe454ea4d3d241a7b > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! AVS3 is the most recent codec from the AVS family. It aims to bring a significant coding improvement over AVS2 and [HEVC](/codecs/video/hevc/). The specification was divided into 2 phases, the first one (main profile) finished in 2019 and the second (high profile) in 2021. AVS3 has more flexible coding tree and block partitioning shemes compared to previous generation AVS codecs. It also employs new DCT-VII and DST-VII transforms. The work on AVS3 was sponsored by organizations including Peking University, Pengcheng Laboratory and Huawei. In 2022 it was [added](https://dvb.org/news/dvb-test-content-for-vvc-and-avs3-codecs-now-available) to worldwide DVB standard alongside [VVC](/codecs/video/vvc/). As of 2024 there several hardware AVS3 decoders including Mediatek and HiSilicon. We should be seeing more AVS3 in hardware since it was added to DVB toolbox. [uavs3](https://github.com/uavs3) is an open source high performance software encoder and decoder of AVS3-P2. The project was initialized by the Peking University Shenzhen Graduate School and over the time was optimized for x86 and arm processors. FFmpeg since version 6 supports only avs3 decoding via [uavs3d](https://github.com/uavs3/uavs3d). Intel partnered with Boya and created [SVT-AVS3](https://www.intel.com/content/www/us/en/developer/articles/technical/scalable-video-technology.html) which to my knowledge is not available to the public. The only open-source encoder available to general public is [uavs3e](/tools/software-encoders/uavs3e/). --- # DSV Canonical URL: https://encode.wiki/codecs/video/dsv/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: ead584d05473a5b522ec0eafaf8007312abddc4bcf0a18def825272565ebeaab DSV or DSV1 (Digital Subband Video 1) is a compact, software-only video codec focusing on simplicity and wavelet-based compression, with features rivaling early MPEG standards. The reference implementation of the codec is designed for flexibility and ease of use in C environments. As of October 2024, DSV1 has been superseded by [DSV2](/codecs/video/dsv2/). ## Features - **Wavelet-based compression:** Uses multiresolution subband analysis (wavelet transform) instead of DCT. More information on wavelet transforms is available on the [DSV2](/codecs/video/dsv2/) page. - **Motion Compensation:** Supports half-pixel motion compensation. - **Chroma Subsampling:** 4:1:1, 4:2:0, 4:2:2, and 4:4:4 formats. - **Adaptive Quantization:** Dynamically adjusts quantization for better quality. - **Frame Types:** Supports intra and inter frames with variable-length closed GOPs (no bidirectional/B-frames, only forward prediction). - **Entropy Coding:** Uses interleaved exponential-Golomb coding for simplicity. - **Compression Range:** Supports a wider range of compression and lower bitrates than MPEG-1/2. The reference DSV encoder features: - **Rate Control:** Single-pass average bitrate (ABR) or constant rate factor (CRF). - **Human Visual System (HVS) Optimization:** Simple intra block mode determination. - **Scene Change Detection:** Based on average luma changes. - **Hierarchical Motion Estimation:** For improved efficiency. - **Adaptive Quantization Stability Tracking:** Improves quality. - **C89 Compatibility:** Written for standard C89, no external libraries, integer-only arithmetic, single-threaded, and portable. ## Limitations - No built-in interlacing support. - Only 8 bits per component supported. - Frame dimensions must be divisible by 2. ### Example Usage The only DSV encoder available is the reference encoder. - Encoding: ``` ./dsv1 e -y -v -inp_bus_cif.yuv -out_saved.dsv -gop12 -w352 -h288 -fps_num30 -qp85 -kbps1100 ``` - Decoding: ``` ./dsv1 d -inp_video.dsv -out_decompressed.yuv -out420p ``` ### Documentation & More Detailed information is available in [the repository](https://github.com/LMP88959/Digital-Subband-Video-1)’s PDF documents (`DSV1_spec.pdf`, `DSV1_encoder.pdf`). More example videos and usage instructions are provided in the README and linked resources in the repo. --- # DSV2 Canonical URL: https://encode.wiki/codecs/video/dsv2/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 10696dd1caa7435fbd39f7f3fafd7d9aa5cc6a9863c3354210d2df64e7bf0d88 **DSV2** (or Digital Subband Video 2) is a lossy and lossless video codec that utilizes a wavelet transform and block-based motion compensation for video compression. It is designed to perform optimally at medium-low to medium-high bitrates for resolutions ranging from CIF (352x288) up to Full HD (1920x1080). In terms of compression efficiency and quality, its performance is comparable to MPEG-4 Part 2 and [AVC/H.264](/codecs/video/avc/) (using only P-frames). The bitstream for version 2.8 was frozen as of June 20, 2025. DSV2 supersedes the original [DSV](/codecs/video/dsv/) codec. ## Core Features DSV2 includes a range of modern codec features, focusing on integer-only operations and avoiding third-party libraries. - **Compression Method**: Employs a multiresolution subband analysis, also known as a wavelet transform, instead of the more common DCT. - **Motion Compensation**: Supports up to quarter-pixel motion compensation and features an "Expanded Prediction Range Mode" (EPRM) for improved prediction. - **Frame Types**: Uses intra (I-frames) and inter (P-frames) with a variable-length closed Group of Pictures (GOP). It does not support bi-directional prediction (B-frames). - **Color Space**: Compatible with multiple chroma subsampling formats, including 4:1:0, 4:1:1, 4:2:0, 4:2:2, and 4:4:4. - **Quality Optimization**: Incorporates adaptive quantization, in-loop filtering, and psychovisual enhancements to improve visual quality. - **Lossless Coding**: Offers a lossless compression mode. ## Encoder Implementation The specific encoder implementation detailed in the project repository has several advanced features: - **Rate Control**: Supports single-pass average bitrate (ABR) and constant rate factor (CRF). - **Scene Detection**: Includes complex algorithms for detecting scene changes. - **Human Visual System (HVS)**: Utilizes HVS-based models for making decisions on intra block modes and adaptive quantization. - **Motion Estimation**: Implements hierarchical motion estimation. ## Limitations The codec has several self-imposed developer limitations as well as technical limitations: - Does not support interlaced video. - Component bit depth is limited to 8 bits. - Frame dimensions must be divisible by two. - The reference implementation is single-threaded and does not use any hardware acceleration, SIMD instructions, or floating-point math. ## Usage The DSV2 reference codec is written in C89 and can be compiled with a standard C compiler or by using the Zig build system. It is the only encoder/decoder pair available for DSV2. Build instructions for the reference implementation as well as source code are available on the [DSV2 GitHub repository](https://github.com/LMP88959/Digital-Subband-Video-2). ### Encoder Usage To encode a video, use the `e` command. The following is a sample command line: ```bash ./dsv2 e -inp=video.y4m -out=compressed.dsv -y4m=1 -qp=60 -gop=48 ``` ### Decoder Usage To decode a DSV2 file, use the `d` command. The following is a sample command line: ```bash ./dsv2 d -inp=video.dsv -out=decompressed.y4m -y4m=1 -out420p=1 ``` ## Wavelets Wavelet-based video codecs like DSV2 utilize wavelet transforms to reduce the amount of data required to represent a digital video. Compared to the widely used [Discrete Cosine Transform](/start-here/terminology/#discrete-cosine-transform-dct) (DCT), wavelets feature essentially the opposite methodology. Each coefficient in a DCT represents a constant pattern applied to an entire block, while each coefficient in a wavelet transform represents a localized pattern applied to a section of the block. Because wavelet transforms can take advantage of large-scale redundancy in an image, they are often used to analyze entire frames at once, or in large overlapping sections. By contrast, DCTs are usually quite small and are intended to cover areas of roughly uniform patterns and complexity (forming the foundation of modern block-based video codecs like [AVC/H.264](/codecs/video/avc/)) ### The Compression Process The core of wavelet compression involves a multi-step process that transforms pixel data into a more compressible format. The CineForm codec serves as a practical example of this process. #### The Wavelet Transform A wavelet can be thought of as a one-dimensional filter that separates low-frequency data (the general, smoother areas of an image) from high-frequency data (the details and edges). To compress a 2D image, this process is applied both horizontally and vertically. For instance, the 2-6 Wavelet used in CineForm calculates low and high-frequency samples. For every two pixels, the low-frequency component is their sum: `low frequency sample = pixel[x] + pixel[x+1]` The high-frequency component is calculated using six input pixels to capture finer details: `high frequency sample = pixel[x] - pixel[x+1] + (-pixel[x-2] - pixel[x-1] + pixel[x+2] + pixel[x+3])/8` This operation is repeated, typically for three levels, on the low-frequency quadrant of the image, progressively concentrating the image's energy into smaller areas. This encompasses the methodology behind the 2-6 Wavelet. #### Quantization After the wavelet transform, the high-frequency data is quantized. The human eye is less sensitive to subtle changes in high-frequency regions, a characteristic exploited by dividing the wavelet output by a quantizer value. This step is a primary source of data reduction, as it discards information that is less likely to be perceived by the viewer. `high frequency sample = (wavelet output) / quantizer` Entroping coding, or [lossless compression](/start-here/lossless/), follows this step. ### Advantages & Disadvantages Wavelet compression presents a different set of trade-offs compared to the block-based DCT approach used in codecs like H.264. #### Advantages - Because wavelet transforms are not confined to sharp-edged blocks, they avoid the "blocking" artifacts and ringing that can appear in DCT-based codecs, especially at lower bitrates. - The efficiency of wavelet codecs like CineForm improves as video resolution increases, making them well-suited for HD, 4K, and 360° video production. - Wavelets can be designed to decode video at lower resolutions at very high speeds, allowing for faster editing workflows. #### Disadvantages Despite their theoretical benefits, wavelet codecs face significant practical challenges that have limited their widespread adoption. - Wavelet codecs lack an efficient method for intra-frame prediction. Unlike H.264, which can predict a block based on its exact neighboring pixels, the overlapping nature of wavelets makes this impossible, resulting in less efficient compression for intra-frames. - To avoid blockiness in motion-compensated frames, wavelet codecs often use Overlapped Block Motion Compensation (OBMC), which is significantly more demanding on CPU resources than standard motion compensation. - At lower bitrates, wavelet codecs can produce a blurry look because they don't preserve visual energy and sharp details as effectively as DCT codecs. This blurriness might result in improved [PSNR](/reference/metrics/psnr/) but can be visually less appealing. Visual aliasing can also occur, where parts of the image appear to be coded at a lower resolution and then poorly scaled up. - Many wavelet codecs do not feature spatial adaptive quantization, a technique that improves visual quality by varying quantization across different areas of an image. This can lead to blurring in areas with subtle textures. --- # ECM Canonical URL: https://encode.wiki/codecs/video/ecm/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 7c4f88bb9624380de73d59f1613e635bb0e552f672651a6efede5820138e0895 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! ECM (Enhanced Compression Model) is a video codec developed by the Fraunhofer Heinrich-Hertz-Institute. It is purportedly designed to succeed H.266, or [VVC](/codecs/video/vvc/). The [Git repo](https://vcgit.hhi.fraunhofer.de/ecm/ECM) provides a bit more information: ```md # ECM reference software This software package is the reference software for Enhanced Compression Model (ECM). The reference software includes both encoder and decoder functionality. Reference software is useful in aiding users of a video coding standard to establish and test conformance and interoperability, and to educate users and demonstrate the capabilities of the test model. The software has been jointly developed by the ITU-T Video Coding Experts Group (VCEG, Question 6 of ITU-T Study Group 16) and the ISO/IEC Moving Picture Experts Group (MPEG, Working Group 11 of Subcommittee 29 of ISO/IEC Joint Technical Committee 1). ``` Other than this generic overview, build instructions are provided in the repo's README. Not much else is known about ECM besides the fact that it will likely become H.267. --- # FFV1 Canonical URL: https://encode.wiki/codecs/video/ffv1/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 029f7fea93a8d037e6f0cb7ebea933d4a150553a74033996de8f0794760d9974 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! FFV1 ([rfc9043](https://datatracker.ietf.org/doc/html/rfc9043)) is a lossless intra-frame video codec designed for archival use and preservation. Created by Michael Niedermayer, it is part of the [FFmpeg](/tools/utilities/ffmpeg/) project. The codec supports a wide range of color spaces, works with YUV and RGB content including alpha channel of color depths ranging from 8 to 16 bits (only up to 14 in case of RGB). It has good parallelization support and achieves very high compression ratios compared to other lossless video encoders such as [UT Video](/codecs/video/utvideo/), albeit at the cost of being more resource-hungry. ## History In 2003, the codec was merged into FFmpeg; however, the bitstream specification was frozen in 2006 (officially FFV1 version 0). Later, in 2009, version 1 came out, covering more video bit depths. Version 2 never got its release, existing only in experimental form. The third bitstream version was frozen in 2013 and is still the latest as of 2024. It added multithreading support and frame integrity checking. There is a fourth version [coming](https://datatracker.ietf.org/doc/draft-ietf-cellar-ffv1-v4/) which might bring better support for color spaces, compression improvements, and maybe proper inter-frame prediction. ## Usage ```bash title="Fast, heavily multithreaded" ffmpeg -i input.mkv -c:v ffv1 -slices 16 out.mkv ``` ```bash title="Slow, highest compression" ffmpeg -i input.mkv -c:v ffv1 -g 60 -slices 4 -context 1 -coder 2 out.mkv ``` ```bash title="Recommended for archival purposes, high compression, multithreaded" ffmpeg -i input.mkv -c:v ffv1 -g 1 -slices 16 -slicecrc 1 -context 1 -coder 2 out.mkv ``` ### Options - `slices` - Slices divide the frame into multiple parts that can be encoded and decoded in parallel. Can only be one of: [`4`, `6`, `9`, `12`, `16`, `24`, `30`], where `4` is the default. - `slicecrc` - Setting it to `1` will enable the decoder to detect errors in the bitstream. Must be enabled for archival use. Can be either `0` or `1`. - `context` - Setting it to `1` will make the encoder use a larger context size, which usually leads to better compression. Can be either `0` or `1`. - `coder` - Sets entropy coding method: - `0` - Golomb-Rice (faster, default) - `1` - Range Coder (used for higher bit depths and better compression) - `2` - Range Coder with custom state transition table (almost the same as `1`) - `g` - Sets GOP size. Must be `1` for archival use. [See below](#intra-frame-only-catch) ## Intra-frame only catch Intra-frame codecs do not use well known inter-frame video coding techniques such as motion compensation, reusing parts of surrounding frames or adapting encoding context based on them. Every frame is independent from one another. Common intra-frame codecs include Motion JPEG (Lossy), Motion JPEG 2000 (Both), and [UT Video](/codecs/video/utvideo/) (Lossless). If you're a careful reader, you might have noticed that setting `GOP size` isn't a common characteristic among intra-frame codecs. In fact, FFV1 can be considered an intra-frame codec only if the `GOP size` is set to `1`. When it's larger than that, its context model depends on other frames found within the GOP, which contradicts the definition of intra-frame video codec. That is why it's highly advised to set `GOP size` to `1` for archiving. This way, if a single frame gets damaged, you'll only lose that frame. If `GOP size` was large, you might lose much bigger part of the video. References: - [Wikipedia](https://en.wikipedia.org/wiki/FFV1) - [RFC9043](https://datatracker.ietf.org/doc/html/rfc9043) - [FFmpeg Docs](https://trac.ffmpeg.org/wiki/Encode/FFV1) - [This Thread](https://forum.shotcut.org/t/exporting-as-ffv1-change-form-to-support-lossless-parameters/41230/19?page=2) - [This Question](https://video.stackexchange.com/questions/24874/what-does-the-context-parameter-mean-when-using-ffv1-in-ffmpeg) --- # HEVC / H.265 Canonical URL: https://encode.wiki/codecs/video/hevc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 8ac1971d9a159947335f175d49fbc995a79c7b413b273851f760f29d96bd87a8 # H.265 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! H.265, also known as High-Efficiency Video Coding (HEVC), is a video compression standard designed to deliver higher-quality videos with reduced file sizes. It is a successor to the [H.264](/codecs/video/avc/) codec and was developed to address the increasing demand for efficient and high-quality video compression, particularly in the context of 4K content. This codec is encumbered by royalties. --- # ProRes Canonical URL: https://encode.wiki/codecs/video/prores/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 2a01a15f628909c3e38f0db610e19607befb3a2cf01326f9919b55d00b25e487 ProRes is a family of lossy video compression codecs developed by Apple Inc. ProRes is designed to serve as a high-quality "visually lossless" video editing codec that offers superior image quality compared to many other editing codecs while still maintaining a relatively compact file size compared to uncompressed video. ProRes uses 4:2:2 or 4:4:4 chroma subsampling depending on the specific subset. This provides greater color fidelity and precision than typical 8-bit 4:2:0 codecs like what can commonly be seen with [AVC](/codecs/video/avc/). ProRes compression uses the [Discrete Cosine Transform](/start-here/terminology/#discrete-cosine-transform-dct) and intra-frame compression techniques; there is no inter-frame compression applied, meaning redundancy between frames isn't utilized to increase compression efficiency; this keeps decoding light, and simplifies the codec. ProRes is able to achieve very high quality video with relatively low data rates around 100-400 MB/s depending on the resolution and specific ProRes variant. ## Format Breakdown There are multiple variants of ProRes that offer different combinations of image quality and compression ratios: Format | Chroma Subsampling | Supported Bit Depth(s) --- | :---: | ---: [ProRes RAW](https://www.apple.com/final-cut-pro/docs/Apple_ProRes_RAW.pdf) | N/A | 12-bit ProRes 4444 XQ | 4:4:4 | 10-bit, 12-bit\* ProRes 4444 | 4:4:4 | 10-bit, 12-bit\* ProRes HQ | 4:2:2 | 10-bit ProRes | 4:2:2 | 10-bit ProRes LT | 4:2:2 | 10-bit ProRes Proxy | 4:2:2 | 10-bit \**ProRes 4444 XQ and 4444 support 12-bit color depth, but the lossless alpha channel is 16-bit if included.* The higher data rate variants like 4444 XQ and HQ prioritize maximum fidelity at the expense of larger file sizes. The lower data rate variants trade off some quality for much smaller file sizes suitable for editing where storage space is constrained. ## Usage Apple ProRes Raw is available in LUMIX cameras & the Sony Alpha FX3 via Atomos Ninja V. Both cameras are capable of recording 12-bit uncompressed RAW video in the ProRes RAW format. ProRes has become widely adopted in professional video post-production workflows due to its quality, performance, metadata handling, and tight integration with Apple's Final Cut Pro editing software & ecosystem. ProRes is supported by most major non-linear editing systems beyond just Final Cut Pro, including Adobe Premiere Pro & BlackMagic's DaVinci Resolve. This cross-platform compatibility makes it a convenient codec for exchange between different video editing applications. While designed primarily as an editing codec, Apple has also implemented hardware acceleration for ProRes encode & decode in recent Mac devices via Apple Silicon starting with the M2 chip. This hardware acceleration enables higher performance for tasks like playback, transcoding, and export compared to software-only implementations. --- # Theora Canonical URL: https://encode.wiki/codecs/video/theora/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: fa3edbaf4f5262ecbfe033d5787c602235e1f8e54a8b4566745417352f215990 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Theora is a legacy video codec first released in 2004. It is derived from VP3, part of the same family of codecs that formed [VP8](/codecs/video/vp8/), [VP9](/codecs/video/vp9/) and [AV1](/codecs/video/av1/). ## Encoding Theora is frequently contained within a ogg (`.ogg` or `.ogv`) container with [vorbis](/codecs/audio/vorbis/) audio. To be filled. --- # UT Video Canonical URL: https://encode.wiki/codecs/video/utvideo/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 5fa7aa79b8452c09dc92245dd8f33f8500bccba69ad4f740ac0a12fe686ab556 # UT Video Codec Suite > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! UT Video Codec Suite is a fast, lossless video codec, developed by Takeshi Umezawa (梅澤 威志, Umezawa Takeshi) and released under the free GNU General Public License. The algorithm of UT video is based on the Huffman code. UT Video was developed as an alternative to HuffYUV, in order to achieve better compression. It can handle color spaces such as YUV422 (ULY2), RGB (ULRG), RGBA (ULRA) and, most recently, YUV420 (ULY0). It has both x86 and x64 builds. Due to its multithreading support, this codec is also capable of encoding HDTV material in real time. The codec requires support for the SSE2 instruction set because it is heavily used for speed optimizations. There are various predction modes, which can be used via [FFmpeg](/tools/utilities/ffmpeg/): - no prediction employed - left neighbour prediction (continuous for the whole slice) - gradient prediction - median prediction You can use FFmpeg to encode utvideo as follows: `ffmpeg -i [input] -c:v utvideo -pred [0,1,2,3] [output]` *References: [Wikipedia](https://en.wikipedia.org/wiki/Ut_Video_Codec_Suite)* --- # VC-1 Canonical URL: https://encode.wiki/codecs/video/vc-1/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 188ee1ef3fd8ba3d5590fecc699af65866467a9b8de71ed0707af94317bd5e45 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! VC-1 is video codec created by Microsoft and released in 2006. It largely aimed to compete with [AVC](/codecs/video/avc/). ## Encoding VC-1 can be contained in `.mp4`, `.mkv`, and `.avi` containers. To be filled ## Decoding VC-1 can be decoded by [FFmpeg](/tools/utilities/ffmpeg/), [VLC](/tools/video-players/), [MPV](/tools/video-players/), and any device that supports blu-ray. --- # VP8 Canonical URL: https://encode.wiki/codecs/video/vp8/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 679c582e1559370d5fda7cfde9b3d90d20ed7562db611012b5e8a89d79a26eb1 VP8 is a video compression format developed by On2 Technologies released in 2008. It was later released as a royalty free codec in 2010 by Google. Its efficiency is competitive with [AVC](/codecs/video/avc/). VP8 was a significant player in the royalty-free codec space and was designed primarily around web video delivery. It is the precursor to [VP9](/codecs/video/vp9/) & [AV1](/codecs/video/av1/), which both further improve video compression efficiency. VP8 has faced criticism for having a messy specification that many considered to be incomplete. It also serves as the basis for the [WebP](/codecs/images/webp/) image format. ## Design Assumptions VP8's design is built upon several core assumptions tailored for web video. - It is designed to operate efficiently within a quality range from "watchable video" (around 30dB [PSNR](/reference/metrics/psnr/)) to "visually lossless" (around 45dB PSNR), anticipating limited network bandwidth. - VP8 was designed to support efficient implementations across a wide breadth of client devices, from low-power mobile/embedded systems to powerful desktops. - VP8 is optimized to handle common web video image formats; 4:2:0 color sampling, 8-bit per channel color depth, progressive scan (not interlaced), and image dimensions up to 16383x16383 pixels. Internally, lossy VP8 works exclusively in 8-bit YUV 4:2:0 (chroma is 1/4 the resolution of luma). The overall architecture involves decomposing video frames into 16x16 luma (Y) and 8x8 chroma (U, V) macroblocks. These macroblocks can be further divided into 4x4 subblocks. ## Technical Features VP8 incorporates features that contribute to its compression efficiency and low computational complexity. ### Frequency Transforms & Adaptive Quantization VP8 utilizes transform coding to process the residual signal after intra or inter predictions. The [Discrete Cosine Transform](/start-here/terminology/#discrete-cosine-transform-dct) is applied to all luma and chroma residual signals. The DCT is used due to its high VP8 defines a 4x4 inverse 2D DCT process as part of its bitstream format and decoding. For macroblocks using 16x16 luma prediction modes, the DC coefficients from the 16 4x4 luma blocks within the macroblock are used to create another 4x4 block that undergoes a 4x4 Walsh-Hadamard Transform (WHT) to reduce redundancy among the DC coefficients in the 16x16 luma area. VP8 defines 128 quantization levels within its operating quality range (~30dB to ~45dB). For each video frame, different quantization levels can be applied to six frequency components: - 1st order luma DC - 1st order luma AC - 2nd order luma DC - 2nd order luma AC - chroma DC - chroma AC Additionally, VP8 includes a region-dependent quantization scheme, allowing macroblocks within a frame to be classified into four different segments, each with its own set of quantization parameters. All calculations in the transform, quantization, dequantization, and inverse transform pipeline can be performed using 16-bit operations. ### Flexible Reference Frames VP8 uses three types of reference frames for inter-prediction: - **Last Frame**: The reconstructed frame immediately preceding the frame currently displayed. - **Golden Frame**: A frame of decompressed data from an arbitrarily distant point in the past. Encoders can use the Golden Frame to improve coding efficiency by maintaining a copy of the background when foreground objects move, allowing occluded regions to be easily reconstructed. Golden frames also help with error resiliency in real-time video scenarios, like video conferencing. - **Alternate Reference Frame (Altref Frame)**: This frame is decoded, but is not necessarily displayed to the user. Its primary purpose is to serve as a reference to improve inter-prediction for other coded frames. Encoders can construct an Altref frame from multiple source frames or using macroblocks from various video frames, providing flexibility for improved compression. For example, it can be used for noise-reduced prediction by creating a "noise-free" reference frame through temporal or spatial filtering. The intelligent use of Golden and Altref frames can compensate for the lack of B-frames (as seen in H.264) by allowing encoders to leverage information from "future" frames without requiring frame reordering in the decoder. ### Intra Prediction VP8 employs two main classes of prediction modes, *intra* and *inter* prediction. Intra prediction Uses data from within the current video frame. It applies to 4x4 luma, 16x16 luma, and 8x8 chroma blocks. Intra prediction modes for 8x8 chroma and 16x16 luma blocks include: - `H_PRED` (horizontal prediction): Fills columns with copies of the left column. - `V_PRED` (vertical prediction): Fills rows with copies of the above row. - `DC_PRED` (DC prediction): Fills the block with a single value, the average of pixels from the above row and left column. For chroma blocks, specific averaging rules apply for edge macroblocks (top row, left column, or top-left corner). - `TM_PRED` (TrueMotion prediction): Unique to VP8, it uses the pixel above and to the left of the block (C), along with pixels from the above row (A) and left column (L). It propagates horizontal differences from A and vertical differences from L to form the prediction block, using the formula `Xij = Li + Aj - C`. TM_PRED is a frequently used mode, typically for 20% to 45% of all intra-coded blocks. For 4x4 luma blocks, six additional directional intra modes are available. As is the case in other video codecs, keyframes rely solely on intra prediction modes. ### Inter Prediction Inter prediction uses data from previously encoded reference frames. A block is constructed using a motion vector to copy a block from one of the three reference frames (last, golden, altref). VP8 employs efficient motion vector coding by reusing vectors from neighboring macroblocks using modes like `NEAREST` and `NEAR`. The `SPLITMV` mode allows flexible coding of arbitrary block patterns within a macroblock by partitioning it into sub-macroblock patterns, each with its own motion vector. This allows a 16x16 macroblock to be partitioned into up to 16 4x4 blocks, each with a (potentially) new motion vector or inheriting from neighbors. Motion compensation in VP8 uses quarter-pixel accurate motion vectors for luma pixels and up to one-eighth pixel accurate motion vectors for chroma pixels. It features a single-stage interpolation process and a set of high-performance six-tap interpolation filters for luma, and four-tap bicubic filters for chroma. This achieves optimal frequency response with high computational efficiency. Chroma motion vectors are calculated by averaging the motion vectors of the four corresponding luma subblocks. Motion vectors are limited to a range of -4096 to +4095 full pixels. ### In-Loop Deblocking Filtering VP8 includes a highly adaptive in-loop deblocking filter that reduces blocking artifacts introduced by the quantization of DCT coefficients. - The filter type (normal or simple) and strength can be adjusted based on different prediction modes and reference frame types. - It supports implicit segmentation, where different filter strengths can be applied to various parts of the image. For instance, stronger filtering might be applied to intra-coded blocks, while inter-coded blocks using the Golden Frame with a (0,0) motion vector might use a weaker filter. - The choice of loop filter strengths is adjustable on a frame-by-frame basis. Loop filter strength can be adjusted per segment (within a frame) based on region-adaptive schemes. A significant innovation is the ability to skip loop filtering entirely for edges between subblocks internal to a macroblock if the macroblock's coding mode is neither `B_PRED` nor `SPLITMV` and it has no DCT coefficients coded. This significantly reduces loop-filtering complexity. Loop filtering is applied to the entire frame after macroblock reconstruction and its results are used in the prediction of subsequent frames. ### Entropy Coding > **Note — Lossless Compression.** > > This section describes the VP8 bitstream's lossless compression step, where all of a lossy bitstream's data values are compressed losslessly. For background on this subject, it is recommended to read the [Lossless Compression](/start-here/lossless/) wiki entry (particularly the section on arithmetic coding). Except for a few header bits, the majority of compressed VP8 data values are coded using a boolean arithmetic coder. This coder losslessly compresses sequences of boolean values where the probabilities of 0 or 1 can be accurately estimated. This approach provides significant data rate reduction compared to simpler methods like Huffman coding. Most symbol values are binarized into a series of boolean values using a tree scheme, where a binary tree is created for a set of symbols. Each non-leaf node in the tree has a probability assigned based on the likelihood of taking a branch. This consistent encoding/decoding style for various bitstream values (e.g., macroblock modes, motion vectors, quantized coefficients) improves module reusability in hardware and software implementations. VP8 models entropy coding contexts using conditional probability distributions for macroblock modes, motion vectors, and quantized transform coefficients. These probability distributions remain stable within a frame and are updated on a per-frame basis. This design achieves lower decoder implementation complexity and allows better error recovery compared to H.264's context-based binary arithmetic coding. VP8's bitstream separates compressed data into two categories: - Macroblock coding modes & motion vectors - Quantized transform coefficients Critically, VP8 allows transform coefficients to be packed into multiple partitions (up to eight), enabling parallel processing in decoders on multi-core processors. For example, in `FOUR_TOKEN_PARTITION` mode, coefficients from macroblock rows 0, 4, 8, etc., are packed into the first partition, rows 1, 5, 9, etc., into the second, and so on. This design significantly improves decoder performance on multi-core systems with minimal impact on compression efficiency or single-core performance. ## Overall Decoding Process A VP8 decoder maintains four YUV frame buffers: the current frame being reconstructed, the immediately previous frame, the most recent golden frame, and the most recent altref frame. The decoding process generally involves: 1. Decoding the frame header, which provides context for the per-macroblock data. 2. Processing macroblock data in raster-scan order. This includes prediction data (modes/motion vectors) and DCT/WHT coefficients of the residual signal. 3. Each macroblock is predicted using intra-prediction (from the current frame) or inter-prediction (from a reference frame). 4. The residue DCT/WHT signal is decoded, dequantized, reverse-transformed, and added to the prediction buffer to produce the reconstructed macroblock. 5. After all macroblocks are processed, the adaptive in-loop deblocking filter is applied to the entire frame to reduce artifacts. 6. Finally, reference frame buffers are updated (e.g., current frame replacing golden or altref frame) and prepared for the next frame. ## VP8 Today Experimental results from initial publishing showed that VP8 encoded files were consistently faster to decode (averaging around 30% faster) than H.264 High Profile files at similar bitrates across various hardware platforms. In terms of encoding quality, VP8 achieved competitive results against the best H.264/AVC encoders at the time in its designed operating range for web video. In practice, reactions to VP8's initial performance were mixed. Moscow State University's initial numbers for the [libvpx](/tools/software-encoders/vpxenc/) encoder (shown below) were lackluster, and the performance claims that accompanied VP8's initial release were not appearing to hold up. ![VP8 RD on Pixar's "Up"](https://raw.githubusercontent.com/av1-community-contributors/images/refs/heads/main/vp8_rd_up.webp) *VP8 vs [x264](/tools/software-encoders/x264/), Xvid on Pixar's "Up"* In her "Diary Of An x264 Developer" blog, Fiona Glaser (co-author of [FFmpeg](/tools/utilities/ffmpeg/)'s native highly optimized VP8 decoder) discussed the positive and negative aspects of the VP8 video format. She acknowledged that VP8 doesn't have to be the best in compression to be useful while the current encoder issues are likely related to the encoder implementation's maturity, but she criticized Google for releasing an incomplete and inconsistent “bitstream guide” instead of a proper spec for the codec. The actual encoder/decoder implementation in libvpx differed from the bitstream specification, so the "bitstream guide" nomenclature took hold in what appeared to be a means of avoiding fixing the core issue. Fiona also praised VP8’s `TM_PRED` intra prediction mode and tree-based arithmetic coding, but criticized its slow deblocking filter and suboptimal inverse transform ordering. Today, [VP9](/codecs/video/vp9/) and to some extent [AV1](/codecs/video/av1/) have largely replaced VP8 as the web's most prominent royalty-free video compression standards. While VP8 may have had a rocky development cycle as well as a lackluster reference encoder, the effects of VP8's introduction are still felt today. WebP is still a widely adopted web image standard, and VP8 set important precedents for web video that helped push the industry forward (e.g. no interlaced video support; Fiona described interlacing as "the scourge of H.264"). Finally, royalty-free video standardization efforts have seen progressively fewer issues with implementation-to-spec parity, helping guide hardware vendors toward successful implementations more effectively. *Sources:* - *[Fiona Glaser's blog](https://web.archive.org/web/20150301015756/http://x264dev.multimedia.cx/archives/486)* - *VP8 Data Format and Decoding Guide – [RFC6386](https://datatracker.ietf.org/doc/rfc6386/)* - *[Technical overview of VP8, an open source video codec for the web ](https://ieeexplore.ieee.org/abstract/document/6012227)* --- # VP9 Canonical URL: https://encode.wiki/codecs/video/vp9/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 1b4bedd4ee9d2c1bdf4d119ebd49b05cd96cb58b32128f244397934c2dcbf8e1 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! VP9, famous for being YouTube's codec of choice for many years, is a royalty free video compression format that competes with [HEVC](/codecs/video/hevc/) and [AVC](/codecs/video/avc/). It is slightly less efficient than HEVC in terms of visual quality, but VP9 is a computationally much simpler codec and therefore is very easy on system resources when decoding. If you're interested in VP9 encoding, please see the [vpxenc](/tools/software-encoders/vpxenc/) or [SVT-VP9](/tools/software-encoders/svt-vp9/) sections. --- # VVC / H.266 Canonical URL: https://encode.wiki/codecs/video/vvc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 21666f73620d10c2ff77d6bd0164890db32395ba62f37162dabd42dbfbe720b9 # H.266 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! H.266, or VVC (Versatile Video Coding), is a codec standardized in 2020 by the Joint Video Experts Team (JVET). It succeeds [H.265](/codecs/video/hevc/), and claims to be 40% more efficient. In practice, it is currently about as efficient as [AV1](/codecs/video/av1/) when using the [VVenC](/tools/software-encoders/vvenc/) encoder, although it is inherently a more complex format which means it will be more difficult to decode. It is encumbered by royalties. [FFmpeg](/tools/utilities/ffmpeg/) 7.0 released in April 2024 comes with native VVC decoder. --- # Encoding guides Canonical URL: https://encode.wiki/guides/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: e53e210aed566c337aa80b6b64936860e7e32ecce3a07623be6700a7a25de537 Guides turn encoding concepts into complete workflows. Each command-oriented guide declares its tested versions and assumptions, explains what success looks like, and includes a validation step. The initial guide set will cover first FFmpeg encodes, AV1 software encoding, quality controls, color and HDR metadata, audio and subtitles, and objective plus visual comparison. --- # Handle audio and subtitles in the final container Canonical URL: https://encode.wiki/guides/audio-and-subtitles/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 1809d0fc48f5f88b09ac082713bac1fa8bfed4673cb917dd2c92410d8ae4ff32 Automatic stream selection may omit a track you expected or choose a different default. Inspect and map deliberately. ```sh ffprobe -v error -show_streams -of json input.mkv ``` For a Matroska output that copies video and subtitles while encoding every audio stream to Opus: ```sh ffmpeg -i input.mkv \ -map 0:v -map '0:a?' -map '0:s?' \ -c:v copy -c:a libopus -b:a 128k -c:s copy \ output.mkv ``` This only works when the target container accepts every selected codec. Streamcopy avoids quality loss but cannot apply filters and does not make an incompatible stream compatible. Image-based subtitles, styled text subtitles, attachments, chapters, and fonts need separate checks. For MP4, text subtitles commonly need conversion to `mov_text`, and some source subtitle types cannot be converted meaningfully: ```sh ffmpeg -i input.mkv \ -map 0:v:0 -map '0:a?' -map '0:s?' \ -c:v copy -c:a aac -b:a 192k -c:s mov_text \ output.mp4 ``` ## Validate Use `ffprobe` to check stream count, codec, language tags, titles, and dispositions. Then play each audio and subtitle track, check synchronization at several timestamps, and verify that default/forced behavior matches the target platform. ## Primary references - [FFmpeg manual stream selection](https://ffmpeg.org/ffmpeg.html#Manual-stream-selection) - [FFmpeg stream specifiers](https://ffmpeg.org/ffmpeg.html#Stream-specifiers-1) - [FFmpeg subtitle options](https://ffmpeg.org/ffmpeg.html#Advanced-Subtitle-options) --- # Encode AV1 with SVT-AV1 and FFmpeg Canonical URL: https://encode.wiki/guides/av1-with-svt-av1/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 0c1ba7d9f7300f53393b37ccbcabf9b59d70277c28758edee34f51f42a342103 Your FFmpeg build must be configured with SVT-AV1. Confirm the wrapper and its locally supported options first: ```sh ffmpeg -hide_banner -h encoder=libsvtav1 ``` ## Encode a representative sample ```sh ffmpeg -ss 00:05:00 -i input.mkv -t 30 \ -map 0:v:0 -map '0:a?' \ -c:v libsvtav1 -preset 6 -crf 30 -pix_fmt yuv420p10le \ -c:a libopus -b:a 128k \ sample-av1.mkv ``` This is a test starting point, not a universal recommendation. Higher SVT-AV1 preset numbers trade compression efficiency for speed; higher CRF values trade quality for smaller output. Ten-bit output can improve compression behavior but requires compatible playback. Confirm the source and target color requirements before changing pixel format. Use `-svtav1-params` only for an option exposed by the installed SVT-AV1 version. It accepts colon-separated `key=value` pairs. Keep the command and exact encoder version with test results. ## Validate ```sh ffmpeg -v error -i sample-av1.mkv -f null - ffprobe -v error -select_streams v:0 \ -show_entries stream=codec_name,profile,width,height,pix_fmt,color_range,color_space,color_transfer,color_primaries \ -of json sample-av1.mkv ``` Watch the entire sample on representative software and hardware decoders. Check gradients, motion, grain, text, dark scenes, seeking, and audio synchronization before running a long encode. ## Primary references - [FFmpeg libsvtav1 wrapper](https://ffmpeg.org/ffmpeg-codecs.html#libsvtav1) - [SVT-AV1 4.0.0 parameters](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/v4.0.0/Docs/Parameters.md) - [SVT-AV1 FFmpeg guide](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/Ffmpeg.md) --- # Choose a codec and encoder Canonical URL: https://encode.wiki/guides/choose-a-codec/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 0c608328493186b8a4cca510e0f789f3c590db6bbebca933268e29d545e30c32 Start with the playback, editing, delivery, and licensing constraints. A codec with better compression is not useful when the target device cannot decode it or the required encoder is unavailable. ## Decision order 1. List every required player, device, editor, and delivery platform. 2. Identify the codecs and profiles all of them accept. 3. Decide whether the job needs lossless, visually lossless, or lossy output. 4. Decide whether size, encode time, decode cost, or broad compatibility has priority. 5. Choose an encoder implementation and test a representative short segment. 6. Choose a container that accepts every video, audio, and subtitle stream you need. | Typical goal | Starting point | Check before committing | | ------------------------- | -------------------------------------------------- | ----------------------------------------------------- | | Broad consumer playback | AVC/H.264 with AAC in MP4 | Profile, level, pixel format, and device limits | | Efficient modern delivery | AV1, VP9, or HEVC | Hardware decode coverage and platform ingest rules | | Editing/intermediate | ProRes, DNxHR, or lossless formats | Editor support, storage, chroma, and bit depth | | Archival preservation | Lossless video and audio in a documented container | Validation, checksums, metadata, and recovery tooling | List the encoders in your FFmpeg build instead of assuming a package includes one: ```sh ffmpeg -hide_banner -encoders ffmpeg -hide_banner -h encoder=libsvtav1 ``` ## Validation Encode a short segment containing motion, texture, gradients, dark regions, and audio transitions. Inspect it on a real target device, then use `ffprobe` to confirm the output streams and metadata: ```sh ffprobe -v error -show_format -show_streams -of json output.mkv ``` Do not compare CRF or preset numbers across different encoders as if they represented the same quality or speed. Compare the resulting media under the same source, scaling, pixel-format, and playback conditions. ## Primary references - [FFmpeg codec documentation](https://ffmpeg.org/ffmpeg-codecs.html) - [FFmpeg stream selection](https://ffmpeg.org/ffmpeg.html#Stream-selection) - [ffprobe documentation](https://ffmpeg.org/ffprobe.html) --- # Compare encoding quality with metrics and visual checks Canonical URL: https://encode.wiki/guides/compare-encoding-quality/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 958594615e75adc5e3bbacfc2dabc0c2fce0111e6b8fdc7c435cf2c4aec097ac Metrics are useful only when the reference and distorted streams represent the same frames in the same geometry and color domain. First verify frame rate, timestamps, duration, crop, scale, pixel format, and color conversion. ## Calculate PSNR or SSIM The first input below is the distorted encode and the second is the reference: ```sh ffmpeg -i distorted.mkv -i reference.mkv \ -lavfi '[0:v][1:v]psnr=stats_file=psnr.log' \ -f null - ``` ```sh ffmpeg -i distorted.mkv -i reference.mkv \ -lavfi '[0:v][1:v]ssim=stats_file=ssim.log' \ -f null - ``` An FFmpeg build with `libvmaf` can also calculate VMAF and write structured output: ```sh ffmpeg -i distorted.mkv -i reference.mkv \ -lavfi "[0:v][1:v]libvmaf=log_fmt=json:log_path=vmaf.json" \ -f null - ``` ## Interpret the results - Preserve per-frame logs, not just an average. - Inspect the lowest-scoring and visually important frames. - Compare multiple metrics that fail in different ways. - Perform blind or controlled visual comparisons on representative displays. - Record the exact command, build versions, source checksum, and any preprocessing. A higher aggregate score does not automatically mean that viewers will prefer an encode, and a small score difference may not be meaningful. Use metrics to find tradeoffs and outliers, then confirm them visually. ## Primary references - [FFmpeg PSNR filter](https://ffmpeg.org/ffmpeg-filters.html#psnr) - [FFmpeg SSIM filter](https://ffmpeg.org/ffmpeg-filters.html#ssim) - [FFmpeg libvmaf filter](https://ffmpeg.org/ffmpeg-filters.html#libvmaf) - [Encode Wiki metrics reference](/reference/metrics/vmaf/) --- # Make and validate a first FFmpeg video encode Canonical URL: https://encode.wiki/guides/first-ffmpeg-encode/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 84498eaf1ff3e5a1795561416de1b2a842bcb40cb90ca2562fa9c52d80c88570 This workflow creates a short test in Matroska. It maps the first video stream, optionally maps audio, encodes video with x264, and encodes audio with Opus. Your FFmpeg build must include `libx264` and `libopus`. ## Inspect the input ```sh ffprobe -v error -show_format -show_streams -of json input.mkv ffmpeg -hide_banner -h encoder=libx264 ffmpeg -hide_banner -h encoder=libopus ``` Confirm the source stream indexes, frame rate, pixel format, resolution, color metadata, audio layout, and subtitles before encoding. ## Encode a 30-second sample ```sh ffmpeg -ss 00:05:00 -i input.mkv -t 30 \ -map 0:v:0 -map '0:a?' \ -c:v libx264 -preset slow -crf 20 -pix_fmt yuv420p \ -c:a libopus -b:a 128k \ sample.mkv ``` `-map` makes stream selection explicit. The `?` makes the audio mapping optional. `-crf` controls the x264 quality/size tradeoff, while `-preset` controls its speed/compression tradeoff. The `yuv420p` pixel format is a compatibility choice and may be wrong for high-bit-depth or HDR sources. ## Validate ```sh ffmpeg -v error -i sample.mkv -f null - ffprobe -v error -show_format -show_streams -of json sample.mkv ``` The decode check should print no errors. Confirm the expected stream count, codecs, duration, resolution, pixel format, and color fields. Finally watch the whole sample on a target player and seek through it. ## Primary references - [FFmpeg transcoding and streamcopy](https://ffmpeg.org/ffmpeg.html#Transcoding) - [FFmpeg manual stream selection](https://ffmpeg.org/ffmpeg.html#Manual-stream-selection) - [ffprobe documentation](https://ffmpeg.org/ffprobe.html) --- # Preserve color and HDR metadata safely Canonical URL: https://encode.wiki/guides/preserve-color-and-hdr/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 957ee2c267d9d48a186454359d5158a195abd588a039ae91a9e48ad012b8cf30 Color errors can survive a successful encode. Do not infer transfer, primaries, matrix, range, chroma location, mastering display, or content-light metadata from resolution or filename. ## Inspect first ```sh ffprobe -v error -select_streams v:0 \ -show_entries stream=pix_fmt,color_range,color_space,color_transfer,color_primaries,chroma_location,side_data_list \ -of json input.mkv ``` If essential fields are absent or contradict the mastering information, stop and resolve the source facts. Assigning metadata does not convert pixels. Converting transfer functions, primaries, range, or chroma requires an intentional color pipeline and a filter that supports the source and target. ## Encode without silently discarding the question - Keep a record of the probed source fields. - Choose an encoder, profile, bit depth, and pixel format capable of the target signal. - Pass explicit color options only when they describe the actual encoded pixels. - Handle HDR10 mastering/content-light metadata and dynamic HDR metadata as separate requirements. - Avoid a casual `-pix_fmt yuv420p` on high-bit-depth or HDR input. For known BT.709 limited-range SDR pixels, explicit video signaling may look like: ```sh -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range tv ``` Those flags are not a generic fix and must not be pasted onto unknown or HDR material. ## Validate Probe the output with the same command, compare it to the intended target, decode the full output for errors, and inspect it through a color-managed playback path. A metadata match alone does not prove the pixel conversion is correct. ## Primary references - [FFmpeg scaler and color options](https://ffmpeg.org/ffmpeg-scaler.html) - [ffprobe stream and side-data output](https://ffmpeg.org/ffprobe.html#Main-options) - [Encode Wiki colorimetry reference](/reference/colorimetry/intro/) --- # Choose quality controls, bitrate, presets, and passes Canonical URL: https://encode.wiki/guides/quality-controls/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 3dcb090f3faf2e184489a67b8ba809bcaf8b874de00221e68e36dd6ab3452c42 Rate control answers a product constraint. Choose it only after deciding which outcome must remain stable. | Requirement | Appropriate starting mode | Main tradeoff | | ----------------------------------------------- | -------------------------------- | --------------------------------------- | | Similar perceptual quality across varied scenes | CRF/constant-quality | Output size is unknown | | Delivery or storage target over a whole program | VBR, often multi-pass | Quality varies to meet the target | | Tight instantaneous transmission bound | Constrained quality, VBV, or CBR | May sacrifice quality in complex scenes | | Diagnostic experiment | Constant quantizer | Usually not a delivery recommendation | ## CRF and presets CRF values are encoder-specific. A value of 30 in SVT-AV1 does not mean the same thing as 30 in x264. Likewise, preset scales differ. Within one encoder, test several values on the same source segment and record output size, encode time, objective metrics, and visual findings. ## One pass or multiple passes For a quality-targeted encode, start with one pass unless the encoder's current documentation gives a specific reason to use more. For a required average bitrate or file size, multi-pass VBR can allocate bits using information gathered across the full program. SVT-AV1 documents multi-pass as most useful for VBR targets and less critical for ordinary CRF work. ## Reproducible comparison Keep all of these fixed while testing one control: - exact source segment and stream mapping; - scaling, cropping, filters, frame rate, pixel format, and color pipeline; - encoder build/version and thread constraints; - audio and subtitle handling; and - metric implementation plus viewing conditions. Never keep a setting merely because it won a single aggregate score. Inspect outliers and visually important scenes. ## Primary references - [FFmpeg codec options](https://ffmpeg.org/ffmpeg-codecs.html) - [SVT-AV1 bitrate-control modes](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/CommonQuestions.md#bitrate-control-modes) - [SVT-AV1 multi-pass guidance](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/CommonQuestions.md#multi-pass-encoding) --- # Encode Wiki Canonical URL: https://encode.wiki/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 0567068ae57ad6d46a45a85d2b19a92d872d211a087e22bf100815e41e0d9996 ## Explore Encode Wiki Learn the vocabulary and tradeoffs behind media encoding. Follow practical, testable workflows for real encoding tasks. Compare formats, encoders, filters, players, and utilities. Read experiments, benchmarks, explanations, and project news. Encode Wiki is a new community project seeded from the AV1 Community Contributors' Codec Wiki. Imported material preserves its source and attribution. New and migrated pages will be expanded and reviewed as the project develops. --- # Reference Canonical URL: https://encode.wiki/reference/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 3b193458f316f8cfe83e7c972e57d5000b2a159e1e9f31e0abe70cc2556c82c2 Use the reference section for terminology, colorimetry, quality metrics, command details, FAQs, and links to primary external sources. --- # Color Formats Canonical URL: https://encode.wiki/reference/colorimetry/format/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 046ef8076056c9e4351bb19640c0909261abce34860c653415addf33758471b5 To represent color values, a format is agreed upon. Color formats are made up of three things, the [color model](#color-models)--which includes the [order of the components](#component-order) and sometimes [chroma subsampling](#chroma-subsampling)-- the [bit depth](#bit-depth), and whether it is a [packed or a planar format](#packed-vs-planar). In some cases, [endianness](#endianness) may be important. ## Color Models A color model is a method of representing colors in a video or image using data. Different color models store color and brightness information in different ways. There are many different color models, but this section will cover the models most commonly used for images and video. ### RGB RGB is probably the most well-known color model, and is primarily used in image encoding. RGB consists of three color channels, Red, Green, and Blue, which are then combined to determine the final color of each pixel. Typically, RGB is the final model that a monitor or TV will use to display images, because the pixels on a screen are made up of red, green, and blue LEDs, although it is not commonly used for video encoding because other models can provide better compression. ### YUV YUV, also known as YCbCr, is the most widely used color model for video encoding. It consists of three components: Y aka Luma, which represents luminance or brightness, and two chroma planes, which represent color. Generally a video player will have to convert a YUV video into RGB before it can be rendered, but there are significant compression benefits to using YUV over RGB for video. The most notable reason to use YCbCr is an optimization called chroma subsampling. This means that the chroma components can be encoded at a lower resolution than the luma components, which results in a smaller output file. You can read more about chroma subsampling [further below](#chroma-subsampling). ### Component order The order in which the components in a color model are arranged is simply represented by writing them out. For example, `RGB` for red first, then green, then blue, or `BGR` for blue, green, red. ## Bit depth A bit depth is how many bits are available to store the sample value. There are two main ways to specify the bit depth in a : - bits per component. Here, `RGB888` reads as `RGB color model, with 8 bits for the red component, 8 bits for the green component, and 8 bits for the blue component` and `RGB565` reads as `RGB color model, with 5 bits for the red component, 6 bits for the green component, and 5 bits for the blue component`. - bits per sample. Here, `RGB24` reads as `RGB color model, with 24 bits in total for the red, green, and blue components`. This is ambiguous, because one does not know exactly how many bits are allocated to each component. `RGB565`, `RGB556`, and `RGB655` (even though the latter ones do not make much sense as the eye is most sensitive to green light) all become `RGB16`. ## Packed vs planar Components can be stored either packed, where all components are interleaved (here, `RGB`): ``` Sample number: 1 2 3 4 5 Data: RGB RGB RGB RGB RGB ``` or stored separately for each component: ``` Sample number: 1 2 3 4 5 Data: R R R R R... Data: G G G G G... Data: B B B B B... ``` In planar formats, many operations can be easier to implement, as it is possible to implement the algorithm once and then operate on all planes. On the other hand, packed formats are simpler and often used in hardware.[^vlc-wiki-yuv] ## Endianness Different computer architectures store numbers differently. For more information, visit [the Wikipedia article on endianness](//wikipedia.org/wiki/Endianness). There are two main ways to store numbers with more than 8 bits (1 is the least significant byte and 4 is the most significant byte, here 4 bytes): - Most significant byte first, little endian, `4321`. This is what x86-family processors use. - Least significant byte first, big endian, `1234`. This is what PowerPC-family processors use. This can be important for color formats, as some computers might store it in their native endianness. VapourSynth doesn't seem to care about endianness, but FFmpeg does. For example, `RGB565` might store its two bytes in `12` or `21` order, and if they are read in the wrong order, it will produce garbage. ## Chroma subsampling In [Y'CbCr](#yuv) signals, there are three widely used variants of chroma subsampling: - 4:2:0 which has half the vertical and horizontal chroma resolution - 4:2:2 which has half the horizontal chroma resolution but full vertical resolution - 4:4:4 which has full chroma resolution (no subsampling) 4:2:2 is not particularly useful over the other options, so this guide will focus on 4:2:0 and 4:4:4. 4:2:0 is the most commonly used format for videos. Nearly every DVD, blu-ray, camera recording, etc. uses 4:2:0 subsampling. This is because, in the majority of cases, human eyes do not notice the reduction in chroma resolution. There is very little benefit to using 4:4:4 in the average case. However, there are some exceptions. The most notable is screen recordings. Things like text overlays, video game UI overlays, etc. have very fine, color-dependent detail that can be destroyed by chroma subsampling and result in an aliased look to the video. Therefore, it is recommended to use 4:4:4 subsampling when recording your screen, and 4:2:0 subsampling in most other cases. ## Common formats | VS name | FFmpeg name | Meaning | | ----------- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | | `GRAY8` | `gray8` | Brightness only, 8 bits, packed | | `GRAY16` | `gray16le`, `gray16be` (the suffix specifies the [endianness](#endianness)) | Brightness only, 16 bits | | `RGB888` | `rgb24` | red, green, blue, 8 bits per component | | `YUV420P8` | `yuv420p` | luma, chroma blue, chroma red, 8 bits per component, planar, 4:2:0 [subsampling](#chroma-subsampling) | | `YUV422P8` | `yuv422p` | luma, chroma blue, chroma red, 8 bits per component, planar, 4:2:2 subsampling | | `YUV444P8` | `yuv444p` | luma, chroma blue, chroma red, 8 bits per component, planar, no subsampling | | `YUV420P10` | `yuv420p10le`, `yuv420p10le` | luma, chroma blue, chroma red, 10 bits per component, planar, 4:2:0 subsampling | | `YUV422P10` | `yuv422p10le`, `yuv422p10le` | luma, chroma blue, chroma red, 10 bits per component, planar, 4:2:2 subsampling | | `YUV444P10` | `yuv444p10le`, `yuv444p10le` | luma, chroma blue, chroma red, 10 bits per component, planar, no subsampling | ## References [^vlc-wiki-yuv]: [YUV - VideoLAN Wiki](https://wiki.videolan.org/YUV/#Packed_formats) --- # Intro Canonical URL: https://encode.wiki/reference/colorimetry/intro/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: e112ead1e99e599f2c9fce091ff577de2a3395c656f8aad87e011f7307eeef35 There are many aspects which determine how the color information for a video is stored and how it is rendered. As technology has improved, new standards developed, and with new techologies such as HDR, new standards continue to develop. However, the result is that it can be confusing to know which color settings to use for a given video. Some properties such as the color format must be set. However, properties such as color range, primaries, matrix coefficients, and transfer function are optional. It is always best practice to set these when you are encoding a video, because if they are not set, the player must make a guess as to what the correct settings are. If it guesses incorrectly, this can lead to the colors of the video being different from what was intended. --- # Matrix Coefficients Canonical URL: https://encode.wiki/reference/colorimetry/matrix/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 4f736b8caf19e0558555e4849edb5b1cd97ba2959c599c3d0e0e16044415deaa Matrix coefficients represent the multiplication matrix that is used when converting from YUV to RGB. As with primaries, the integer values are defined within universal specifications, and as such they will be the same across all encoding and playback tools. The following values are available: ### 0: Identity Specifies that the identity matrix should be used, i.e. this data is already in an RGB-compatible colorspace. This matrix coefficient setting is used in the following standards: - GBR (often referred to as RGB) - YZX (often referred to as XYZ) - IEC 61966-2-1 sRGB - SMPTE ST 428-1 (2019) ### 1: BT.709 BT.709 is the standard used for modern high-definition video, and is a safe default assumption. This matrix coefficient setting is used in the following standards: - Rec. ITU-R BT.709-6 - Rec. ITU-R BT.1361-0 conventional colour gamut system and extended colour gamut system (historical) - IEC 61966-2-4 xvYCC709 - SMPTE RP 177 (1993) Annex B ### 2: Unspecified This value indicates that no color matrix is set for the video, and the player must decide which value to use. mpv will use the following heuristics in this case: ``` if width >= 1280 || height > 576 { "BT.709" } else { "SMPTE 170M" } ``` ### 4: BT.470M BT.470M is a standard that was used in analog television systems in the United States. ### 5: BT.470BG BT.470BG is a standard that was used for European (PAL) television systems and DVDs. This matrix coefficient setting is used in the following standards: - Rec. ITU-R BT.470-6 System B, G (historical) - Rec. ITU-R BT.601-7 625 - Rec. ITU-R BT.1358-0 625 (historical) - Rec. ITU-R BT.1700-0 625 PAL and 625 SECAM - IEC 61966-2-1 sYCC - IEC 61966-2-4 xvYCC601 ### 6: SMPTE 170M SMPTE 170M is a stanrard that was used for NTSC television systems and DVDs. Its matrix coefficients are equivalent to BT.470BG. This matrix coefficient setting is used in the following standards: - Rec. ITU-R BT.601-7 525 - Rec. ITU-R BT.1358-1 525 or 625 (historical) - Rec. ITU-R BT.1700-0 NTSC - SMPTE ST 170 (2004) ### 7: SMPTE 240M SMPTE 240M was an interim standard used during the early days of HDTV (1988-1998). ### 8: YCgCo The YCoCg color model, also known as the YCgCo color model, is the color space formed from a simple transformation of an associated RGB color space into a luma value and two chroma values called chrominance green and chrominance orange. ### 9: BT.2020 Non-Constant Luminance BT.2020 is a standard used for ultra-high-definition video, i.e. 4K and higher. It may be used with or without HDR, as HDR is defined by the transfer characteristics. If you do not know if you want non-constant or constant luminance, you probably want non-constant. If you have a video with an unset matrix coefficient, it is safer to assume BT.709 by default. Videos which are BT.2020 should already have their color metadata set. This matrix coefficient setting is used in the following standards: - Rec. ITU-R BT.2020-2 (non-constant luminance) - Rec. ITU-R BT.2100-2 Y′CbCr ### 10: BT.2020 Constant Luminance This is a variant of BT.2020 with constant luminance values, represented using the YcCbcCrc colorspace. You probably want the non-constant luminance variant instead, unless you know you want this one. ### 11: SMPTE 2085 SMPTE 2085 is a standard used with HDR signals in the XYZ colorspace. I've never actually seen it used in the wild. ### 12: Chromaticity-Derived Non-Constant Luminance I'm not really sure when you would use this. ### 13: Chromaticity-Derived Constant Luminance I'm not really sure when you would use this. ### 14: ICtCp ICtCp is an alternative colorspace developed for use with HDR and wide gamut video, by Dolby because they love doing extra stuff like this. I've never actually seen it used in the wild. --- # Color Primaries Canonical URL: https://encode.wiki/reference/colorimetry/primaries/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: d9ca867b94370f87abb5c53ec9b7efafc67ada1727e38b67ae1448b50c163f73 This section details the first of three settings that are important for retaining accurate color when encoding videos, those settings being primaries, color matrix, and transfer characteristics. Color primaries are used to indicate the correct coordinates for the red, blue, and green colors. There are historical reasons for [why so many standards exist](https://xkcd.com/927/), and this guide will not go in depth into history lessons, but will explain what primaries are available and when to use each one. Note that for primaries, matrices, and transfer, you can view the values that are set on a video using a tool like [MediaInfo](https://mediaarea.net/en/MediaInfo). If there are no values set, the player will need to guess which values to use. A safe default assumption for most modern videos is BT.709, although this may vary depending on source and resolution for the video. It is strongly recommended to set the correct values when encoding. Each setting has at least one name and exactly one integer value representing it--most encoder softwares will accept one or more of the names, but some tooling such as Vapoursynth and MKVToolnix accepts the integer values instead. The integer values are defined within universal specifications, and as such they will be the same across all encoding and playback tools. ### 1: BT.709 BT.709 is the standard used for modern high-definition video, and is a safe default assumption. This color primary setting is used in the following standards: - Rec. ITU-R BT.709-6 - Rec. ITU-R BT.1361-0 conventional colour gamut system and extended colour gamut system (historical) - IEC 61966-2-1 sRGB or sYCC - IEC 61966-2-4 - Society of Motion Picture and Television Engineers (SMPTE) RP 177 (1993) Annex B ### 2: Unspecified This value indicates that no color primary is set for the video, and the player must decide which value to use. mpv will use the following heuristics in this case: ``` if matrix == "BT.2020" { "BT.2020" } else if matrix == "BT.709" { "BT.709" } else if width >= 1280 || height > 576 { "BT.709" } else if height == 576 { "BT.470BG" } else if height == 480 || height == 488 { "SMPTE 170M" } else { "BT.709" } ``` ### 4: BT.470M BT.470M is a standard that was used in analog television systems in the United States. This color primary setting is used in the following standards: - Rec. ITU-R BT.470-6 System M (historical) - United States National Television System Committee 1953 Recommendation for transmission standards for color television - United States Federal Communications Commission (2003) Title 47 Code of Federal Regulations 73.682 (a) (20) ### 5: BT.470BG BT.470BG is a standard that was used for European (PAL) television systems and DVDs. This color primary setting is used in the following standards: - Rec. ITU-R BT.470-6 System B, G (historical) - Rec. ITU-R BT.601-7 625 - Rec. ITU-R BT.1358-0 625 (historical) - Rec. ITU-R BT.1700-0 625 PAL and 625 SECAM ### 6: SMPTE 170M SMPTE 170M is a standard that was used for NTSC television systems and DVDs. - Rec. ITU-R BT.601-7 525 - Rec. ITU-R BT.1358-1 525 or 625 (historical) - Rec. ITU-R BT.1700-0 NTSC - SMPTE ST 170 (2004) ### 7: SMPTE 240M SMPTE 240M was an interim standard used during the early days of HDTV (1988-1998). Its primaries are equivalent to SMPTE 170M. ### 8: Film This represents generic film using Illuminant C. ### 9: BT.2020 BT.2020 is a standard used for ultra-high-definition video, i.e. 4K and higher. It may be used with or without HDR, as HDR is defined by the transfer characteristics. If you have a video with unset color primaries, it is safer to assume BT.709 by default. Videos which are BT.2020 should already have their color metadata set. This color primary setting is used in the following standards: - Rec. ITU-R BT.2020-2 - Rec. ITU-R BT.2100-2 ### 10: SMPTE 428 SMPTE 428 is used for D-Cinema Distribution Masters, aka DCDM. This color primary setting is used in the following standards: - SMPTE ST 428-1 (2019) - (CIE 1931 XYZ as in ISO 11664-1) ### 11: DCI-P3 DCI-P3 is a color space created by [Digital Cinema Initiatives](https://www.dcimovies.com). DCI-P3 has similar chromaticities (end points) and color gamut to AdobeRGB, which is more widely used in graphic arts and photography. DCI-P3 features a substantially larger gamut than sRGB. ### 12: Display P3 Display P3 is a standard developed by Apple that is used across iPhones, iPads, iMacs, MacBooks, and Apple-branded external displays released since 2016. Display P3 differs from DCI-P3 in a couple of ways: - Unlike DCI-P3, Display P3's white point is the same as the sRGB/AdobeRGB white point of 6500K - DCI-P3 has a gamma of 2.6, while Display P3 uses sRGB's gamma, which is closer to 2.2 ### 22: EBU Tech 3213 The EBU Tech 3213 standard defines the colorspace used by PAL/SECAM television systems in 1975, defining the chromaticities of primary colors and the white reference. It has largely been superseded by newer standards like SMPTE 170M and is considered obsolete for modern applications. The transfer function was never properly defined for this colorspace. --- # Color Range Canonical URL: https://encode.wiki/reference/colorimetry/range/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: be74f3a4aacf3cd40ef59487498e8cae7a3a53a36832f70c383ff03a1d8939aa Range is a concept that describes the valid values for a pixel. Typically, RGB will use full range and YUV will use limited range. What does this mean? For 8-bit video, full range indicates that all values between 0-255 may be used to represent a color value. On the other hand, limited range indicates that only values between 16-235, or 16-240 for chroma, are valid, and any values outside that range will be clamped to fit in that range. These expand to equivalent ranges for high bit depth videos. Why is limited range a thing that exists? Essentially, it's due to historical reasons, but it's a convention that we are stuck with today. Even though full range may provide slightly better color accuracy, it is far less meaningful for high bit depth content, and even HDR blu-rays use limited color range. Therefore, it is recommended to follow existing conventions. --- # Transfer Characteristics Canonical URL: https://encode.wiki/reference/colorimetry/transfer/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 9576722516aa143406bfe4fbb6a48818ebbdb29e0ce7c57b2b78bd49ad1a9f54 Transfer characteristics, also known as transfer functions, define the shaping used to compress the luminance or "brightness" of a video, we use the Transfer Characteristics to determine how to convert from a compressed video to a video that is in linear light, as well as converitng a linear light video to a compressed video. These functions are known as EOTF, OETF, or OOTF. Note that in some transfer functions like sRGB, you need to undo the OETF. The specification defines how to encode sRGB, and how a display should decode it, but not how to convert it back to linear light. In cases like these, it is typical to simply undo the encoding with an inverse transfer function, instead of using the function the display is to use. This luminance can often be refered to as gamma, however due to the term gamma also being used to refer to a myriad of other things, it is better to avoid this term when possible. Integer values much like with primaries are often used to define universal consants in the transfer functions. Transfer functions can either be absolute functions which specify the peak and minumum luminance, or be relative functions which specify how to display a pixel within a range between maximum and minimum intensity. These transfer functions typically have a nominal reference brightness, but this can be more of a suggestion rather then a hard requirement in some cases. Commonly seen with sRGB. The following values are available: ### 1: BT.1886 BT.1886 is the standard used for most modern, SDR video, and is a safe default assumption. This transfer function is used in the following standards: - Rec. ITU-R BT.709-6 - Rec. ITU-R BT.1361-0 conventional colour gamut system (historical) ### 2: Unspecified This value indicates that no transfer function is set for the video, and the player must decide which value to use. mpv will always assume BT.1886 in this case. ### 4: BT.470M BT.470M is a standard that was used in analog television systems in the United States. This transfer represents a power function with a gamma of 2.2. This transfer function is used in the following standards: - Rec. ITU-R BT.470-6 System M (historical) - United States National Television System Committee 1953 Recommendation for transmission standards for color television - United States Federal Communications Commission (2003) Title 47 Code of Federal Regulations 73.682 (a) (20) - Rec. ITU-R BT.1700-0 625 PAL and 625 SECAM ### 5: BT.470BG BT.470BG is a standard that was used for European (PAL) television systems and DVDs. This transfer represents a power function with a gamma of 2.8. ### 6: SMPTE 170M SMPTE 170M is a stanrard that was used for NTSC television systems and DVDs. Its transfer function is equivalent to BT.1886. This transfer function is used in the following standards: - Rec. ITU-R BT.601-7 525 or 625 - Rec. ITU-R BT.1358-1 525 or 625 (historical) - Rec. ITU-R BT.1700-0 NTSC - SMPTE ST 170 (2004) ### 7: SMPTE 240M SMPTE 240M was an interim standard used during the early days of HDTV (1988-1998). ### 8: Linear This value indicates that the content is already in linear light. ### 9: Logarithmic 100 Indicates a logarithmic transfer function with a 100:1 range. ### 10: Logarithmic 316 Indicates a logarithmic transfer function with a (100 \* sqrt(10)):1 range. ### 11: XVYCC Used in standard IEC 61966-2-4. I have no idea what this actually is. ### 12: BT.1361E This was intended to be a standard for "future" television systems, but it never really came into use. ### 13: sRGB Represents the sRGB colorspace. This transfer function is used in the following standards: - IEC 61966-2-1 sRGB (with MatrixCoefficients equal to 0) - IEC 61966-2-1 sYCC (with MatrixCoefficients equal to 5) Note: Content often specifies that it is encoded with sRGB but in reality, is encoded in a pure 2.2 function. While close there is a discrepency which often shows in the form of shadows taking on the wrong luminance. Specifying the encoding as BT.470M can be used to work around this. ### 14: BT.2020 10-bit Typically used with ultra-high-definition 10-bit SDR video. Its transfer function is equivalent to BT.1886. ### 15: BT.2020 12-bit Typically used with ultra-high-definition 12-bit SDR video. Its transfer function is equivalent to BT.1886. ### 16: PQ aka SMPTE 2084 PQ is the most widely used transfer function for HDR content. It allows for a wider range of luminance to be represented than conventional transfer functions. This transfer function is used in the following standards: - SMPTE ST 2084 (2014) for 10-, 12-, 14- and 16-bit systems - Rec. ITU-R BT.2100-2 perceptual quantization (PQ) system ### 17: SMPTE 428 SMPTE 428 is used for D-Cinema Distribution Masters, aka DCDM. ### 18: HLG aka Hybrid Log-Gamma HLG is an alternative transfer function for HDR content used by some televisions. This transfer function is used in the following standards: - ARIB STD-B67 (2015) - Rec. ITU-R BT.2100-2 hybrid log- gamma (HLG) system --- # Content license and attribution Canonical URL: https://encode.wiki/reference/content-license/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 7e319093aa77ecf90dbb8f799e5c511b9ac7f03e6b5577af251641ad312271f6 Unless an individual page, dataset, image, or other asset states different terms, Encode Wiki prose and media are licensed under the [Creative Commons Attribution-ShareAlike 4.0 International license](https://creativecommons.org/licenses/by-sa/4.0/). You may share and adapt that material, including commercially, provided you give appropriate credit, link to the license, indicate whether changes were made, and distribute adaptations under the same license. An item-specific notice takes precedence. The initial corpus is migrated from the AV1 Community Contributors' [Codec Wiki](https://github.com/av1-community-contributors/codec-wiki). Migrated pages record the original path, source revision, migration date, and whether they were modified. The full repository notice is available in [`NOTICE.md`](https://gitgud.io/encode/encode.wiki/-/blob/master/NOTICE.md). New application, test, migration, and infrastructure code is separately available under the MIT License. --- # FAQ Canonical URL: https://encode.wiki/reference/faq/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: f08250257880fa06e3f79bcfdaec7e9bb9e0dedc39501e26e2ea6e6630774d7d ## Why are you doing this? Multimedia encoding & the digital compression space is an incredible field that many tech enthusiasts, professionals, & laymen have no easy entry point to. Wikipedia has a vast amount of information on many of the individual topics covered here but doesn't offer a cohesive way to engage with the entire sphere of knowledge as a whole. While this site started as a lighthearted guide (you'll see the remnants of this strewn about the various wiki entries), it has quickly become an endeavor to unite digital compression aficionados to make the knowledge more accessible for all. ### But alternatives exist. Why not contribute there? While this is true, this is easier said than done. - Multimedia Wiki is not as active as it used to be, & a new effort makes sense to carry past efforts forward. - [guide.encode.moe](https://guide.encode.moe/) is stagnant and mostly focused on fansub, docs & personal experiences that are scattered around the Internet. - There are sources littered about that explain pieces of the larger puzzle, but these serve as small drops in a bucket of vast incoherency & don't meaningfully remedy the steep learning curve for understanding multimedia compression without background knowledge. ### How do I get started as a contributor? See our Contribution Guide page in the sidebar. ### Why "Codec Wiki"? This wiki is mostly going to be focused on multimedia compression, & the term "Codec" is already widely recognized & understood. While other topics like video filtering & general compression algorithms may be covered, the main focus remains multimedia compression. --- # Privacy Policy Canonical URL: https://encode.wiki/reference/legacy/codec-wiki-privacy-policy/ Collection: docs Status: archived License: CC BY-SA 4.0 Checksum: f3b9e07b166b7cc2b02308d59df2ead16ec05bee6acfcb5783d761af4893fc9b This site is hosted on GitHub Pages, & usage of this site is subject to [GitHub's Privacy Policy](https://docs.github.com/en/site-policy/privacy-policies/github-privacy-statement). GitHub may store information about your visit in the form of log files. Furthermore, this site uses a self-hosted instance of [Plausible Analytics](https://plausible.io) located in Singapore. Plausible Analytics is a lightweight and open source web analytics platform for website traffic analysis. We do not track, collect nor store any personally identifiable information; Plausible Analytics collects only aggregated information, which does not allow us to identify any visitor to our website. This information is collected to understand how users interact with the Codec Wiki and improve our efforts to better suit the needs of these users. We do not employ the use of cookies. The following information is collected: - Page URL - HTTP Referer - Browser & Browser Version - Operating System type & version - Device type - Country, region, & city Given this information, we ensure: - Analytics data is not shared explicitly with advertising companies or any other companies in general. It was made available to everyone for free on a public analytics page that is no longer available. - Analytics data isn't explicitly sent to any third parties - Analytics data is not monetized As a wiki dedicated to making multimedia codec knowledge more accessible, analytics information was made public via that former analytics page. You may opt out with various means of ad & tracker blocking. An example would be [UBlock Origin](https://ublockorigin.com/), which blocks our analytics script. If you have any questions, comments or concerns, you may reach out to site maintainer Gianni Rosato via [grosatowork@proton.me](mailto:grosatowork@proton.me). *Last updated 12 September 2023* --- # Terms of Use Canonical URL: https://encode.wiki/reference/legacy/codec-wiki-terms-of-use/ Collection: docs Status: archived License: CC BY-SA 4.0 Checksum: 85139bb604e78d12bac6dcce3a959435c6070ff660a55dc3299bccd7de655c00 These terms & conditions outline the rules & regulations for the use of the Codec Wiki Website as a visitor, contributor, or any other party who falls under the jurisdiction of these terms. By accessing this website we assume you accept these terms & conditions. Do not continue to use the site if you do not agree to all of the terms & conditions stated on this page. The following terminology applies to these Terms & Conditions & all other Agreements: “Client”, “You” & “Your” refers to you, the person log on this website & compliant to these terms & conditions. “Ourselves”, “We”, “Our” & “Us”, refers to the site's owner. “Party”, “Parties”, or “Us”, refers to both the Client & ourselves. Any use of the above terminology or other words in the singular, plural, capitalization and/or they/he/she, are taken as interchangeable & therefore as referring to the same. 1. Cookies We may employ the use of cookies in the future. At the time of writing, we do not. If accessing the site employs the use of cookies, you will be prompted to agree to use cookies in agreement with our Privacy Policy. As of writing, our Privacy Policy does not require users' consent to the use of cookies due to the fact that we do not use cookies. Some interactive websites use cookies to let them retrieve user details for each visit. Cookies may be used by our website to enable the functionality of certain areas to make it easier for people visiting our website. 2. Licensing Unless otherwise stated, the site's contributors and/or its licensors own the intellectual property rights for all material on the Codec Wiki. All intellectual property is distributed under CC BY-SA 4.0 unless it is specified otherwise. You may assess the terms & conditions of this Creative Commons license via this link: https://creativecommons.org/licenses/by-sa/4.0/. Using assets and/or intellectual property from the Codec Wiki site is subject to these terms unless otherwise specified. Under CC BY-SA 4.0, you may: > Share - copy and redistribute the material in any medium or format. > Adapt - remix, transform, and build upon the material for any purpose, even commercially. No party can revoke these freedoms as long as you follow the license terms. The terms are as follows: > Attribution - You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. > ShareAlike - If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. Contributors may not apply legal terms or technological measures that legally restrict others from doing anything the license permits. Parts of this website may offer an opportunity for users to post & exchange opinions & information in certain areas of the site. We do not filter, edit, publish or review Comments prior to their presence on the website. Comments do not reflect the views & opinions of contributors, its agents and/or affiliates. Comments reflect the views & opinions of the person who posts their views & opinions. To the extent permitted by applicable laws, the site's host & wiki contributors shall not be liable for the Comments or any liability, damages or expenses caused and/or suffered as a result of any use of and/or posting of and/or appearance of the Comments on this website. Part of the nature of a community-developed wiki is the ability for members of the community to make contributions to the contents of the site. Contributors are responsible for their own contributions to the site, & we shall not be liable for the contributions or any liability, damages or expenses caused and/or suffered as a result of any use of and/or posting of and/or appearance of the contributions to this website. We reserve the right to monitor all Comments and/or contributions & to remove any Comments and/or contributions which can be considered inappropriate, offensive or causes a breach of these Terms & Conditions. You warrant & represent that: You are entitled to post the Comments/contributions on our website & have all necessary licenses & consents to do so; The Comments/contributions do not invade any intellectual property right, including without limitation copyright, patent or trademark of any third party; The Comments/contributions do not contain any defamatory, libelous, offensive, indecent or otherwise unlawful material which is an invasion of privacy The Comments/contributions will not be used to promote unlawful activity. You hereby grant the Codec Wiki a non-exclusive license to use, reproduce, edit & authorize others to use, reproduce & edit any of your Comments/contributions in any & all forms, formats or media. 3. iFrames You may not create frames around our Webpages that alter in any way the visual presentation or appearance of our Website without proper attribution per CC BY-SA 4.0 where the license applies. Elsewhere, written permission is required if an asset is not under the CC BY-SA 4.0 license. 4. Content Liability We shall not be held responsible for any content that appears on your Website. You agree to protect & defend us against all claims that arise on your Website. No link(s) should appear on any Website that may be interpreted as libelous, obscene, or criminal, or which infringes, otherwise violates, or advocates the infringement or other violation of, any third party rights. 5. Your Privacy Please read our Privacy Policy. 6. Reservation of Rights We reserve the right to request that you remove all links or any particular link to our Website. You approve to immediately remove all links to our Website upon request. We also reserve the right to amend these terms & conditions & its linking policy at any time. By continuously linking to our Website, you agree to be bound to & follow these terms & conditions. 7. Removal of links from our website If you find any link on our Website that is offensive for any reason, you are free to contact & inform us at any moment. We will consider requests to remove links but we are not obligated to or so or to respond to you directly. We do not ensure that the information on this website is correct, we do not warrant its completeness or accuracy; nor do we promise to ensure that the website remains available or that the material on the website is kept up-to-date. 8. Disclaimer To the maximum extent permitted by applicable law, we exclude all representations, warranties & conditions relating to our website & the use of this website. Nothing in this disclaimer will: limit or exclude our or your liability for death or personal injury; limit or exclude our or your liability for fraud or fraudulent misrepresentation; limit any of our or your liabilities in any way that is not permitted under applicable law; or exclude any of our or your liabilities that may not be excluded under applicable law. The limitations & prohibitions of liability set in this Section & elsewhere in this disclaimer: (a) are subject to the preceding paragraph; & (b) govern all liabilities arising under the disclaimer, including liabilities arising in contract, in tort & for breach of statutory duty. As long as the website & the information on the website are provided free of charge, we will not be liable for any loss or damage of any nature. --- # Contribution Guide Canonical URL: https://encode.wiki/reference/legacy/contribution-guide/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 54b9aba14ccacc24d739b10e2ce6cf87b39d2525b7b04b96644726a391ef9a50 Codec Wiki - community-maintained wiki for all things encoding. ## Before You Contribute 1. By contributing to the Codec Wiki, you are communicating that you have read & agreed to our Terms & Conditions, Privacy Policy, & Code of Conduct. 2. Ensure your understanding of the material you're contributing is sufficient to a point where it is useful to the project. It is perfectly acceptable not to get everything right the first time, but always double-check your contributions for factual correctness. **Our current priority is filling out the existing pages with content. Please assist in doing this, if possible, before considering adding new pages.** If you're unsure the content in your entry is completely correct or you believe your entry needs review, please attach the following message at the top of your entry: > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. If you're aware your entry is too short or incomplete, please add the following message to the top of your entry: > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. If you've added a new page & you aren't sure what should go there (this isn't recommended while there are still so many empty pages to be filled), add the following message as your page entry: > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! ### Connect With Us If you'd like to join the "AV1 for Dummies" Discord server to communicate with other passionate contributors helping this project, please join using the widget below: Alternatively, we have a (soon to be) bridged Revolt server linked [right here](https://rvlt.gg/eSERRhSG). Revolt is an open-source Discord alternative, which you can read more about on [the current project about page](https://stoat.chat/about). ## Clone & Push Instructions **Make sure to clone from & edit the** `main` **branch only, & push your final changes to the** `deployment` **branch according to the instructions below. Also be sure to use node 18 LTS, as later versions tend to be troublesome.** *don't forget to add unimportant files to the .gitignore before making any commits* 1. Clone from the `main` branch to start to make a contribution: ```bash % git clone git@github.com:av1-community-contributors/av1-wiki.github.io.git -b main ``` 2. Test your changes locally before making a commit: ```bash % yarn % yarn start ``` 3. Push changes to `main` branch: ```bash % git add . % git commit -m "Commit Message" % git push -u origin main ``` 4. Deploy to `deployment` branch to make live on site: ```bash % yarn deploy ``` *Docusaurus Info* ## Website This website is built using [Docusaurus 3](https://docusaurus.io/), a modern static website generator. ### Installation ``` $ yarn ``` ### Local Development ``` $ yarn start ``` This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server. ### Build ``` $ yarn build ``` This command generates static content into the `build` directory and can be served using any static contents hosting service. ### Deployment See the initial instructions at the top. --- # Machine-readable access Canonical URL: https://encode.wiki/reference/machine-access/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 77c7cc6ee0d893666cb91302178846796dad1e44a215fb07b79930a7d6e5b4b1 Encode Wiki publishes the same public content in deterministic, machine-readable forms. These interfaces are generated during the static build and require no account or API key. ## Start with the index [`/api/v1/index.json`](/api/v1/index.json) enumerates every public document and article. Each record contains its stable ID, canonical page URL, clean Markdown and JSON retrieval URLs, status, tags, dates, authors, source and license metadata, headings, outbound references, byte count, and SHA-256 checksum. Retrieve an individual record using the URL in its `apiUrl` field. The corresponding `markdownUrl` returns plain Markdown with a short attribution header. Do not construct URLs from titles; follow the URLs in the index so nested identifiers remain opaque to clients. ## Contracts and release detection - [`/api/v1/openapi.json`](/api/v1/openapi.json) describes the read-only API using OpenAPI 3.1. - [`/api/v1/schema.json`](/api/v1/schema.json) is the JSON Schema for an individual corpus record. - [`/api/v1/release.json`](/api/v1/release.json) reports the Git content version, deterministic timestamp, corpus checksum, counts, and knowledge-bundle checksums. The API path changes only for breaking contract revisions. Compare `schemaVersion` before parsing and `contentVersion` or `contentChecksum` to detect a new release. Page checksums cover cleaned Markdown only. ## Bulk and agent-oriented files - [`/llms.txt`](/llms.txt) is a compact curated map of high-value pages and complete interfaces. - [`/llms-docs.txt`](/llms-docs.txt) and [`/llms-articles.txt`](/llms-articles.txt) contain full, cleaned collection text. - [`/llms-full.txt`](/llms-full.txt) combines both collections and may be too large for smaller context windows. - [`/api/v1/knowledge.ndjson.gz`](/api/v1/knowledge.ndjson.gz) is the complete corpus as gzip-compressed newline-delimited JSON for ingestion by future search, Ask AI, or MCP services. All API and text responses permit cross-origin reads and use short shared-cache lifetimes. The downloadable `.gz` file is an actual gzip payload, not an HTTP content-encoding wrapper. Its uncompressed and compressed checksums are in the release record. ## Attribution and content status Machine clients must retain each record's `canonicalUrl`, `license`, sources, provenance, and modification notice when redistributing migrated material. Treat `needs-review` content as potentially stale and `archived` content as historical. The API does not turn those records into current recommendations. --- # Butteraugli Canonical URL: https://encode.wiki/reference/metrics/butteraugli/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 8122bca8ca936e31c8fbf9deadaebeb694d0e5c4b198bbae9fba8a3189d52e17 # Butteraugli > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! Butteraugli is a perceptual distance metric that estimates the psychovisual similarity of two images. It is capable of outputting a heatmap containing differences between two input clips. While a [reference implementation by Google](https://github.com/google/butteraugli) exists, most people will want to use the [HIP/CUDA implementation `vship`](https://codeberg.org/Line-fr/Vship). Vship is intended for use in Vapoursynth scripts or as a standalone CLI tool without Vapoursynth and runs on GPU (Nvidia and AMD Radeon). ## Installing The [Release page](https://codeberg.org/Line-fr/Vship/releases) of the repository provides up-to-date binaries for Windows users. #### Compiling To compile Vship, follow the [instructions](https://codeberg.org/Line-fr/Vship#installation) on the repository. #### Archlinux AUR On archlinux, you may use the [AUR](https://wiki.archlinux.org/title/Arch_User_Repository) to install. Simply use your favorite AUR helper to install `vapoursynth-plugin-vship-cuda-git` (Nvidia) or `vapoursynth-plugin-vship-amd-git` (AMD Radeon) ```bash paru -S vapoursynth-plugin-vship-amd-git ``` ## Running ### Using the standalone FFVship library ```bash usage: ./FFVship [-h] [--source SOURCE] [--encoded ENCODED] [-m {SSIMULACRA2, Butteraugli}] [--start start] [--end end] [-e --every every] [-t THREADS] [-g gpuThreads] [--gpu-id gpu_id] [--json OUTPUT] [--list-gpu] Specific to Butteraugli: [--intensity-target Intensity(nits)] ``` ### In Vapoursynth scripts Example script: ```py import vapoursynth as vs core = vs.core # Load reference and distorted clips ref = core.bs.VideoSource("reference.mp4") dist = core.bs.VideoSource("distorted.mp4") # Calculate Butteraugli scores # intensity_multiplier controls sensitivity result = ref.vship.BUTTERAUGLI(dist, distmap=0, numStream = 4) # Extract scores from frame properties (three different norms available) scores_2norm = [frame.props["_BUTTERAUGLI_2Norm"] for frame in result.frames()] scores_3norm = [frame.props["_BUTTERAUGLI_3Norm"] for frame in result.frames()] scores_infnorm = [frame.props["_BUTTERAUGLI_INFNorm"] for frame in result.frames()] # Get all scores in one pass all_scores = [[frame.props["_BUTTERAUGLI_2Norm"], frame.props["_BUTTERAUGLI_3Norm"], frame.props["_BUTTERAUGLI_INFNorm"]] for frame in result.frames()] # Print average scores print(f"Average Butteraugli 3Norm distance: {sum(scores_3norm) / len(scores_3norm)}) print(f"Average Butteraugli 2Norm distance: {sum(scores_2norm) / len(scores_2norm)}) print(f"Average Butteraugli MaxNorm distance: {sum(scores_infnorm) / len(scores_infnorm)}) ``` > **Tip — Graphical visualization.** > > You can optionally generate visual distortion maps with: > ```py > # Set distmap=1 to visualize distortion > distmap_result = ref.vship.BUTTERAUGLI(dist, distmap=1) > > # The resulting clip is a grayscale visualization of distortions > distmap_result.set_output() > ``` ## Scoring Butteraugli outputs score in the range 0..inf with lower being better (closer to the source). A score below 1.0 is usually considered very good. --- # CVVDP Canonical URL: https://encode.wiki/reference/metrics/cvvdp/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: b550eb21e13a7d0be9bf5145f95030f327e2c55e2ae3c76304ef99c386ef2734 # CVVDP Via the [CVVDP GitHub README](https://github.com/gfxdisp/ColorVideoVDP/blob/main/README.md): > ColorVideoVDP is a full-reference visual quality metric that predicts the perceptual difference between pairs of images or videos. Similar to popular metrics like PSNR, SSIM, and DeltaE 2000 it is aimed at comparing a ground truth reference against a distorted (e.g. blurry, noisy, color-shifted) version. > This metric is unique because it is the first color-aware metric that accounts for spatial and temporal aspects of vision. The description of the metric process below is borrowed from the [fcvvdp documentation](https://github.com/halidecx/fcvvdp/blob/main/doc/cvvdp.md). ## Metric Process ### Initialization and Display Modeling Before processing pixels, the metric models the viewing environment through a specified display's angular resolution (pixels per degree). - Using specified display parameters (resolution, diagonal size, viewing distance), CVVDP calculates Pixels Per Degree (PPD). This determines how large a pixel appears to the eye. - The Contrast Sensitivity Function (CSF) determines how sensitive the eye is to specific spatial frequencies. It maps the display's frequency bands (derived from PPD) to sensitivity values. Available display model presets include: | Model | Description | | ------------ | -------------------------------------------------------- | | `fhd` | 24" FullHD monitor, 200 cd/m², office lighting (default) | | `4k` | 30" 4K monitor, 200 cd/m², office lighting | | `hdr_pq` | 30" 4K HDR, 1500 cd/m², low light | | `hdr_hlg` | 30" 4K HDR HLG, 1500 cd/m², low light | | `hdr_linear` | 30" 4K HDR linear, 1500 cd/m², low light | | `hdr_dark` | 30" 4K HDR, 1500 cd/m², dark room | | `hdr_zoom` | 30" 4K HDR, 10000 cd/m², close viewing | ### Input Loading and Display Mapping - Input images (uint8, uint16, or float) are converted to linear float RGB. If the input is integer-based, sRGB gamma decoding (approx. 2.4 power) is applied. - The linear RGB values are converted into absolute physical light units (nits) based on the display model. - SDR: clips values between 0 and 1, scales by max luminance, adds black level & reflected ambient light. - HDR: Performs tone mapping (PQ/HLG), clips to the display's peak luminance, adds black level and reflections. ### Color Space Conversion 1. Linear RGB is converted to the CIE XYZ color space 2. XYZ is transformed to DKL (Derrington-Krauskopf-Lennie), an opponent color space that models what is used by the human brain. - L: Luminance, a.k.a. achromatic brightness (L+M cones) - RG: Chromatic difference (L-M cones) - YV: S-cone opponent channel (S - (L+M)) ### Temporal Decomposition If the input is video (FPS > 0), the metric analyzes how pixel values change over time. It maintains `TemporalRingBuf` to store previous frames. - The code applies Finite Impulse Response (FIR) filters to the history of DKL frames (so, temporal filtering) - Low temporal frequency information (static or slow-moving) is stored in the _sustained channels_. Calculated for Luminance (Y), Red-Green (RG), and Yellow-Violet (YV). - High temporal frequency information (flicker or fast motion) is in the _transient channel_. Only calculated for luminance. We now have 4 channels to process spatially: Y_sus, RG_sus, YV_sus, Y_trans. ### Spatial Decomposition The visual system processes different sizes of features (frequencies) independently. CVVDP implements a _Gaussian pyramid_ to simulate this. 1. The image is repeatedly downscaled (blurred and subsampled). 2. At each level in the pyramid, local contrast is computed. 3. The upscaled version of the next lower level is subtracted from from the current level (Gaussian difference) 4. This difference is normalized by the local background luminance (`L_BKG`) to get _Weber Contrast_. ### CSF Weighting & Difference Calculation - For every pixel at every pyramid level, the contrast is multiplied by the CSF sensitivity. This scaling depends on: - Spatial Frequency, which is determined by the pyramid level - Background Luminance, where brighter areas generally have lower sensitivity to absolute differences - Channel, because the eye is less sensitive to chroma changes (RG/YV) than luma changes. - The absolute difference between the Reference and Distorted contrast values is calculated. ### Visual Masking This is the most complex step. It is designed to account for the fact that artifacts are harder to see in textured areas. - The code computes the minimum activity between the reference and distorted signals - A Gaussian blur is applied to this activity map to simulate the spatial extent of masking - Activity in one channel can mask errors in another. The code computes a masking denominator using weighted sums of activity from all 4 channels - The final difference d is compressed using a non-linear sigmoid-like function (not going to explain it here, probably best to read the code for more details) ### Pooling & Scoring - The masked differences are aggregated across the image using a Minkowski norm (Power of 4), then averaged - Scores from all pyramid levels and all four channels are summed - Scores are accumulated over frames using a power sum - The final raw quality metric (Q) is mapped to _JOD (Just Objectionable Difference)_, which is a more meaningful perceptual score. 10.0 is a perfect match (no visible difference), and lower scores mean the quality is worse | Score | Interpretation | | ---------- | ---------------------------------- | | 10.0 | Images are identical | | 9.0 - 10.0 | Barely visible difference | | 8.0 - 9.0 | Slight visible difference | | 7.0 - 8.0 | Noticeable but acceptable | | 5.0 - 7.0 | Clearly visible, somewhat annoying | | 3.0 - 5.0 | Very visible, annoying difference | | < 3.0 | Large, unacceptable difference | ## Implementations Several different implementations of CVVDP are available. Third-party implementations tend to outperform the reference tools. ### CVVDP [CVVDP](https://github.com/gfxdisp/colorvideovdp) is the first-party reference implementation of CVVDP by the University of Cambridge. It is implemented in Python, and makes use of libraries that allow utilization of the CPU or GPU. ### Vship [Vship](https://codeberg.org/Line-fr/Vship) is a GPU-accelerated metrics toolkit compatible with Vapoursynth. It also features its own standalone FFVship binary, available independent of Vapoursynth. Vship's CVVDP implementation is an order of magnitude faster than the reference implementation. ### fcvvdp [fcvvdp](https://github.com/halidecx/fcvvdp/tree/main) is a fast CPU-based CVVDP implementation by Halide Compression. It claims to be over 221% faster than the reference implementation with multithreading, making it useful in environments without access to GPUs. ## Visualization The graph below (from fcvvdp's docs) visualizes the CVVDP metric process: ![CVVDP](https://mermaid.ink/svg/pako:eNp9VNty2jAQ_ZUdPXSaGZIJDiSBh84EiCkJJATTS2LyoNoLaGpLrizT0JB_71q-5NILLyBz9uw5Z1d-ZIEKkXXZMlI_gzXXBuaDhQT6pNm3lebJGkZSGMEj8YsboWTx55k_EGkS8S14CQbpPezvf9j1eRRkETeY7qDnT6cDeAc9LkNwNf7IUAYC0_uivpcXQN_PuaHvuYScY5wozSNwRWRQV0iU4UK-UTRFve9qHiNMtQowTYVcFZABMSaZgRkuiZI0GhjFfIWFQjj3hzyOOQwwt10grIsJHaOy47k1MxYSuYaxWK3NDlx_NuyBUfD19i7_GlyOS7RboBswGzbg9jP011xKjCiCoV9bmpFA6GXLJeqy7I2jGvnW0NDSu6NZGcsOPvpelhpO8sK6GdxW_e__VTby55rLVKA0VRnc_idjL-H51P8Q9JFSG9kwLyjMjP7hEqZbmoYIba5kJBX5ppTkFxZ86dfbAWMVEPEX_IYa-koazVNTgi8teOyfJUm0tYvhoczpNsJsqSSfBkn5j-4JT7_naeezzfOmtcMCMrbcEyvkwG7IJi1W5Bl5_xJ6VUIrynOJerUtIVc24TqBXpRRyNd-FZuXaOQhqGVVXZZdFzdFqzTdr8YwEQ8E2MG0tF31czMZvAhyQp6mVtiNn0No_s_CqSh5mcpfdmyqVFSP8aZY2xZcKR3vYFYLL1El18zCvCwGtaFpjXFjV9vzi-vnBUpXmXkWOhf0eCQNUs9c-w7mz9fgNffcevlEXhK4yegFQxOmq3VxPXjFW9t5Me1-sSbF4ZM9fH7_3hWSmlRUlmNvjzXYSouQdY3OsMFi1DHPj-wxr14ws8YYF6xLP0Ouvy_YQj5RTcLlnVJxVaZVtlqz7pJHKZ2yJKQ9HghOJuP6Kc0hRN1XmTSs2-50HMvCuo_sgc6HzYOjpnN0eOy0O8edTrvVYFvWPekcHLVO2i2nedpuNZ1W86nBftm-hwenbcdpN0-PnY5z2Dk6bjYYhsIoPSne2PbF_fQbjzvS_g) --- # PSNR Canonical URL: https://encode.wiki/reference/metrics/psnr/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 2215d9354c3503005a556708bf517cfb12b3b2a76613b0d1de38753a42dd6fb0 PSNR (Peak Signal-to-Noise Ratio) is one of the most widely used objective full-reference image and video quality metrics. It measures the ratio between the maximum possible signal power and the power of corrupting noise, expressed in decibels (dB). The metric is calculated using the Mean Squared Error (MSE) between a reference image and a distorted image. The theory and math behind PSNR are well covered in [Wikipedia's PSNR entry](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio). ## Video Compression PSNR is widely used in video encoding applications because it is fast to compute, making it practical for real-time encoding decisions, and because it provides a consistent mathematical basis for comparing different encoding approaches. It is also used within video encoders to help make compression decisions. ### Inside Video Encoders Within video encoders, PSNR plays a crucial role in rate-distortion optimization (RDO), which is the process of finding the optimal balance between bitrate and quality. Encoders use PSNR as an in-loop metric when evaluating different encoding decisions, such as mode selection, motion estimation, and quantization parameter (QP) selection. For example, when deciding between different prediction modes or block sizes, the encoder will calculate the PSNR impact of each option along with its bit cost to determine the best choice. ### Limitations While PSNR is widely used due to its simplicity and computational efficiency, it has several notable limitations. The metric has notoriously poor correlation with the human eye's perception of quality, as PSNR is highly sensitive to all pixel-level errors when many do not have any perceptually relevant impact. When PSNR is used to inform RDO in video encoding, it can lead to suboptimal quality decisions; to combat this, encoders have to creatively take the human visual system into account. We cover some of this in the [Psychovisual entry](/start-here/psychovisual/). PSNR's weaknesses as a full reference distortion metric have led to the development of more advanced metrics like [SSIM](/reference/metrics/ssim/), [VMAF](/reference/metrics/vmaf/), and [XPSNR](/reference/metrics/xpsnr/), which attempt to better model human visual perception. ## Practical Use Cases PSNR is commonly used for evaluating image and video compression algorithms, assessing streaming quality, comparing codec performance, and more. Many video encoding tools and analysis suites report PSNR values for different luma/chroma components (Y, U, V) separately, as well as a weighted average. Despite its limitations, PSNR continues to be an important tool in the video compression field, particularly when used in conjunction with other quality metrics and subjective evaluation methods. --- # SSIM Canonical URL: https://encode.wiki/reference/metrics/ssim/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 80d5ffe95379bde3c0d4d215ad4a39a3a4ed6c0f6edff66bd203b50c8f37e364 The Structural Similarity Index Measure (SSIM) is a full-reference image and video quality metric that quantifies image fidelity degradation caused by processing such as lossy compression. Published in 2004 as part of an issue of [*IEEE Transactions on Image Processing*](https://en.wikipedia.org/wiki/IEEE_Transactions_on_Image_Processing), SSIM attempts to address the limitations of traditional metrics like [Peak Signal-to-Noise Ratio (PSNR)](/reference/metrics/psnr/) by evaluating visual quality based on the structural information that humans naturally use to assess visual quality. ## Overview SSIM works by comparing three key elements between the original and processed images: luminance, contrast, and structure. The luminance comparison measures the similarity of the average pixel intensities between the two images. These three comparisons are combined to produce a single similarity score ranging from -1 to 1, where 1 indicates perfect structural similarity. One of SSIM's main advantages is its ability to better align with human visual perception compared to traditional metrics. SSIM recognizes that pixels have strong inter-dependencies, especially when they are spatially close, which makes SSIM particularly effective at detecting changes in structural information that human observers would notice, such as blurring, blocking artifacts, or noise. As an in-loop metric in video encoders to improve decisionmaking, SSIM is more computationally expensive than PSNR, and doesn't always yield drastic improvements in fidelity per bit. [Psychovisual](/start-here/psychovisual/) encoder options are still necessary in many cases to achieve the best perceptual efficiency. ## Limitations In multimedia compression, SSIM can be more valuable in optimization scenarios where the goal is to maintain optimal perceptual quality for a given size. However, SSIM doesn't perfectly correlate with the human visual system; newer metrics like [XPSNR](/reference/metrics/xpsnr/) and [SSIMULACRA2](/reference/metrics/ssimulacra2/) have been developed to correlate more closely with human perception. Modern variations and extensions of SSIM have been developed to address specific needs. Multi-scale SSIM (MS-SSIM) evaluates images at different scales to better match human visual perception. Color SSIM variants have been proposed to better handle color information, and SSIMULACRA (succeeded by SSIMULACRA2) was developed to improve correlation with human perception. These adaptations have improved upon SSIM's perceptual goals in an ever-changing multimedia compression landscape. --- # SSIMULACRA2 Canonical URL: https://encode.wiki/reference/metrics/ssimulacra2/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 0998372e1207dbc91d580a15ffff5b8974ff09680970b7016d09dd10319aba30 # SSIMULACRA2 SSIMULACRA2 is a visual fidelity metric based on the concept of the multi-scale structural similarity index measure (MS-SSIM), computed in a perceptually relevant color space, adding two other (asymmetric) error maps, and aggregating using two different norms. It is currently the most reputable visual quality metric according to its correlation with subjective results, and is considered a very robust means of comparing encoders. It is debatable whether [Butteraugli](/reference/metrics/butteraugli/) is better for very high fidelity, but SSIMULACRA2 is considered the best for medium/low fidelity comparisons. ## Scoring The score that SSIMULACRA 2 outputs is simple: a number in range -inf..100. According to the developers of the metric, for image quality assessment, SSIMULACRA 2 scores correlate to subjective visual quality as follows: - Very high quality: `90` and above - High quality: `70` to `90` - Medium quality: `50` to `70` - Low quality: Below `50` ## Metric Breakdown A step-by-step description of the SSIMULACRA2 metric follows. The steps assume we have two input images, a reference image as well as a distorted image to be compared and scored. ### Convert sRGB to Linear RGB Undo the sRGB gamma curve to obtain linear light values for each RGB channel. This converts perceptual-encoded pixel values into physically meaningful intensities. ### Transform to XYB Map linear RGB into an opsin-inspired space that separates perceptual channels. Steps: apply an absorbance-like linear transform, clamp negatives, take a cube root to compress dynamic range, add small biases, then mix into three channels (X, Y, B). Resulting channels are tuned to align better with human vision than raw RGB. ### Normalize Values Apply slight shifts and scalings so all channel values are positive and stable for later statistical operations. This prevents division instability and extreme ratios in subsequent steps. ### Build a Multi-Scale Image Pyramid Create multiple downscaled versions of both images (typically several scales, each half the previous dimension). Each scale captures structure at a different spatial frequency. The metric computes statistics independently at every scale. ### Blurred Statistics For each scale and each perceptual channel, compute local blurred statistics. Local blurred quantities are computed on each pixel using a separable spatial blur: - Local mean (blurred image). - Local second moments: blurred squared values and blurred cross-products between reference and distorted channels. These give local variance and covariance estimates analogous to SSIM’s variance/covariance. ### Similarity & Artifacts **Similarity map (SSIM-like):** Combine local means, variances, and covariance into a per-pixel similarity value. This value measures structural agreement while using small stabilizing constants to avoid division by zero. Similarity is clamped to a sensible range and emphasizes structural fidelity. **Edge / artifact map:** Compare local deviations from local means between distorted and reference images to detect: - New artifacts (excess local detail or harsh edges introduced by distortion). - Lost detail (original detail suppressed or blurred). Compute per-pixel artifact and lost-detail measures, and preserve higher-order statistics to capture outliers. ### Aggregate statistics per channel and scale Across each channel and scale compute compact summaries: - Mean of the per-pixel similarity and artifact/lost-detail measures. - Higher-order moment summaries (fourth-moment based measures reduced by a fourth-root) to detect heavy tails and strong local errors. These condensed statistics encode both average behavior and extreme localized errors. ### Weighted combination of statistics Multiply each aggregated statistic by a pre-tuned weight. Sum all weighted terms across channels and scales to produce a single scalar accumulator. Weights are learned/tuned to map the diverse statistics into a perceptually meaningful predictor. ### Nonlinear mapping to final score Pass the accumulator through a nonlinear curve (polynomial and a power-law transform). This mapping compresses the predictor into a bounded perceptual score. The final value is expressed on a convenient scale where higher means better and values near the top indicate imperceptible differences. ## Notes - The metric examines structure and edge behavior separately. It penalizes both new artifacts and lost detail. - Multi-scale analysis makes it sensitive to distortions at different spatial frequencies. - Aggregating higher-order moments preserves sensitivity to rare but visually important outliers. ## Implementations There are a couple of different SSIMULACRA2 implementations available, some useful in different contexts. ### Cloudinary's SSIMULACRA2 [Cloudinary's SSIMULACRA2 implementation](https://github.com/cloudinary/ssimulacra2) is the reference implementation written in C++. It comes from the libjxl project, the reference implementation of the [JPEG XL](/codecs/images/jxl/) image codec. ### `vapoursynth-zip` Filter [`vapoursynth-zip`](https://github.com/dnjulek/vapoursynth-zip) is a collection of filters for use with [Vapoursynth](/tools/filtering/vapoursynth/). It is written in Zig, and features a SSIMULACRA2 implementation. ### fssimu2 [fssimu2](https://github.com/gianni-rosato/fssimu2) is a fast SSIMULACRA2 implementation written in Zig. It is designed for speed, claiming to be up to 14% more performant while using just 50% of the memory. It displays a recorded error of ~1.5% relative to Cloudinary's reference implementation, and it achieves 99.7% correlation according to the Pearson correlation coefficient documented in the README. It is based on Julek's Zig implementation. ### Vship [Vship](https://codeberg.org/Line-fr/Vship) is a GPU-accelerated metrics toolkit compatible with Vapoursynth. It also features its own standalone FFVship binary, available independent of Vapoursynth. Vship's SSIMULACRA2 implementation is an order of magnitude faster than CPU-based implementations, and has reportedly high correlation with the reference implementation. ### `ssimulacra2_rs` [`ssimulacra2_rs`](https://github.com/rust-av/ssimulacra2_bin) is a binary interface to the [Rust implementation of the SSIMULACRA2 metric](https://github.com/rust-av/ssimulacra2). It is notable for being one of the first independent implementations, as well as one of the first to consider video inputs. --- # VMAF Canonical URL: https://encode.wiki/reference/metrics/vmaf/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: cc82e7ba51c9c8591a85e0319ca913effd1209d9fee6583281d785f6f0c83463 # VMAF > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Short for Video Multimethod Assessment Fusion, VMAF is a full reference video quality assessment algorithm developed primarily by Netflix. ## Installation Vmaf comes as a part of [libvmaf](https://github.com/Netflix/vmaf). There are two ways it is commonly used: * As an [FFmpeg](/tools/utilities/ffmpeg/) filter * As a standalone binary The instructions below are written for Linux & macOS. On Windows, you can use the [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install) to follow along. #### Standalone Binary In order to build from source, follow the instructions below. 0. Install the required dependencies via your package manager of choice. The necessary dependencies are `nasm`, `ninja-build`, `doxygen`, & `xxd`. 1. Clone the repository & enter the corresponding directory ```bash title="Clone & Enter" git clone https://github.com/Netflix/vmaf/ cd vmaf/ ``` 2. Compile with `meson` & `ninja` ```bash meson setup libvmaf libvmaf/build --buildtype release -Denable_float=true sudo ninja -vC libvmaf/build install ``` Now, you can run the VMAF binary with the following command: ```bash vmaf --help ``` If you would not like to build from source, you may grab the latest build from the VMAF [GitHub releases](https://github.com/Netflix/vmaf/releases) for your operating system. Now, you can: ```bash /path/to/vmaf --reference refrence.y4m --distorted distorted.y4m ``` **Tip:** If the VMAF binary exists but is not market as executable, you might need to `chmod +x /path/to/vmaf` Explainer on command line flags can be found [here](https://github.com/Netflix/vmaf/blob/master/libvmaf/tools/README.md) The disadvantage of using the bin is that you need .yuv|y4m files, you can that overcome by using named pipes Simple example using ffmpeg as a decoder: ```bash # create the pipes mkfifo ref.pipe mkfido dist.pipe # run these each in a new terminal, order docent matter ffmpeg -v error -i ref.mkv -strict -1 -f yuv4mpegpipe - > ref.pipe ffmpeg -v error -i dist.mkv -strict -1 -f yuv4mpegpipe - > dist.pipe # after starting the two ffmpeg processes, # start the vmaf in a new terminal /path/to/vmaf --reference ref.pipe --distorted dist.pipe # delete the pipes after usage rm ref.pipe dist.pipe ``` The Advantages of this are: * No need for a ffmpeg build with `--enable-libvmaf` * Clear & simple usage of VMAF's various options, like `--aom_ctc` Disadvantages are: * Difficult/awkward to use without a wrapper script #### FFmpeg Filter If you are not sure if you have VMAF installed, you can check by running `ffmpeg -help` and looking for whether or not the `--enable-libvmaf` flag appears in the banner that is printed to the terminal. If you do not see this, you will need to build ffmpeg from source with the `--enable-libvmaf` flag or grab a pre-compiled build of FFmpeg with the flag enabled. *Via the [VMAF github repo](https://github.com/Netflix/vmaf/blob/master/resource/doc/ffmpeg.md):* ## Using VMAF with FFmpeg After installing `libvmaf`, you can use it with [FFmpeg](http://ffmpeg.org/). Under the FFmpeg directory, configure, build and install FFmpeg with: ```shell script ./configure --enable-libvmaf make -j4 make install ``` Using FFmpeg+libvmaf is very powerful, as you can create complex filters to calculate VMAF directly on videos of different encoding formats and resolutions. For the best practices of computing VMAF at the right resolution, refer to our [tech blog](https://medium.com/netflix-techblog/vmaf-the-journey-continues-44b51ee9ed12). We provide a few examples how you can construct the FFmpeg command line and use VMAF as a filter. Note that you may need to download the test videos from [vmaf_resource](https://github.com/Netflix/vmaf_resource/tree/master/python/test/resource). Below is an example on how you can run FFmpeg+libvmaf on a pair of YUV files. First, download the reference video [`src01_hrc00_576x324.yuv`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/yuv/src01_hrc00_576x324.yuv) and the distorted video [`src01_hrc01_576x324.yuv`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/yuv/src01_hrc01_576x324.yuv). `-r 24` sets the frame rate (note that it needs to be before `-i`), and `PTS-STARTPTS` synchronizes the PTS (presentation timestamp) of the two videos (this is crucial if one of your videos does not start at PTS 0, for example, if you cut your video out of a long video stream). It is important to set the frame rate and the PTS right, since FFmpeg filters synchronize based on timestamps instead of frames. The `log_path` is set to standard output `/dev/stdout`. It uses the `model_path` at location `/usr/local/share/model/vmaf_float_v0.6.1.json` (which is the default and can be omitted). ```shell script ffmpeg -video_size 576x324 -r 24 -pixel_format yuv420p -i src01_hrc00_576x324.yuv \ -video_size 576x324 -r 24 -pixel_format yuv420p -i src01_hrc01_576x324.yuv \ -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \ [1:v]setpts=PTS-STARTPTS[distorted]; \ [distorted][reference]libvmaf=log_fmt=xml:log_path=/dev/stdout:model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json:n_threads=4" \ -f null - ``` The expected output is: ```shell script [libvmaf @ 0x7fcfa3403980] VMAF score: 76.668905 ``` Below is a more complicated example where the inputs are packaged `.mp4` files. It takes in 1) a reference video [`Seeking_30_480_1050.mp4`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/mp4/Seeking_30_480_1050.mp4) of 480p and 2) a distorted video [`Seeking_10_288_375.mp4`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/mp4/Seeking_10_288_375.mp4) of 288p upsampled to `720x480` using bicubic, and compute VMAF on the two 480p videos. Bicubic is used as the recommended upsampling method (also see the [techblog](https://medium.com/netflix-techblog/vmaf-the-journey-continues-44b51ee9ed12) for more details). ```shell script ffmpeg \ -r 24 -i Seeking_30_480_1050.mp4 \ -r 24 -i Seeking_10_288_375.mp4 \ -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \ [1:v]scale=720:480:flags=bicubic,setpts=PTS-STARTPTS[distorted]; \ [distorted][reference]libvmaf=log_fmt=xml:log_path=/dev/stdout:model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json:n_threads=4" \ -f null - ``` The expected output is: ```shell script [libvmaf @ 0x7fb5b672bc00] VMAF score: 51.017497 ``` See the [FFmpeg's guide to libvmaf](https://ffmpeg.org/ffmpeg-filters.html#libvmaf), the [FFmpeg Filtering Guide](https://trac.ffmpeg.org/wiki/FilteringGuide) for more examples of complex filters, and the [Scaling Guide](https://trac.ffmpeg.org/wiki/Scaling) for information about scaling and using different scaling algorithms. ### Note about the model path on Windows Due to Windows not having a good default for where to pull the VMAF model from, you will always need to specify `model_path` when calling libvmaf through `ffmpeg`. However, you will need to be careful about the path you pass to `model_path`. If you are using a relative path for your `model_path`, you can completely ignore this whole section, else if you are trying to use an absolute Windows path (`D:\mypath\vmaf_v0.6.1.json`) for your `model_path` argument, you will need to be careful so `ffmpeg` passes the right path to `libvmaf`. The final command line will depend on what shell you are running `ffmpeg` through, so you will need to go through the following steps to make sure your path is okay. 1. Convert all of the backslashes `\` to forward slashes `/` (`D:/mypath/vmaf_v0.6.1.json`) 2. Escape the colon `:` character by using a backslash `\` (`D\:/mypath/vmaf_v0.6.1.json`) 3. Then escape that backslash with another backslash (`D\\:/mypath/vmaf_v0.6.1.json`) 4. The next step will depend on the shell that will run `ffmpeg`: For PowerShell and Command Prompt, this will be enough and your final `ffmpeg` command line will look something like ```powershell ./ffmpeg.exe -i dist.y4m -i ref.y4m \ -lavfi libvmaf=model_path="D\\:/mypath/vmaf_v0.6.1.json" \ -f null - ``` > **Info — Quoting the path.** > > Note: I only quoted the path part for trivial reasons and in this specific case, it can be unquoted or you can quote the whole part after lavfi starting from `libvmaf` to `json` and it should give the same result due to neither shell treating the `\` as a special character For bash or specifically msys2 bash, it has some additional considerations. The first thing to know is that bash treats the backslash character `\` a bit special in that it's an escape character normally when not put inside single quotes. The second thing to know is that msys2's bash attempts convert a posix-like path (`/mingw64/share/model/vmaf_v0.6.1.json`) to a Windows mixed path (`D:/msys2/mingw64/share/model/vmaf_v0.6.1.json`) when passing arguments to a program. Normally, this would be fine, however, in our case, this works against us since we cannot allow it to convert the path to a normal path with an un-escaped colon. For this, we will need to not only escape the escaped backslash, but we will also need to pass the `MSYS2_ARG_CONV_EXCL` environment variable with the value of `*` to make sure it doesn't apply that special conversion on any of the arguments ```bash MSYS2_ARG_CONV_EXCL="*" \ ./ffmpeg.exe -i dist.y4m -i ref.y4m -lavfi \ libvmaf=model_path="D\\\:/mypath/vmaf_v0.6.1.json" -f null - ``` > **Info — Quotes.** > > Note: in this case, the quotes are not as trivial as the PowerShell/cmd version, as removing the quotes entirely will require you to re-escape the backslash resulting in 4 total backslashes, but quoting the whole argument will be fine. > **Note — Single Quotes.** > > Second Note: if you use single quotes around the path, it will be fine as well and the final command line would look like > > ```bash > MSYS2_ARG_CONV_EXCL="*" \ > ./ffmpeg.exe -i dist.y4m -i ref.y4m -lavfi \ > libvmaf=model_path='D\\:/mypath/vmaf_v0.6.1.json' -f null - > ``` > > with only a double backslash instead of a triple. ## Scoring scores range from 0 to 100, and are best interpreted in a linear way, 100 meaning perfect quality, 0 meaning not recognisable, more info [in the Best Practices section here](https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12) It aligns with mean opinion scores (MOS) really well at low/medium bitrates, as stated [in this benchmark](https://videoprocessing.ai/benchmarks/video-quality-metrics_both.html) ## Some weaknesses * Newer codecs like AV1 and VVC introduce new kinds of artifacting that v0.6.2 (current model as of Jan 2024) doesn't recognise, that's why its performance might degrade, for example, high motion scenes being affected badly * It's bad at "transparent" levels of quality, kinds of quality that the average viewer might not notice * Synthetic grain throws off scores, this issue is not isolated to vmaf, but it should be noted regardless > **Tip — With ffmpeg you can disable application of synthetic grain.** > > place `-filmgrain 0` before `-i` in the above ffmpeg commands, limited to decoding with dav1d > TODO: replace this tip with an export_side_data solution * As of January 2024, it doesn't work on HDR content, nothing prevents you from feeding it un-tonemapped PQ but scores will be off * In contrast with SSIMULACRA2, it focuses on appeal, not necessarily on fidelity. They often align, but not always. * Due to the ML nature, comparing the same video to itself will not always result in a score of 100 ### Comparing to SSIMULACRA2 One big advantage over SSIMULACRA2 is the inclusion of some temporal information. This means that VMAF weights frames that have a lot of motion higher. Meanwhile, SSIMULACRA2 based solutions compare each frame to the reference frame individually, since it is an image metric at heart. VMAF also wins in speed and general ease of use. # Additional resources - [VMAF Documentation on GitHub](https://github.com/Netflix/vmaf/tree/master/resource/doc) - [Medium Article by Netflix](https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12) --- # XPSNR Canonical URL: https://encode.wiki/reference/metrics/xpsnr/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 14be1bac82a540fb8fac9e56a38286c4802100e2de57a4a769ae30a2b7fafa8e > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. XPSNR is a full-reference distortion measurement algorithm for video quality assessment, based on the concept of the peak signal to noise ratio (PSNR), tuned to better reflect the human visual system. XPSNR is considered to be a highly reputable visual quality metric according to its correlation with subjective results relative to its impressive speed. Unlike image-first metrics like [SSIMULACRA2](/reference/metrics/ssimulacra2/), XPSNR is a video-focused metric, and uses temporal information to compute its score. The reference implementation by Fraunhofer HHI is a [FFmpeg 7.0 plug-in](https://github.com/fraunhoferhhi/xpsnr), and there is a [community-built standalone implementation](https://github.com/gianni-rosato/sxpsnr) available as well. For more details about the algorithm and its design, [a scientific paper is available](https://www.itu.int/pub/S-JOURNAL-ICTS.V3I1-2020-8). ## Installation XPSNR comes as an FFmpeg 7.0 plug-in. The process of installing it is somewhat straightforward: 1. Download the source code for FFmpeg 7.0: ```bash git clone -b release/7.0 https://git.ffmpeg.org/ffmpeg.git ffmpeg ``` 2. Download the code for the XPSNR plug-in: ```bash git clone https://github.com/fraunhoferhhi/xpsnr ``` 3. Copy the plug-in files from XPSNR's code to FFmpeg's: ```bash cp xpsnr/libavfilter/* ffmpeg/libavfilter/ ``` 4. Configure and compile FFmpeg: ```bash cd ffmpeg ./configure ... make -j $(nproc) ``` ## Usage XPSNR is used as an FFmpeg filter, similarly to the built-in PSNR plugin: ```bash ./ffmpeg -i ref.mkv -i test.mkv -lavfi xpsnr="stats_file=xpsnr.log" -f null - ``` ## Notes * Being the reference implementation an FFmpeg plug-in and not a standalone binary is inconvenient for some workflows. Currently, there are some attempts at making a [standalone XPSNR binary](https://github.com/gianni-rosato/sxpsnr), although they are not very widespread yet. * XPSNR uses a PSNR-like scoring system: a logarithmic scale in range 0..inf. This is considerably harder to interpret than other metrics (e.g. [SSIMULACRA 2](/reference/metrics/ssimulacra2/) or [VMAF](/reference/metrics/vmaf/)); and the threshold to what constitutes good quality is content-dependent. (Preliminary testing seems to indicate that anything above a XPSNR score of 42.00 is visually lossless.) * XPSNR scores are computed independently for the luma (Y) and two chroma (U, V) components of videos, thus actually providing 3 scores. Although the developers argue that using the minimum of these 3 scores as "definitive score" (i.e. better correlates to human impressions), some people in the community prefer using a weighted sum: `(4 * XPSNR_Y + XPSNR_U + XPSNR_V) / 6`. ## Comparing to SSIMULACRA 2 The main advantage of XPSNR over SSIMULACRA 2 is that it is considerably faster to compute. XPSNR can achieve real-time speeds for 1080p 24 fps video, making it more convenient to quickly compare test encodes. Regarding their visual assessment capabilities, XPSNR and SSIMULACRA 2 seem to complement each other quite well, having both their own strong points and weak points with specific video content. If possible, using both at the same time provides great benefits for video quality assessment and comparison. --- # Privacy Canonical URL: https://encode.wiki/reference/privacy/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 1ec1b0b891f38086fecb7a36dd82e9155a0277fc5c9bdde4fb60fce10842cce1 Encode Wiki is a static site. Reading public pages does not require an account, and the site does not currently provide comments, forms, or an AI chat service. The production domain uses a self-hosted [Umami](https://umami.is/) analytics service at `u.servers.moe` to understand aggregate site usage. The local analytics loader injects the tracker only when the browser hostname is exactly `encode.wiki`, and the tracker itself is also restricted to that domain. Local development and `dev.encode.wiki` therefore do not contact or record production analytics. No analytics API or administrative credential is included in the site. The hosting and analytics servers necessarily receive ordinary network request data such as IP addresses, user-agent headers, requested URLs, timestamps, and error information. Encode Wiki does not use advertising cookies or sell reader data. This notice will be revised before any feature that accepts user-submitted data is launched. Questions or privacy concerns can be reported through a confidential issue in the project's GitLab repository. --- # Resources Canonical URL: https://encode.wiki/reference/resources/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 185c6f7dc87b45d96d79ea0713b209953f95b94d54dbb05efbaecd749e7c1328 Encoding resources that you might find useful. - https://guide.encode.moe - Filtering and fansubbing. - https://lvsfunc.encode.moe/en/latest - lvsfunc documentation. - https://silentaperture.gitlab.io/mdbook-guide/introduction.html - SilentAperture's Advanced Encoding guide, mostly about filtering. - https://x265.readthedocs.io/en/master - x265 technical documentation, made by MulticoreWare themselves. - http://www.chaneru.com/Roku/HLS/X264_Settings.htm - x264 settings. - https://kokomins.wordpress.com/2019/10/10/anime-encoding-guide-for-x265-and-why-to-never-use-flac - Anime encoding guide by Kokomins. Has some pretty good advices regarding psychovisual stuff and x265. - https://wiki.xiph.org/Main_Page - Xiph Wiki. --- # Start here Canonical URL: https://encode.wiki/start-here/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: d0193bfaad11352482d7d6a13ddaacf6cee436055a029f63d24dfcc6c47d5466 Encoding transforms information into another representation. For media, that usually means using a codec to make audio, video, images, or subtitles practical to store and distribute. This section will help you understand: - what codecs, formats, encoders, and containers each do; - the tradeoff between quality, size, speed, compatibility, and complexity; - how lossy and lossless compression differ; and - how to recognize and investigate common encoding artifacts. Most of this material will be migrated from the existing Codec Wiki with explicit attribution and a visible review status. --- # Prologue Canonical URL: https://encode.wiki/start-here/encoding-basics/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 9112160926f97eaa05673029eefad8eb9241899458c4082160314df792baf15e Multimedia compression has allowed us to communicate information and expression in novel ways as the unsung hero of the modern Web. Despite this, it is often difficult to uncover information about codec technology that is accurate, informed, and accessible. This wiki aims to demystify multimedia compression while connecting codec enthusiasts to create a sink of knowledge for everyone. The Codec Wiki is not geared toward the mathematics, adoption/patent politics, or specifications of specific coder/decoder implementations – sources like Wikipedia cover these details. We are focused on making accurate, useful information highly accessible. ### What is a Codec A codec, shortened from coder/decoder, is a system that handles digital data according to a specification. Typically this means compressing and decompressing digital media. Codecs are used to encode media for storage and transmission, and then decode for playback, editing, etc. Multimedia codecs generally either [discard data](/start-here/psychovisual/) using [lossy](/start-here/video-artifacts/) compression to reduce filesize, or they use clever lossless compression tricks to maintain a mathematically identical stream to the input media while still reducing filesize. Lossless compression can be reversed to be the exact same as the input data, while lossy compression does not share this quality as it discards data for smaller filesizes. Some common uses of codecs include: - Video compression: Video codecs like [H.264](/codecs/video/avc/), [VP9](/codecs/video/vp9/) & [AV1](/codecs/video/av1/) allow digital video files to be compressed to much smaller sizes. A video codec can encode a video stream while it is being recorded or before it is distributed, and decode it when it is played back. This allows video to be shared more quickly while using less storage & bandwidth. - Audio compression: Audio codecs like [MP3](/codecs/audio/mp3/), [AAC](/codecs/audio/aac/), and [Opus](/codecs/audio/opus/) compress audio files like songs & podcasts. This allows them to be easily distributed & stored. - Image Compression: Image codecs compress images while maintaining a versatile featureset for the myriad of ways one may decide they'd like to compress an image. Color depth, HDR, transparency, color space information, EXIF data, and many other factors are relevant when working with images. - Data Compression: General compression algorithms like [ZIP](/codecs/data/zip/) & [zstd](/codecs/data/zstd/) are designed to compress *any* kind of data, not just multimedia specific data. This includes web assets, executables, text archives, and even entire filesystems. --- # high-dynamic-range Canonical URL: https://encode.wiki/start-here/high-dynamic-range/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 040d1a9edea4652d233a12ef2f80981bc66968b56e661414fd9d84322c79ef17 # High Dynamic Range HDR (High Dynamic Range) is a technology used in modern TVs and displays to produce more vibrant and lifelike images. In simple terms, it allows your TV to display a wider range of colors and brightness levels than standard displays. This means that you can see more details in both bright and dark areas of an image, which can make movies, TV shows, and video games look much more realistic. HDR10 works by using metadata that tells your TV how to display the content in the best way possible. This metadata includes information about the maximum brightness level and color gamut of the content, which allows your TV to adjust its settings to match the content being displayed. In other words, HDR10 helps your TV display images that are closer to what the content creators intended you to see, resulting in a more immersive viewing experience. ## HLG HLG (Hybrid log-gamma) is a type of HDR video format that was developed to optimize video for both standard dynamic range (SDR) and HDR displays, jointly developed by the BBC and NHK. To understand how HLG works, it's helpful to know that the way we perceive brightness and color in a video is different from how it's captured and displayed on a screen. Brightness and color information is usually captured in a logarithmic curve, while SDR displays typically reproduce the image with a gamma curve. HDR displays, on the other hand, reproduce the image with a different type of curve, known as the Perceptual Quantizer (PQ) curve. The HLG curve is a hybrid of these two curves, which means that it's optimized for both SDR and HDR displays. It's designed to work with a wider range of brightness levels than SDR displays, but also be backward compatible with SDR displays. In simpler terms, the HLG curve is a way of capturing and displaying video that works well on both SDR and HDR displays. It's like a bridge between the way video is captured and the way it's displayed, and it's designed to optimize the video for a wider range of brightness levels than traditional SDR video. The result is video content that looks more realistic and vivid on both SDR and HDR displays. ## HDR10 HDR10 is an open high-dynamic-range video (HDR) standard announced on 27 August 2015 by the Consumer Technology Association. It is the most widespread of the HDR formats. It only allows static metadata. ## HDR10+ HDR10+ is basically an upgrade to the previous HDR10 by adding dynamic metadata support (in ``.json``) to optimize each scene's content light level as the director intended. ## Dolby Vision Dolby Vision is proprietary HDR format developed by Dolby Laboratories and a direct competitor to HDR10+. --- # Lossless Compression Canonical URL: https://encode.wiki/start-here/lossless/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 72bf3589f4429f91c0c96eccdd8f4c7e3de2450bad301a6da11dd62021de6180 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Lossless compression is a method of data compression that allows the original data to be perfectly reconstructed from the compressed data. This is particularly important in applications where perfectly preserving the original fidelity of a medium is critical, such as in archiving, generic data compression, & professional media editing. To understand how lossless compression works, we will first delve into the concepts of redundancy, entropy, and specific compression techniques oft used in lossless compression. ### Redundancy & Entropy The concepts of redundancy & entropy are important to understand as you continue reading. **Redundancy** refers to the repetitive or predictable elements in data. These elements do not add new information and can be efficiently encoded to reduce the overall data size without losing any information. **Entropy**, in the context of information theory, is a measure of the unpredictability or randomness of data. Lower entropy implies higher redundancy, implying that the data is theoretically more compressible. In lossless compression, the goal is to reduce redundancy and encode data as efficiently as possible based on its entropy. ### Techniques in Lossless Compression 1. **Run-Length Encoding (RLE)**: RLE is a simple form of lossless compression where sequences of the same data value (runs) are stored as a single data value and a count. This technique is effective for compressing data with long runs of identical samples, such as silence or constant tones. For example, the sequence `AAAAABBBCC` could be encoded as `5A3B2C`. 2. **Huffman Coding**: Huffman coding is an entropy encoding algorithm used for lossless data compression that works by separating an input into component symbols and replacing each symbol with a code. The algorithm builds a binary tree, with each leaf node representing a symbol separated from the input data, and the path from the root to the leaf representing the binary code for that symbol. Huffman coding is effective when the probability distribution of the input characters is known and can be exploited. Imagine you are storing the state of a traffic light; it is either green, yellow, red, or off for maintenance. As the operator, you have determined that it is green 50% of the time, red 40% of the time, yellow 9% of the time, and disabled 1% of the time. Because there are four options, you can accurately represent all of the possible symbols in our example using two bits. Green could be `00`, red `01`, yellow `10`, and off `11`. While assigning two-bit codes accurately conveys the information, we're storing an average of two bits per symbol; we can reduce the average number of bits per symbol by taking the probabilities into account here. We'll assign green to `0` since it appears the most frequently; this is the first leaf on our binary tree. Next, we have the leaves that stem from the `1` code; red can simply be `11`, while yellow can be `100` and the disabled symbol can be represented by `101`. This gives us the following Huffman codes: - Green (50%): `0` - Red (40%): `11` - Yellow ( 9%): `100` - Disabled ( 1%): `101` Represented by the following tree: ![Traffic light Huffman tree](https://raw.githubusercontent.com/av1-community-contributors/images/main/color-huffman-tree-svg.svg) Now, if we do the math by multiplying the probability by the length of each code and taking the weighted sum: (50% • 1) + (40% • 2) + (9% • 3) + (1% • 3) = **1.6** We end up with an average of **1.6 bits per symbol**. Even in our relatively simple example, this shows how Huffman coding can save space quite effectively while still losslessly representing the same information. 3. **Arithmetic Coding** Arithmetic coding is another entropy encoding technique that represents an entire message as a single number in the interval `[0, 1)`. Unlike Huffman coding, which assigns fixed binary codes to component symbols separated from an input, arithmetic coding represents multiple symbols with a single floating-point number *q* which must be within the range `0.0 ≤ q < 1.0`. Arithmetic coding is particularly effective when the probability distribution of the symbols is skewed, as it can produce a more compact representation than Huffman coding. It is usually slower than Huffman coding. Let's return to our traffic light example. Let's say you aren't satisfied with our previous Huffman coding result, which has produced an average of 1.6 bits per symbol, and you'd like to use arithmetic coding instead. In arithmetic coding, each of our symbols should first be placed on a range within the interval `[0, 1)` based on its probability. Then, we can narrow down this range as we encode more symbols, eventually arriving at a single number that represents the entire sequence. The steps follow below. Let's use the same probabilities from before: - Green (50%) - Red (40%) - Yellow ( 9%) - Disabled ( 1%) We'll assign ranges to each symbol as follows: - Green: `[0.00, 0.50)` - Red: `[0.50, 0.90)` - Yellow: `[0.90, 0.99)` - Disabled: `[0.99, 1.00)` Now, let's encode a sequence of traffic light states: "Green, Red, Yellow, Green" 1. Start with the interval `[0, 1)` 2. First symbol (Green): - `new_low = low + (high - low) • cumulative(s) / total` - `new_high = low + (high - low) • (cumulative(s) + prob(s)) / total` - `new_low = 0 + (0.5 - 0) • 0 / 1.0` - `new_high = 0 + (0.5 - 0) • (0 + 0.5) / 1.0` - Narrow range to `[0.00, 0.50)` 3. Second symbol (Red): - From previous range: `[0.00, 0.50)` - `new_low = 0 + (0.50 - 0) * 0.50 / 1` - `new_high = 0 + (0.50 - 0) * (0.50 + 0.40) / 1` - Red is in the range `[0.25, 0.45)` 4. Third symbol (Yellow): - From previous range: `[0.25, 0.45)` - `new_low = 0.25 + (0.45 - 0.25) * (0.50 + 0.40) / 1` - `new_high = 0.25 + (0.45 - 0.25) * (0.50 + 0.40 + 0.09) / 1` - Yellow is in the range `[0.43, 0.448)` 5. Fourth symbol (Disabled): - From previous range: `[0.43, 0.448)` - `new_low = 0.43 + (0.448 - 0.43) * (0.50 + 0.40 + 0.09) / 1` - `new_low = 0.43 + (0.448 - 0.43) * (0.50 + 0.40 + 0.09 + 0.01) / 1` - Green is in the range `[0.44782, 0.448)` The final interval is `[0.44782, 0.448)`. Any number in this range (we'll pick the lower bound, which is inclusive of 0.44782) can represent our entire sequence "Green, Red, Yellow, Disabled". To decode, we would start with 0.44782 and use our original probability ranges to determine which symbol it corresponds to, then update the value and repeat the process. In this example, we aren't saving any space by using arithmetic coding because our sample is too short to have any pattern to effectively exploit. With longer sequences, arithmetic coding approaches the theoretical entropy limit of **1.408 bits per symbol**: `-(0.50 * log2(0.50) + 0.40 * log2(0.40) + 0.09 * log2(0.09) + 0.01 * log2(0.01)) ≈ 1.408` It is important to note that in practice, there often are additional considerations not present in this simplified example with regard to managing precision. 4. **Prediction and Residual Encoding**: Prediction involves using previous data to predict future data. The difference between the predicted and actual data (residual) is encoded instead of the actual data. Linear Predictive Coding (LPC) is a common method where a linear function of previous samples is used to predict the current sample. The residuals typically have lower entropy and can be encoded more efficiently. --- # Lossy Compression Canonical URL: https://encode.wiki/start-here/lossy/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 44f88bdaff83ef7ac887da3fa41ef84d43f3aaed6cdefd6f8f059e43717ed2b9 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Lossy multimedia compression reduces the file size of multimedia data by permanently removing some of the information. This process leverages the limitations of the human senses, fidelity metrics, or appeal metrics to discard information that is considered less salient. The goal of lossy compression is to reduce the file size of multimedia data while maintaining a desired level of quality. ## How Lossy Compression Works Lossy compression works by analyzing the input signal and removing parts of it that are less salient. Some processes by which this is done include: - **Perceptual Coding** (audio): This technique removes audio frequencies that are outside the range of human hearing or masked by other sounds. - **Quantization** (audio): This process reduces the precision of certain audio components, which can significantly reduce file size without a noticeable impact on perceived quality. *More coming soon* --- # psychovisual Canonical URL: https://encode.wiki/start-here/psychovisual/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: aa4fd29272d32a6d26cfb211841f80845664f7797b40bd695544cd7e7f82b3f0 "Psychovisual fidelity" is a common term used in video encoding to describe the quality of an encoded video as perceived by the human visual system. It has a number of alternate terms that mean the same thing: - "Perceptual quality" - "Subjective quality" - "Visual quality" Psychovisual options are introduced into encoders to combat the limitations of traditional decision-making inside video encoders, which tends to prioritize efficiency as measured by simple metrics like [PSNR](/reference/metrics/psnr/). The amount of care and attention an encoder directs towards preserving psychovisual fidelity is often indicative of an encoder's maturity, and is a key reason why mature encoders like [x264](/tools/software-encoders/x264/) and [x265](/tools/software-encoders/x265/) are so widely regarded as being well-designed and effective. ## Explanation Via the [x265 documentation](https://x265.readthedocs.io/en/master/cli.html#psycho-visual-options): > Left to its own devices, video encoders will make mode decisions based on a simple rate distortion formula, trading distortion for bitrate. This is generally effective except for the manner in which this distortion is measured. It tends to favor blurred reconstructed blocks over blocks which have wrong motion. The human eye generally prefers the wrong motion over the blur and thus x265 offers psycho-visual adjustments to the rate distortion algorithm. An important concept that is essential to understanding psychovisual fidelity is the idea behind *visual energy*. While the x265 documentation snippet above correctly identifies that the human eye prefers wrong motion over blur, a similar concept applies in the case of detail preservation. The human eye is more forgiving in the presence of detail – even if it is wrong detail – than in the presence of blur due to our preference for visual energy. Psychovisual options are often tailored to preserving visual energy in the encoded video. ### Fidelity Versus Appeal Via [Cloudinary's blog](https://cloudinary.com/blog/what_to_focus_on_in_image_compression_fidelity_or_appeal): > Fidelity in images is about visually preserving the original; appeal is about hiding the compression artifacts. Depending on your priority, you would compress images with either of these approaches to reduce the file size while still maintaining a reasonable level of visual “quality” ... Many people report that they prefer blurring in videos over blocking artifacts, which is a conflation of the value of fidelity versus appeal. While blurring can be beneficial at lower bitrates to help reduce the visibility of blocking artifacts, it is generally preferable to view a slightly blurry image rather than a blocky one with *distracting* artifacts. This underscores how essential it is to strike a balance with psychovisual options in encoders; heavily utilizing these settings can negatively effect visual quality in certain scenarios, like when encoding at low fidelity where artifacts would be erroneous enough that they would be distracting. ### Psychovisual Options The following are some common psychovisual options that are available in x265, from the x265 documentation linked prior: > `--psy-rd` will add an extra cost to reconstructed blocks which do not match the visual energy of the source block. The higher the strength of `--psy-rd` the more strongly it will favor similar energy over blur and the more aggressively it will ignore rate distortion. If it is too high, it will introduce visual artifacts and increase bitrate enough for rate control to increase quantization globally, reducing overall quality. psy-rd will tend to reduce the use of blurred prediction modes, like DC and planar intra and bi-directional inter prediction. > `--psy-rdoq` will adjust the distortion cost used in rate-distortion optimized quantization (RDO quant), enabled by `--rdoq-level` 1 or 2, favoring the preservation of energy in the reconstructed image. `--psy-rdoq` prevents RDOQ from blurring all of the encoding options which psy-rd has to choose from. At low strength levels, psy-rdoq will influence the quantization level decisions, favoring higher AC energy in the reconstructed image. As psy-rdoq strength is increased, more non-zero coefficient levels are added, and fewer coefficients are zeroed by RDOQ's rate distortion analysis. High levels of psy-rdoq can double the bitrate which can have a drastic effect on rate control, forcing higher overall QP, and can cause ringing artifacts. psy-rdoq is less accurate than psy-rd, it is biasing towards energy in general while psy-rd biases towards the energy of the source image. But very large psy-rdoq values can sometimes be beneficial. In summary, `--psy-rd` attempts to preserve the visual energy of the encode relative to the source video, while `--psy-rdoq` attempts to make decisions generally in favor of preserving visual energy. Currently, [x264](/tools/software-encoders/x264/) includes both of the above options alongside a number of other psychovisual options, and is considered the most mature psychovisually optimized open source video encoder. While it doesn't have `--psy-rdoq`, [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/) is an AV1 encoder that includes a number of psychovisual options (including `--psy-rd`, as of version 3.0.0). ### Metrics Currently, it is difficult to measure the effectiveness of psychovisual options in encoders without subjective visual testing. While metrics like [VMAF](/reference/metrics/vmaf/) and [SSIM](/reference/metrics/ssim/) are designed to be more perceptually accurate than PSNR, they are far from perfect and are usually misleading. More modern metrics like [SSIMULACRA2](/reference/metrics/ssimulacra2/) and [XPSNR](/reference/metrics/xpsnr/) are designed to be more perceptually accurate than others, and while SSIMULACRA2 largely achieves this goal, newer metrics are still not perfect and usually reward disabling all psychovisual options even when the options clearly help preserve psychovisual fidelity. ## Conclusion Psychovisual options are essential to preserving visual quality. While perceptually driven features are hard to develop, difficult to test and measure, and somewhat rare, they are generally considered to be beneficial to the encoding process. The inclusion of well-built psychovisual options in encoders often signifies maturity and effectiveness, contributing to the overwhelmingly positive legacies of encoders like x264 and x265 which have consistently prioritized visual quality to great effect. --- # Terminology Canonical URL: https://encode.wiki/start-here/terminology/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 552d28982641ee78c041f6a8c240e8177cec0df0b424a8e8eba18fd2f6946949 When learning about encoding technology, it is important to understand the vast terminology that is often used to describe concepts that are often not very complex to understand. ## Bitstream A *bitstream* or *bit stream* is a media file, the kind that is played in a media player. It consists of a [container](#container) wrapping multiple [elementary streams](#elementary-stream) ## Lossy / Lossless *Lossy* encoding throws out some of the detail to achieve a smaller size. Often, this is an acceptable trade-off, but if you need a perfect recreation of the data, you need *lossless* encoding. ## Elementary stream An elementary stream is an audio, video, or subtitle track. Basically, it's the compressed data you want to [mux](#muxing) into the container. ## Muxing Putting elementary streams into a container, which preserves them without making any changes to the data. ## Codec A codec (**co**der/**dec**oder) is the piece of code that actually encodes the data you put in. It takes as input and produces as output an elementary stream. More information is provided [in the prologue](/start-here/encoding-basics/#what-is-a-codec) under "What is a Codec". ## Filter A filter is a piece of code you can apply to the data to make something about it different, for instance sharpening, removing artifacts, shakiness, denoising, scaling, overlay, etc. ## Muxer/Demuxer The pieces of code that [mux](#muxing) or do the reverse, getting elementary streams from the container. ## Bitstream filter A bitstream filter is a filter that is directly applied to the [bitstream](#bitstream) in order to change something about the container, for instance, convert frame types, or corrupt some packets. ## Container A container is a format for putting one or more elementary streams into one file, which is then called a [bitstream](#bitstream). A video container is a digital file format that holds video and audio data, as well as additional information such as subtitles, metadata, and chapter markers. It acts as a "wrapper" that packages all these elements into a single file that can be played on various devices and software platforms. Think of it like a container you might use to transport goods - the video and audio data are like the items being transported, while the container itself provides a structure and organization for the contents. Some kinds of containers: #### MP4 / M4V This is likely the most common container you've encountered, & has near universal compatibility. Has a limited maximum amount of streams. The supported video codecs are [H.264](/codecs/video/avc/), [H.265](/codecs/video/hevc/), [H.266](/codecs/video/vvc/), DivX, Xvid, [VP9](/codecs/video/vp9/) (Unofficial, hacky), and [AV1](/codecs/video/av1/) (Unofficial, hacky). For audio codecs it's many of the various flavors of [AAC](/codecs/audio/aac/), [MP3](/codecs/audio/mp3/), [FLAC](/codecs/audio/flac/) (Unofficial), [Opus](/codecs/audio/opus/) (Unofficial, hacky). For subtitles only MPEG-4 Timed Text (TTXT) is supported. The best tool to work with this container is MP4Box, but FFmpeg also works. #### MOV Similar to MP4, but less supported. Made with Apple Quicktime in mind, supports ProRes. #### MKV / MKA / MKS / MK3D Also known as Matroska, allows an unlimited amount of video/audio/subtitle streams and any codec that probably still exists in Area 51, you can put literally anything in there and it won't even care, MPEG-2/DivX/H.266/Theora/Thor/RealVideo/MJPEG/AVS3/AMR-WB, you name it. All around best container for working with if you have the choice. #### WebM A container made with web streaming in mind. WebM is a stripped-down subset of MKV that only allows free & open source codecs such as VP8, VP9 or AV1 for video alongside Vorbis or Opus for audio. It is a common misconception that [WebVTT](/codecs/subtitles/webvtt/) tracks always work natively in browsers when within a WebM container; in practice, WebMs containing WebVTT subtitles *will usually not* play back the subtitles in browsers. WebVTT subtitles can be utilized with the `` element instead, meaning they exist outside the WebM container itself. More info in the [Mozilla Web Docs](https://developer.mozilla.org/en-US/docs/Web/Media/Audio_and_video_delivery/Adding_captions_and_subtitles_to_HTML5_video). ## Transcoding Taking an elementary stream & converting it to another format, lossless or lossy, using an encoder of some kind. For example, if I convert a lossless [FFV1](/codecs/video/ffv1/) video to a lossy AV1 video using an encoder like [rav1e](/tools/software-encoders/rav1e/), I have *transcoded* this lossless video to AV1. Transcoding doesn't have anything to do with the container. ## RDO RDO, or Rate-Distortion Optimization, is a technique used to find the best trade-off between the bit rate & the quality of lossily encoded content. RDO can be metric-based, optimizing to score well on metrics like [PSNR](/reference/metrics/psnr/) or [SSIM](/reference/metrics/ssim/). ## Perceputal / Psychovisual / Psychoacoustic "Psychovisual quality" (for videos), "Psychoacoustic quality" (for audio), or "perceptual quality" is a term used to describe the perception of quality of distorted media by the human senses. The goal of any multimedia codec is to minimize data while maintaining perceived quality, and optimizing around human perception theoretically yields the best performance within any limited set of coding techniques (like when using an older codec). Our model of human perception continues to evolve, which makes modelling perceptual quality very difficult. Presently, the metrics [SSIMULACRA2](/reference/metrics/ssimulacra2/) (Images/Video) & [Butteraugli](/reference/metrics/butteraugli/) (Video) are considered the most accurate to our human visual system. ## Discrete Cosine Transform (DCT) The Discrete Cosine Transform is a mathematical transformation that can transform discrete data into the frequency domain. This discrete data could be pixels in an image/video compression block or data points recorded temporally representing an audio recording. This algorithm is a particularly good choice for image, video, music, & speech compression because it has high energy compaction relative to our understanding of fidelity in media. High energy compaction means the DCT is able to represent a signal with a small number of significant coefficients. --- # video-artifacts Canonical URL: https://encode.wiki/start-here/video-artifacts/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 5b878e5a93601b9c2bdde4e4f77ccce3b379989e06728e0bc63d1c1f41c08bc8 # Spotting Video Artifacts XKCD about compression artifacts *https://xkcd.com/2414* Video artifacts are visual distortions or anomalies that appear in a video, which can affect the quality of the image. Artifacts are first categorized by whether they’re time/sequence-based (temporal) or location-based (spatial). If you can see the artifact when the video is paused, then it’s probably a spatial artifact. If it’s much more visible while the video plays, then it’s likely temporal. ## Moiré Pattern Moiré pattern on a brick building Moiré patterns are large-scale spatial interference pattern produced when a pattern in the source image and the manner in which the encoder operates are slightly out of alignment spatially. The artifacts generated by the encoder then introduce strange, swirling effects in the source image's pattern upon decoding. ## Staircase Effect Staircase Effect The staircase effect is a spatial artifact that occurs when diagonal straight or curved edges that should be smooth take on a jagged appearance, looking somewhat like a set of stair steps. This is the effect that is being reduced by "anti-aliasing" filters. ## Color Bleed Color Bleed Color bleeding, as its name suggests, occurs when the edges of one color in the image unintentionally bleeds or overlaps into another color. The cause of color bleeding is usually related to the compression algorithm's handling of chrominance information. Chrominance refers to the color information in an image or video, and it's usually compressed separately from the luminance (brightness) information. In some compression algorithms, the chrominance information may be compressed more aggressively than the luminance information, resulting in a loss of color accuracy and detail. ## Ringing Ringing artifact on a star symbol Note the blue and pink fringes around the edges of the star above (as well as the stepping and other significant compression artifacts). Those fringes are the ringing effect. It is very similar to mosquito noise which will be talked about down below. ## Blocking Blocking artifacts on a cat To understand this artifact, we first need to know what DCT (Discrete Cosine Transform)-based compression is. It's a way of reducing the size of a digital image or video file by analyzing the data in small chunks (called "blocks") and then using mathematical formulas to transform that data into a more compact representation. This transformed data can then be stored or transmitted more efficiently. However, this compression technique can sometimes lead to visible "distortions" in the compressed image or video. One of the most common types of distortion is the "blocking artifact," which occurs when the edges of the compressed blocks become visible, creating a grid-like pattern over the image or video. In simpler terms, imagine if you take a picture and then try to compress it to make the file size smaller. The compression algorithm divides the picture into small blocks and tries to simplify the information in each block. But when there's not enough bitrate/data, the algorithm simplifies the information too much, resulting in visible blocks or a grid pattern on the picture. This is what we call the DCT blocking compression artifact. ## Banding/Contouring Banding Banding or contouring is a specific form of posterization in which the color blocks form bands or stripes in the image. This occurs when the video is encoded with too coarse a quantization (Technique used in video compression that reduces the amount of data needed to store or transmit a video by rounding off certain values in the video's signal) configuration and a lack of available color gradient. As a result, the video's contents show a "layered" look, where instead of smooth gradients and transitions, the transitions from color to color are abrupt, causing strips of color to appear. ## Mosquito Noise Mosquito noise surrounding an iron fence Mosquito noise appears as small, moving, and blocky artifacts around sharp edges or fine details in videos or images. They are mostly found in videos that are compressed using the DCT algorithm we talked about earlier, such as MPEG videos or JPEG images. --- # Tools Canonical URL: https://encode.wiki/tools/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 7227e2e87dfd6dbcc8734b19e030f98296e68116d50252631ae3f787789e1cc6 Tool pages document software and hardware encoders, filters, media players, muxers, inspectors, and utilities. Version-sensitive instructions carry a last-verified date and tested version information. --- # Antialiasing Canonical URL: https://encode.wiki/tools/filtering/antialiasing/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Basics of Filtering Canonical URL: https://encode.wiki/tools/filtering/basics/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: c0095af1e63aa7b1ca1b548e88737c51a16ae7a549fabae55b7800af058b582e # Basics of Filtering with Vapoursynth ## Intro to Filters In order to do filtering, we are going to need some filters. Vapoursynth includes some basic filters, but there are a lot more useful filters built by the community that can be incredibly useful to us. For example, we are going to make use of some of the functions in [vs-tools](https://github.com/Jaded-Encoding-Thaumaturgy/vs-tools). If you are on Arch, the recommended way to install this is via the AUR, using the `vapoursynth-plugin-vstools-git` package. For other operating systems, it is simpler to install all of the JET plugins at once, using the [vs-jet utility](https://github.com/Jaded-Encoding-Thaumaturgy/vs-jet). Note that these do require Python, which you should already have installed if you've installed Vapoursynth. You may have noticed in the previous section that all Vapoursynth scripts use Python syntax. It is not necessary to know Python in order to write Vapoursynth scripts, although it can be helpful for some more advanced techniques. ## Filter Order Order matters when applying filters to a video. There are certain actions that must be taken at the very start, and certain actions that are better to be done later in a filter chain. The following image shows a recommended order for many of the most common filter actions: ![filter chain](/img/vs2.webp) If this seems a bit complicated at first, that's okay. We will learn about each of these steps one at a time, and when each of these filters might be needed. ## Bit Depth and Colorimetry In most cases, the first action we want to take on a video is to ensure the correct [matrix, primaries, and transfer characteristics](/reference/colorimetry/intro/) are set, and convert the video to 16-bit. The colorimetry properties are important for ensuring accurate colors, and 16-bit helps filters to work in higher quality and create fewer artifacts. The currently recommended way to handle this is with the `vstools.initialize_clip` function. This function will handle both ensuring that colorimetry is set, and converting to 16-bit with appropriate dithering if needed. We can add it to a script in the following way: ```python import vapoursynth as vs core = vs.core clip = core.lsmas.LWLibavSource(source="myinput.mkv") import vstools clip = vstools.initialize_clip(clip) clip.set_output(0) ``` If the video has existing colorimetry data set on it, `initialize_clip` will keep the existing data. Otherwise, it will assume colorimetry based on the video resolution. Usually, those assumptions will be correct. However, there may be cases where we know that the assumptions are incorrect, and we need to set them manually. For example, we may have a 4k video that we know was upscaled from a 1080p video, and we want to force BT.709 colorimetry. In that case, we can specify the properties to `initialize_clip`, and it will use the properties we specify instead of making guesses. ```python from vstools import Primaries, Matrix, Transfer clip = vstools.initialize_clip(clip, primaries=Primaries.BT709, matrix=Matrix.BT709, transfer=Transfer.BT709) ``` `initialize_clip` also has a companion function called `finalize_clip` which will, by default, convert the video down to 10-bit with dithering, and if the video is in limited color range, clip all pixels to fit within the limited color range (this would be done by the player on playback, but if we do it now, we can save a few bits for the encoder). The function also takes a `bits` parameter in case you would like to finalize to 8-bits instead. As a result, a very basic template for a Vapoursynth script would be something like this: ```python import vapoursynth as vs core = vs.core clip = core.lsmas.LWLibavSource(source="myinput.mkv") import vstools clip = vstools.initialize_clip(clip) // // Add filters inside this section // clip = vstools.finalize_clip(clip) clip.set_output(0) ``` ## Cropping ![screenshot](/img/crop_1.jpg) Sometimes, you may encounter a source that has black bars on the sides or on the top and bottom. Generally, we don't like to keep these, and want to remove them. This process is called "cropping". First, we need to figure out how many pixels to crop from each side. Fortunately, vs-preview has a tool to make this easy. Using this tool, which is found under the "Misc" section in the bottom right of the app, allows us to experiment with crop values and find the correct ones without having to reload the Vapoursynth script. ![crop tool](/img/crop_tool.webp) Once we flip the tool to "On", we can adjust each of the dimensions until we see the black borders disappear. We want to try to get as exact as possible, although the crop values must be a multiple of 2 when working with YUV420 video (which is the most common). In this case, we know we want to remove from the top and bottom, so we will adjust those values until we find the right ones. Note that the size of the bars may often be the same on both sides, but not always, so do be careful to check both sides. ![zoom tool](/img/zoom_tool.avif) vs-preview's zoom tool shown here can also be helpful to ensure we've gotten the right values. Once we've found the correct values, we can click "Copy cropping command" to copy our value to the clipboard. We can then paste it into our Vapoursynth script, being sure to add the appropriate variable name for our clip. ```python clip = clip.std.Crop(0, 0, 138, 138) ``` Remember to turn off the crop tool in vs-preview, or else we'll be doubling the crops from here on. Now, go ahead and reload the preview with Ctrl+R. No more black bars. ![screenshot](/img/crop_2.jpg) ## Resizing Resizing is the process of changing the resolution of a video, whether that's making it bigger or smaller. Note that although the general idea is that "higher resolution is better", we generally don't want to upscale content. This simply makes output videos larger in file size, but doesn't result in an increase in quality, because the higher resolution is artificial. However, there are times when you may want or need to resize a video. Unfortunately, this is not as simple as it sounds, because we want to select a high-quality resizing method. With this section, we'll introduce [slow.pics](https://slow.pics/), a very useful tool for comparing screenshots. Here, we'll be using it to show the differences between various resizers. Vapoursynth comes with several resizers built-in, which will be the easiest to use. Of these, we'll be showcasing the following: ```python clip = clip.resize.Bicubic(1920, 1080) clip = clip.resize.Lanczos(1920, 1080) clip = clip.resize.Spline36(1920, 1080) ``` Bicubic is the most widely used resizer. It is very fast and produces decent quality, but may produce less sharpness than more advanced scalers. Lanczos produces more sharpness, but may also produce more [ringing](/tools/filtering/dehalo/). Spline36 uses a spline-based resizing method, and produces results somewhere between Bicubic and Lanczos. However, using plugins, we have access to some more advanced scalers. ```python clip = vskernels.Catrom().scale(clip, 1920, 1080) clip = vskernels.Catrom(sigmoid=True).scale(clip, 1920, 1080) clip = vskernels.Hermite(linear=True).scale(clip, 1920, 1080) clip = vsscale.SSIM().scale(clip, 1920, 1080) ``` Here we will look at downscalers. Catrom, short for Catmull-Rom, is a variant of Bicubic which uses different parameters in order to achieve a better balance between sharpness and ringing. It is often considered the best of the Bicubic kernels for image quality. In the second example, you can see that we pass an additional `sigmoid=True` parameter to `Catrom`. This performs the scaling using a sigmoid function, which flattens the ends of the scaling curve to reduce ringing. The third example here, Hermite, is another variant of Bicubic, and we are passing `linear=True` to perform the rescaling in linear light rather than in gamma light. This kernel is very good in regard to not producing ringing, but it does have the effect of thinning line art. Therefore, although some users prefer it, other users dislike it. Fourth, we have SSIM, which is a downscaler tuned around the SSIM video quality metric. This downscaler is intended to prioritize preserving detail and sharpness when downscaling. In [this comparison](https://slow.pics/c/mb5Rm3k3), we can see the differences between the different scalers, on an image that was upscaled to 4k by nnedi3, then downscaled back using the scaler noted in the image title. Here we have a couple of other [examples](https://slow.pics/c/kQvDOMYh) comparing in particular the [higher-quality scalers](https://slow.pics/c/UVwS62Vs). Of these, my current recommendation would be `Catrom(sigmoid=True)` for anime, and `SSIM` for live action. Some users may prefer `Hermite(linear=True)` for anime. For upscaling, we have the following high quality options: ```python clip = vskernels.EwaLanczos().scale(clip, 3840, 2160) clip = vsaa.Nnedi3().scale(clip, 3840, 2160) clip = vsscale.Waifu2x().scale(clip, 3840, 2160) ``` Ewa-Lanczos is a variant of lanczos intended for high sharpness, making it good for upscaling, where we want an image to look sharper so that it does not appear upscaled. Nnedi3 is a neural-network-based upscaler which attempts to interpolate every second line to produce a sharp, upscaled image. It is generally safe for use on all content, and produces good results. Waifu2x is another AI-assisted upscaler, this one tuned specifically on anime. It produces very high quality and sharp images. However, the results of using it on live action content are unknown, so it is recommended to only use it on anime. [Here](https://slow.pics/c/CpISIk1z) we can see the effects of these upscalers. My current recommendation would be `EwaLanczos` for live action, and `Waifu2x` for anime. ```python clip = soifunc.good_resize(clip, 1920, 1080) clip = soifunc.good_resize(clip, 1920, 1080, anime=True) ``` Lastly, we have `soifunc.good_resize`, which is a smart hybrid scaler which automatically chooses the best scaler depending on whether we are upscaling or downscaling, and may use a different scaler for chroma than it does for luma for even better quality. Note that the scalers chosen by this function may change over time as new knowledge surfaces, and are based on the above recommendations. We can also add the `anime=True` parameter (false by default) to change the scalers to be tuned more toward the fine line art that is present in anime, rather than the highly detailed elements such as skin and hair that are present in live action. With these features, it is intended to be an easy-to-use general purpose high-quality upscaler. ## Trimming Sometimes, we may need to cut out a portion of the video, for example, if we want to encode only the opening theme, or if our source has commercials that we want to remove. Vapoursynth supports this using Python's slice syntax. ```python clip = clip[1805:3185] ``` This snippet will trim the clip to include frames 1805-3184. Note that frame 3185 is _not_ included. Now, let's say we wanted to _exclude_ this segment. We can do this by making clips of the segments we'd like to keep, and joining them together. ```python clip = clip[:1805] + clip[3185:] ``` This snippet will create trims for the start of the clip to and including frame 1804, and from frame 3185 to the end of the clip. It then joins them together into one clip, in the order we specify. If we wanted to reverse the segments, so that segment 2 comes first, we could do that simply by changing the order: ```python clip = clip[3185:] + clip[:1805] ``` --- # Deband Canonical URL: https://encode.wiki/tools/filtering/deband/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Decombing Canonical URL: https://encode.wiki/tools/filtering/decombing/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Dehalo Canonical URL: https://encode.wiki/tools/filtering/dehalo/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Deinterlace Canonical URL: https://encode.wiki/tools/filtering/deinterlace/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Denoise Canonical URL: https://encode.wiki/tools/filtering/denoise/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 2606d3eff8e26593b8c880fb49bbecf5a4f6b83be9a1adc3f8395b4f5b9b37a2 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! ## Overview Denoising involves removing random noise from the video. Such noise can result from film grain, signal interference or simply low light conditions. In any case, noise can greatly reduce compression efficiency especially if the video codec doesn't support film grain synthesis. In [FFmpeg](/tools/utilities/ffmpeg/) there are two filters available for denoising: - [hqdn3d](#hqdn3d) - [nlmeans](#nlmeans) ## hqdn3d hqdn3d is a fast, high quality 3d denoising filter which improves compressibility. Can be applied to images and videos. ### Usage ```shell ffmpeg -i input.mp4 -vf hqdn3d output.mp4 ``` The default configuration should be fine for most use cases. If you still see too much noise you can adjust the `luma_spatial` parameter (other parameters are derived from it by default). Higher `luma_spatial` value will result in stronger denoising. By default it is set to `4`. ```shell ffmpeg -i input.mp4 -vf hqdn3d=8 output.mp4 # which is the same as ffmpeg -i input.mp4 -vf hqdn3d=8:6:12:9 output.mp4 ``` > **Caution — Setting `luma_spatial` to larger values could result in ghosting and [banding](/start-here/video-artifacts/#bandingcontouring) artifacts..** > > For description of all four parameters take a look [here](https://ffmpeg.org/ffmpeg-filters.html#hqdn3d-1). ## nlmeans nlmeans uses Non-Local Means algorithm to do denoising. Each pixel is compared to similar pixels based on their surroundings (context). The size of such context is expressed as `r`x`r`. The filter is rather slow and doesn't parallelize well. Only use it in cases the video contains a lot of noise or you need very high quality denoising. In all other cases [hqdn3d](#hqdn3d) will be more efficient. ### Usage ```bash ffmpeg -i input.mp4 -vf nlmeans output.mp4 ``` The default configuration should be fine for most use cases. ```bash ffmpeg -i input.mp4 -vf nlmeans=s=3.0:r=31:p=15 output.mp4 ``` Stronger denoising with larger research and patch size. Might be useful for ultra high quality denoising in 4K+ resolutions but you might struggle to achieve even 0.1 fps. ```bash ffmpeg -i input.mp4 -vf nlmeans=s=1.0:r=5:p=3 output.mp4 ``` Prioritize speed over quality. ### Parameters - `s` - Denoising Strength where `1.0` is the lightest and also the default and the strongest is `30.0` although I wouldn't recommend going above `10.0`. - `r` - Research Size where `15` is the default, it must be an odd number ranging from `0` to `99`. The higher the value, the slower denoising will be. - `p` - Research Size where `7` is the default and, it must be an odd number ranging from `0` to `99`. For description of all possible parameters take a look [here](https://ffmpeg.org/ffmpeg-filters.html#nlmeans-1). # Notes - [hqdn3d](#hqdn3d) may create visual artifacts like ghosting, [banding](/start-here/video-artifacts/#bandingcontouring) and [blocking](/start-here/video-artifacts/#blocking) - [nlmeans](#nlmeans) creates much less noticeable artifacts like cartoonish look but only for very noisy inputs --- # Graining Canonical URL: https://encode.wiki/tools/filtering/graining/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Inverse Telecine Canonical URL: https://encode.wiki/tools/filtering/ivtc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0cd80b9330a438df5d6292fce4b5bdb1c3c0a4265cb8d2d49820b77295c5ebc > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! --- # Stabilizing Canonical URL: https://encode.wiki/tools/filtering/stabilizing/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 5797f192b3a58b4a61e57768b3c9e53c8a00e77a5183da80c0f25f1c89479d7c > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! ## Overview Stabilizing is the process of reducing unwanted camera movement and shakes in video clips using [FFmpeg](/tools/utilities/ffmpeg/). This improves overall encoding efficiency by minimizing unpredictable global movement, such as that from handheld cameras. The recommended method for stabilizing videos with FFmpeg is to use the VidStab library, which requires a build of FFmpeg compiled with `--enable-libvidstab`. VidStab offers two filters within FFmpeg: ```shell ffmpeg -hide_banner -filters | grep vidstab ... vidstabdetect V->V Extract relative transformations, pass 1 of 2 for stabilization (see vidstabtransform for pass 2). ... vidstabtransform V->V Transform the frames, pass 2 of 2 for stabilization (see vidstabdetect for pass 1). ``` The `vidstabdetect` filter is used in the first pass to generate a video transformations file (`.trf`), while `vidstabtransform` is employed in the second pass to apply those transformations. ## Usage To stabilize a video using default parameters, follow these two steps: ```shell ffmpeg -i input.mp4 -vf vidstabdetect -f null - ffmpeg -i input.mp4 -vf vidstabtransform output.mp4 ``` After running the first command, a `transforms.trf` file will be created in the directory where you executed FFmpeg. Once the stabilization process is complete, you can safely delete this file. The resulting `output.mp4` video will have reduced shakiness. For stabilizing high-framerate videos with strong camera movement: ```shell ffmpeg -i input.mp4 -vf vidstabdetect=shakiness=8:result=a.trf -f null - ffmpeg -i input.mp4 -vf vidstabtransform=smoothing=30:zoom=-5:input=a.trf output.mp4 ``` > **Tip — Remember to set appropriate video/audio codec parameters in the command before `output.mp4`. You must not use `-c:v copy`, as the video will undergo transformations..** > > ### vidstabdetect Parameters - `result` - Sets the output `.trf` file location - `shakiness` - Adjusts movement reduction, with `1` being the least and `10` the most reduction (highest stabilization). Default is `5`. - `accuracy` - Controls movement reduction accuracy. Lower values use less CPU but may be less accurate. FFmpeg's minimum allowed value is `3`. Processing speed was approximately `21 fps` at `3` and `14 fps` at `15`. For a complete list of parameters, refer to the [vidstabdetect documentation](https://ffmpeg.org/ffmpeg-filters.html#vidstabdetect-1). ### vidstabtransform Parameters - `input` - Specifies the input `.trf` file created by `vidstabdetect` - `smoothing` - Determines the number of frames considered for future and past movement estimation. Default is `10`. - `zoom` - Adjusts the zoom percentage, with `0%` being the default. Negative values create a zoom-out effect. - `interpol` - Sets the type of interpolation used: - `no` - No interpolation - `linear` - Only horizontal - `bilinear` - Faster but may result in blurry output (default) - `bicubic` - Slower See the [vidstabtransform documentation](https://ffmpeg.org/ffmpeg-filters.html#vidstabtransform-1) for more details. ## Notes - Stabilization is a lossy process that can reduce video quality due to zoom and interpolation effects. - Some users may notice overall wobbliness in stabilized videos, especially at higher stabilization levels. This is an inherent characteristic of this filter. - Depending on your use case, consider employing two-pass encoding along with these stabilization steps. --- # Vapoursynth Canonical URL: https://encode.wiki/tools/filtering/vapoursynth/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 29ceb3d2de964d0b2c5dc45d95b48e393a1b8b8f9d971b23d583d4b9f71e9c02 > VapourSynth is an application for video manipulation. Or a plugin. Or a library. It’s hard to tell because it has a core library written in C++ and a Python module to allow video scripts to be created. *Fredrik Mellbin, creator of VapourSynth* Vapoursynth script {" "} # Introduction In the realm of video processing, one will frequently encounter media with various quality issues. These can range from minor imperfections to significant degradation, including: - Excessive film grain or noise that significantly increases bitrate due to its unpredictable nature - Visible banding artifacts - Unwanted halos around objects - Interlacing issues in older, unrestored footage - Telecine artifacts from improper film-to-video conversion - And more, on our [video artifacts](/start-here/video-artifacts/) page. To address these challenges, video filtering techniques are employed. Currently, there are three primary software frameworks used for video filtering: 1. [FFmpeg](/tools/utilities/ffmpeg/) 2. [VapourSynth](https://vapoursynth.com) 3. [AviSynth](http://avisynth.nl/index.php/Main_Page) VapourSynth is designed as a 21st-century upgrade and rewrite of AviSynth, which was originally created by Ben Rudiak-Gould, Edwin van Eggelen, Klaus Post, Richard Berg, and Ian Brabham in May 2000. One of the most attractive features of this complete rewrite is its improved multithreading capability, an area where AviSynth struggled due to its aging infrastructure. Some longtime AviSynth users are reluctant to switch to VapourSynth, preferring to stick with a familiar workflow. There is nothing wrong with this preference, as both tools have their merits in video processing; that being said, the Codec Wiki's *Filtering* section focuses primarily on VapourSynth (and occaisonally FFmpeg). It is important to note that working with VapourSynth requires a basic understanding of Python, as the filtering process involves scripting. ## Installation ### Microsoft Windows - At the time of writing, Python 3.12 is required. This will change in the future so consult from [their website](http://www.vapoursynth.com/doc/installation.html) - Download the installer (`.exe`) unless you require portability - Install it ### Arch Linux Currently, Arch is the best Linux distribution for working with Vapoursynth due to the fact that the vast majority of filters and plugins are available in the AUR. This makes installing and updating filters easy. If you are not already, we recommend using an AUR helper such as [paru](https://github.com/Morganamilo/paru) or [yay](https://github.com/Jguer/yay). To install vapoursynth, simply install the `vapoursynth` package from the official repositories using pacman or your preferred AUR helper. Plugins are all prefixed with `vapoursynth-plugin-`, such as `vapoursynth-plugin-lsmashsource-git`, and as such can be discovered easily. ### Other Linux Vapoursynth is supported on all Linux distributions. Installation methods may vary by distribution. Contributions would be helpful to provide instructions for more distributions. ## Previewing There are currently two leading previewers for Vapoursynth. If you want to preview your scripts with capabilities such as seeking, you will need to use one of these applications. The first is YomkioR's [Vapoursynth Editor](https://github.com/YomikoR/VapourSynth-Editor/), which includes a built-in code editor alongside a video previewer. This makes it extremely easy to set up for users who are new to Vapoursynth. The second is JET's fork of [vs-preview](https://github.com/Jaded-Encoding-Thaumaturgy/vs-preview), which is a standalone previewer with utilities for tasks such as cropping, screenshotting, and uploading comparisons. This tool is more advanced than Vapoursynth Editor, but does not include an editor, so you will need to pair it with an editor such as Visual Studio Code. The Github for vs-preview includes instructions for setting this up. ## Output Vapoursynth provides a command-line utility called `vspipe` for outputting filtered video. Using this utility to pipe y4m video is the most common way to use Vapoursynth with an encoder. For example, the following command would pipe the output from a Vapoursynth script into x264: `vspipe -c y4m input.vpy - | x264 --demuxer y4m -o output.mkv -` ## Source Filters For Vapoursynth to produce output, it has to load a video in some way. This way is with source filters. The most basic method is using `BlankClip` to create a clip of a certain resolution and frame rate. For example, the following script would give us a blank clip at 640x480: ```python import vapoursynth as vs core = vs.core clip = core.std.BlankClip(width=640, height=480) clip.set_output(0) ``` But a plain black video isn't very useful, is it? We want to load real videos so we can do filtering on them. For this, there are a few different source filters we can look at. ### LSmashSource [LSmashSource](https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works) is a source filter using lsmash as the underlying source library. It is the most commonly used source filter, and is generally reliable for most source formats, though may have frame accuracy issues when seeking with certain input formats, such as VC-1. If this is a concern, it can be recommended to encode to lossless first before using any encoding methods that require seeking, such as av1an. Here is an example of loading a video file using LSmashSource: ```python import vapoursynth as vs core = vs.core clip = core.lsmas.LWLibavSource(source="input.mkv") clip.set_output(0) ``` ### ffms2 [ffms2](https://github.com/FFMS/ffms2) is a source filter based on ffmpeg. It generally should give the same results as LSmashSource, especially when using the git version which has fixed support for newer formats such as AV1. If having issues with a source file with LSmashSource, ffms2 can be a good fallback to try. ```python import vapoursynth as vs core = vs.core clip = core.ffms2.Source(source="input.mkv") clip.set_output(0) ``` ### BestSource [BestSource](https://github.com/vapoursynth/bestsource/) is also based on ffmpeg, but uses additional techniques to ensure frame accuracy in all scenarios with all input formats. The downside is that it must decode the entire video during indexing, which means the first load of a given video will take longer. Subsequent loads will be quick, because BestSource caches the index in a file. ```python import vapoursynth as vs core = vs.core clip = core.bs.VideoSource( source="input.mkv", cachepath="/" ) clip.set_output(0) ``` We add the `cachepath="/"` setting to emulate the behavior of the other source filters, which is to place the index file next to the source video. This helps with portability and cleaning up after ourselves. The default behavior is to place the index file in a temporary directory local to the machine. --- # AMF Canonical URL: https://encode.wiki/tools/hardware-encoders/amf/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 31fe420f7607913c9cc7e3afe18483c13ff442b22dce8d4f6c1ee72cf5ae5d84 > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. AMF for AMD GPUs allows applications to take advantage of the dedicated video encoding & decoding hardware present in AMD GPUs. The AMD Advanced Media Framework (AMF) is a low-level API developed by AMD that enables developers to leverage hardware-accelerated video encoding & decoding on AMD GPUs. By utilizing specialized hardware on the GPU's media block, video encoding and decoding tasks can be offloaded from the CPU, resulting in drastic speed & efficiency increases. AMF provides multimedia processing functionality to applications, and competes with Nvidia's [NVENC](/tools/hardware-encoders/nvenc/) & Intel's [QSV](/tools/hardware-encoders/qsv/) for similar functionality. AMF provides support for various video codecs, including [H.264](/codecs/video/avc/) , [H.265](/codecs/video/hevc/), [VP9](/codecs/video/vp9/), and more recently [AV1](/codecs/video/av1/) on the latest supported GPUs. The GPU's encoding capabilities are especially useful for compressing video content in real-time, where speed is of greater importance than coding efficiency. Hardware-accelerated video encoding using AMF usually significantly improves encoding performance at low compression efficiency compared to software-based encoding solutions. It usually allows for higher-quality output at lower bitrates when encoding much faster than real time, such as at 60-200 fps. This is particularly beneficial for applications that require real-time encoding, such as live streaming, video conferencing, and game recording. However, slower software encoding solutions almost always offer improvements in fidelity per bit compared to hardware encoding. For offline re-encoding & storage, software encoding is generally preferred. AMF in particular is not known for having strong compression efficiency, as it is hampered by AMD's comparably weak media blocks which are usually outperformed by other hardware implementations from Nvidia, Intel, & Apple. AMF is designed to integrate seamlessly with popular media frameworks and libraries, such as [FFmpeg](/tools/utilities/ffmpeg/) and GStreamer. These frameworks often include AMF support, allowing developers to easily incorporate hardware-accelerated encoding into their applications without the need for low-level API programming. AMF is compatible with a wide range of AMD GPUs, including both discrete and integrated graphics solutions. It supports various operating systems, including Windows and Linux, making it accessible to developers across different platforms. --- # Mediacodec Canonical URL: https://encode.wiki/tools/hardware-encoders/mediacodec/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: eb1da6b24d447decc56b470385935363f3c1ec315ac45b77993c45f1ab2121fb The Android's MediaCodec framework is a part of Android's multimedia framework that provides access to low-level media encoder & decoder components. It is similar to [VideoToolbox](/tools/hardware-encoders/videotoolbox/) on Apple devices. Hardware acceleration with MediaCodec is used for processing audio, video, and compressed data. One of the key features of the MediaCodec framework is its support for automatic media transcoding within the operating system. Introduced in Android 12, media transcoding features of the operating system allow devices to use more modern, storage-efficient media formats for video capture while maintaining compatibility with apps. For devices with compatible media transcoding enabled, Android can automatically convert videos recorded in formats such as [H.265](/codecs/video/hevc/) when the videos are opened by an app that doesn't support the format. This allows apps to function even when videos are captured in newer formats on the device. ## Usage In order to view your device's supported hardware and software encoders exposed by the MediaCodec framework, it is advised to download the open source [Codec Info](https://play.google.com/store/apps/details?id=com.parseus.codecinfo) application. Once you know how to properly interact with your device's hardware encoders, [FFmpeg](/tools/utilities/ffmpeg/) will help you transcode videos easily from the command line. ### FFmpeg Testing for this piece was done on the Google Pixel 8, which featurs the Tensor G3 SoC. It is Exynos-based, so [H.264](/codecs/video/avc/), H.265 (HEVC), and [VP9](/codecs/video/vp9/) hardware acceleration for encoding are provided by the Exynos media block. AV1 encoding and decoding are available on the Tensor G3 provided by a custom Google multimedia block. The Exynos's hardware implementation for encoding H.264 and H.265 does not support CQ (Constant Quality) encoding, so a target bitrate must be provided for either CBR (Constant Bitrate) or VBR (Variable Bitrate) encoding. Google's AV1 implementation is in the same situation. Some example MediaCodec encoding commands with FFmpeg: ```bash title="H.264 encoding (VBR, target bitrate 4000K, 250-frame GOP size)" ffmpeg -i input.mkv -c:v h264_mediacodec -codec_name c2.exynos.h264.encoder -bitrate_mode 1 -b:v 4000K -g 250 output.mp4 ``` ```bash title="H.265 encoding (VBR, target bitrate 4000K, 250-frame GOP size)" ffmpeg -i input.mkv -c:v hevc_mediacodec -codec_name c2.exynos.hevc.encoder -bitrate_mode 1 -b:v 4000K -g 250 output.mp4 ``` VP9 encoding produces video that is severely distorted relative to the bitrate, and AV1 encoding produces broken files without metadata. ```bash title="VP9 encoding (VBR, target bitrate 9000K, 250-frame GOP size)" ffmpeg -i input.mkv -c:v vp9_mediacodec -codec_name c2.exynos.vp9.encoder -bitrate_mode 1 -b:v 9000K -g 250 output.mkv ``` ```bash title="AV1 encoding (VBR, target bitrate 8000K, 250-frame GOP size)" ffmpeg -i input.mkv -c:v av1_mediacodec -codec_name c2.google.av1.encoder -bitrate_mode 1 -b:v 8000K -g 250 output.mp4 ``` Just run `ffmpeg -help encoder=hevc_mediacodec` or `ffmpeg -help encoder=h264_mediacodec` for more info on how to use your MediaCodec encoders. You can choose a value for `-codec_name` based on what is shown in the Codec Info app. *Sources* (1) MediaCodec | Android Developers. https://developer.android.com/reference/android/media/MediaCodec. (2) Media | Android Open Source Project. https://source.android.com/docs/core/media. --- # NVENC Canonical URL: https://encode.wiki/tools/hardware-encoders/nvenc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 4bca08d90475dc1543b760338425cf00cf64769cc093422338497cd0e3028bae > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. NVENC for NVIDIA GPUs is a dedicated hardware video encoding engine integrated into NVIDIA's graphics processors. It allows applications to leverage dedicated multimedia encoding hardware to accelerate video encoding tasks, significantly improving performance when compared to CPU-based software encoding. It competes with similar frameworks like Intel's [QSV](/tools/hardware-encoders/qsv/) & AMD's [AMF](/tools/hardware-encoders/amf/). The primary purpose of NVENC is to offload the computationally intensive video encoding workloads from the CPU to the dedicated multimedia hardware on the GPU, thereby freeing up CPU resources for other tasks. This is particularly beneficial in scenarios where fast video encoding is required, such as screen recording, streaming, & video conferencing. NVENC supports a range of popular video codecs, including [H.264](/codecs/video/avc/) , [H.265](/codecs/video/hevc/), [VP9](/codecs/video/vp9/), and more recently [AV1](/codecs/video/av1/) on their latest GPUs. It provides hardware-accelerated encoding capabilities for these codecs, typically achieving real-time or faster than real-time encoding performance, depending on resolution, bitrate, and hardware capability. While NVENC excels in encoding speed, it generally sacrifices some compression efficiency compared to modern high-quality CPU-based software encoders at slower presets. NVENC is designed to be easily integrated into various multimedia frameworks and applications. It is supported by popular tools like [FFmpeg](/tools/utilities/ffmpeg/), OBS Studio, and others, allowing developers to seamlessly leverage GPU-accelerated encoding without the need for low-level programming. When compared to AMD's AMF and Intel's QSV, NVENC is known for its high encoding performance, low latency, and broad compatibility with NVIDIA GPUs across different platforms. It is particularly popular among game streamers, content creators, and video professionals who require fast encoding speeds for their workflows. In terms of video compression efficiency, NVENC & QSV trade blows while AMF is generally left behind. --- # QSV Canonical URL: https://encode.wiki/tools/hardware-encoders/qsv/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 1dbb28176c1a3d2052af8dac4475edf61aca495cb1d346a87a9056bae44f6f30 > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. QSV (QuickSync Video) is Intel's hardware video encoding/decoding platform integrated into many of their modern CPUs with integrated graphics processors (iGPUs) & their Arc graphics cards. It allows applications to offload video encoding, decoding, and processing tasks to the dedicated media engines on Intel's dedicated multimedia hardware, often providing significant performance gains compared to CPU-based software encoding. QSV competes with similar frameworks like Nvidia's [NVENC](/tools/hardware-encoders/nvenc/) & AMD's [AMF](/tools/hardware-encoders/amf/) (Since the transition to Apple Silicon, QSV on Intel Macs competes with Apple's [VideoToolBox](/tools/hardware-encoders/videotoolbox/) on macOS devices). The key purpose of QSV is to accelerate video encoding, decoding, and processing workloads by leveraging specialized fixed-function hardware present in Intel's graphics processors. This dedicated hardware is distinct from the general-purpose compute units, and is designed specifically for multimedia tasks. QSV aims to deliver high encoding/decoding performance while operating efficiently. QSV supports a wide range of video codecs including [H.264](/codecs/video/avc/) , [H.265](/codecs/video/hevc/), [VP9](/codecs/video/vp9/), and more recently [AV1](/codecs/video/av1/) on their latest discrete & integrated GPUs. A major advantage of QSV is that it is ubiquitous on most modern Intel CPUs with integrated graphics, making hardware-accelerated video encoding accessible across a wide range of systems. Applications can easily leverage QSV acceleration through APIs like Intel Media SDK, VA-API, or via integration with popular multimedia frameworks like [FFmpeg](/tools/utilities/ffmpeg/), GStreamer, & others. Hardware-accelerated video encoding with QSV usually significantly improves encoding performance at low compression efficiency compared to software-based encoding solutions. It usually allows for higher-quality output at lower bitrates when encoding much faster than real time, such as at 60-200 fps. This is particularly beneficial for applications that require real-time encoding, such as live streaming, video conferencing, and game recording. While QSV is not designed for highly efficient offline file encoding, where quality is prioritized over speed. It is worth noting that QSV is almost always better than AMF from AMD & competitive with NVENC from Nvidia in terms of compression efficiency. --- # VideoToolbox Canonical URL: https://encode.wiki/tools/hardware-encoders/videotoolbox/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 2a747d8f18110dbc7f10cda774d9f271a9f5b1fa0156923ce361cd39cf2ab18f > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. Apple's VideoToolbox is a low-level framework that provides direct access to hardware encoders and decoders. It offers services for video compression and decompression, as well as conversion between raster image formats stored in CoreVideo pixel buffers. The VideoToolbox encoder works by compressing video data for various applications such as low-latency conferencing, live streaming, and offline transcoding. It supports hardware encoding on most Macs from 2011 and later, and uses Apple's Media Engine on devices with Apple T2 chips or Apple Silicon. The encoder can be configured to optimize encoding for specific applications, and it supports various video codecs including [H.264](/codecs/video/avc/), [H.265](/codecs/video/hevc/), with support for H.265 8-bit and 10-bit encoding. It is worth noting that the VideoToolbox encoder is designed for applications that require direct access to hardware encoders and decoders. Apple's VideoToolbox framework also supports hardware accelerated video decoding for a number of video codecs. As of the Apple M3, these include [H.264](/codecs/video/avc/), [H.265](/codecs/video/hevc/), [ProRes](/codecs/video/prores/), ProRes RAW, and [AV1](/codecs/video/av1/). ## Usage Encoding with Videotoolbox on macOS is possible via [FFmpeg](/tools/utilities/ffmpeg/), a versatile command line utility, or Handbrake, a GUI for video encoding. ### FFmpeg To use H.264 or H.265 (HEVC) hardware encoding in macOS via VideoToolbox, just use the encoder `-c:v h264_videotoolbox` or `-c:v hevc_videotoolbox` for H.264 or HEVC respectively. Here are some example commands for encoding with VideoToolbox on Apple Silicon via FFmpeg: ```bash title="H.264 encoding (high profile)" ffmpeg -i input.mkv -c:v h264_videotoolbox -profile 100 -q:v [0-100] output.mp4 ``` ```bash title="8-bit HEVC encoding (main profile)" ffmpeg -i input.mkv -c:v hevc_videotoolbox -profile 1 -q:v [0-100] -tag:v hvc1 output.mp4 ``` ```bash title="10-bit HEVC encoding (main10 profile)" ffmpeg -i input.mkv -c:v hevc_videotoolbox -profile 2 -q:v [0-100] -tag:v hvc1 output.mp4 ``` Just run `ffmpeg -help encoder=hevc_videotoolbox` or `ffmpeg -help encoder=h264_videotoolbox` for more info. ### Handbrake [HandBrakeCLI](https://handbrake.fr/docs/en/latest/cli/cli-options.html) can be installed via [Homebrew](https://formulae.brew.sh/formula/handbrake): ```bash brew install handbrake ``` Here are some example commands for encoding with VideoToolbox on Apple Silicon via HandBrakeCLI: ```bash title="H.264 encoding with VideoToolbox" HandBrakeCLI -i input.mkv -o output.mp4 --encoder vt_h264 --quality [0-100] ``` ```bash title="H.265 encoding with VideoToolbox" HandBrakeCLI -i input.mkv -o output.mp4 --encoder vt_h265 --quality [0-100] ``` ```bash title="H.265 10-bit encoding with VideoToolbox" HandBrakeCLI -i input.mkv -o output.mp4 --encoder vt_h265_10bit --quality [0-100] ``` Use `HandBrakeCLI --help` to see all available VideoToolbox options (and encoders, under **Video Options**). *Sources* (1) Video Toolbox | Apple Developer Documentation. https://developer.apple.com/documentation/videotoolbox. (2) HandBrake Documentation — Apple VideoToolbox. https://handbrake.fr/docs/en/latest/technical/video-videotoolbox.html. (3) Apple's T2 chip makes a giant difference in video encoding for most .... https://appleinsider.com/articles/19/04/09/apples-t2-chip-makes-a-giant-difference-in-video-encoding-for-most-users. --- # aom-av1-lavish Canonical URL: https://encode.wiki/tools/software-encoders/aom-av1-lavish/ Collection: docs Status: archived License: CC BY-SA 4.0 Checksum: 11418f996bb27bf467fe1747f501e93a30e78215a82b8282a86d9d025e3643bc # aom-av1-lavish > **Note — Community Fork.** > > This entry is about a fork of aomenc called aom-psy101. If you'd like to learn about the mainline aomenc encoder before reading, visit our [aomenc wiki entry](/tools/software-encoders/aomenc/). Mainline aomenc is unfortunately not perfect. It suffers from bad defaults, a heavy focus on the perceptually flawed [PSNR](/reference/metrics/psnr/) metric, misleading settings, and other issues. Fortunately, there are a couple of forks developed by the encoding community that were created to combat aomenc's underlying issues. > **Caution — No Longer Maintained.** > > aom-av1-lavish is no longer maintained as of 4th June 2024. The information in this entry is unaffected by this, but the fact that the project is no longer maintained should be taken into consideration as a user. - [aom-av1-psy](https://github.com/BlueSwordM/aom-av1-psy) *No longer maintained as of 13th January 2023* - [aom-av1-lavish](https://github.com/Clybius/aom-av1-lavish) *No longer maintained as of 4th June 2024* - [aom-psy101](/tools/software-encoders/aom-psy101/) - [aom-av1ador](https://github.com/porcino/aom-av1ador) These forks fix up the poor decisions made by the original AOM devs and most importantly introduce new parameters and tunes to help fine-tune the encoder even more. [aom-av1-lavish](https://github.com/Clybius/aom-av1-lavish) is a fork of aomenc that aims to improve the encoding quality and speed of AV1. It is developed by [Clybius](https://github.com/Clybius), a talented AV1 community developer also well-known for working on [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/). ## FFmpeg aomenc is available in FFmpeg via ``libaom-av1``, check if you have it by running ``ffmpeg -h encoder=libaom-av1``. You can input non-FFmpeg standard aomenc parameters via ``-aom-params``. > **Caution — Mainline aomenc.** > > Unless you compile FFmpeg yourself with aom-av1-lavish, you will be using the mainline aomenc. Compiling from source yourself with the aomenc libraries provided by aom-av1-lavish is the only way to use it with FFmpeg. ## Installation #### Linux *A precompiled AVX2-optimized binary of aom-av1-lavish can be installed for x86_64 Linux via [rAV1ator CLI](/tools/utilities/rav1ator-cli/). However, it is always recommended to build from source.* But if you want to compile the community forks, you can also do that. CMake, Perl, GNU Make, and nasm (assuming x64, if x86 use yasm) will be needed for compilation. ```bash title="Clone the aom-av1-lavish repo Endless_Merging branch, cd and create build folder" git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging cd aom-av1-lavish && mkdir -p aom_build && cd aom_build ``` ```bash title="CMake configuration" cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native" ``` The CMake config above will statically build aomenc while disabling docs (which requires Doxygen), extra tunes, tests, and decoders. While also applying native CPU optimizations to help speed up the encoder. ```bash title="Compile the encoder" make -j$(nproc) ``` The resulting binary will be in the same folder you are on (`aom_build`). Or, optionally, you can install it to your system, which may need elevated permissions. ```bash make install ``` #### macOS macOS is very similar to Linux. Note that some commands may have to be run with `sudo`, which I won't explicitly include for security reasons. **Homebrew** Installing the Homebrew package manager is a well documented process at this point: ```bash /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" ``` Installing mainline libaom is as simple as running the following: ```bash brew update && brew upgrade brew install aom ``` FFmpeg can also be installed via brew. **Building From Source** If you want aom-av1-lavish instead of mainline, you'll have to compile from source. Things are very similar to Linux, with a few oddities: - macOS sometimes doesn't have a `/usr/local/bin` by default. You can fix this by doing `mkdir /usr/local/bin`. - Homebrew installs *everything* in its own directory structure. If you're building things from source that rely on libraries from libvmaf, libjxl, etc, make sure to copy them from `/opt/homebrew/lib` to `/usr/local/lib`. Finding them is a matter of `ls | grep "keyword"` & copying what looks reasonable to be associated with the tool you're using. - Building most things from source will have instructions for \*nix which work for both macOS & Linux. Even if it says Linux, there's a good chance it'll work on macOS as well, & it is always worth trying Linux build instructions on Mac. aom-av1-lavish requires some additional steps, though. If you want to make the most out of your hardware & eke out every last drop of quality, it may be worth building aom-av1-lavish from source. The first step is to clone it from the Endless Merging branch, which contains all of the latest lavish improvements: ```bash git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging cd aom-av1-lavish ``` Now, you need to make some manual changes to the source code until [this commit](https://github.com/Clybius/aom-av1-lavish/pull/1/files) is merged to fix build errors. - Add the line `#include "aq_variance.h"` at line 19 in `av1/encoder/encodeframe_utils.c` - Comment out line 2546 in `av1/encoder/speed_features.c`. This line is `const int qindex_thresh_cdef_sf_s1_s3_l2[2] = { 92, 48 };` & becomes `// const int qindex_thresh_cdef_sf_s1_s3_l2[2] = { 92, 48 };`. Now you can continue to build according to the Linux instructions below. Obviously you'll need cmake, which you can install with homebrew along with any other tools. While still in the `aom-av1-lavish` directory: ```bash mkdir -p aom_build && cd aom_build cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native" make -j$(nproc) # This may need to be run as root. If it doesn't work properly, you can always copy the binary into /usr/local/bin manually: make install ``` Now you can run `aomenc --help | grep "AOMedia" -C 3` to see if lavish installed. If you're getting the same output as above, you may need to copy the `aomenc` executable to `/opt/local/bin`, `/usr/local/bin`, & `/opt/homebrew/bin` if you already installed mainline aomenc. Running the version info command again, the correct output should look something like this: ```bash % aomenc --help | grep AOMedia -C 3 Included encoders: av1 - AOMedia Project AV1 Encoder Psy v3.6.0 (default) Use --codec to switch to a non-default encoder. ``` Notice how it says `AOMedia Project AV1 Encoder Psy` instead of `AOMedia Project AV1 Encoder`. You should be all set after this to start using aom-av1-lavish. #### Windows The pre-built Windows version linked by the original guide on September 6, 2023 is no longer available. Join the [AV1 Discord server](https://discord.gg/vpREHAvYvh) and head to #community-builds for updated versions, you can opt to compile it yourself with the instructions below. **The Compiling Route:** Full credits to u/Turbulent-Bend-7416 on Reddit for [this post](https://www.reddit.com/r/AV1/comments/s6eh5f/how_to_compile_av1_in_windows_without_crying) on how to compile aomenc. This guide requires **MSYS2**, specifically **MinGW-W64**. Install it if you haven't yet. First, install the required dependencies: ```bash pacman -S cmake git perl yasm nasm python3 doxygen mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake base-devel ``` Now, clone the aom-av1-lavish repo in the Endless_Merging branch and create the folders: ```bash git clone https://github.com/Clybius/aom-av1-lavish -b Endless_Merging cd aom-av1-lavish && mkdir -p aom_build && cd aom_build ``` Then we can start compiling with some build optimizations for your CPU: ```bash cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native" make -j$(nproc) ``` The resulting binary will be available within your home folder of the location where you installed MSYS2 (usually `C:`), navigate there and the to the aom-av1-lavish folder and it should be there. Built files should be in the "Debug" folder Don't share binaries compiled with native CPU optimizations unless the person you're sharing to has the same CPU architecture, as this will lead to missing instructions being used and slowing down encode speeds. ## Usage ### AV1 Encoding > **Note — The way aomenc was developed requires 2-pass to take full advantage of its efficiency which include better rate controls and encoding features. So always use 2 passes when encoding..** > > ```bash title="Simple Y4M input with CQ 22, 1 pass, and raw ivf bitstream output" aomenc --end-usage=q --cq-level=32 --bit-depth=10 --passes=1 --ivf -o output.ivf input.y4m ``` ```bash title="Pipe from FFmpeg" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc - --end-usage=q --cq-level=32 --bit-depth=10 --passes=1 --ivf -o output.ivf ``` ```bash title="Pipe from FFmpeg, 2-pass, pass 1" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc - --end-usage=q --cq-level=32 --bit-depth=10 --passes=2 --pass=1 --fpf-log=aom-pass.log --ivf -o output.ivf ``` ```bash title="Pipe from FFmpeg, 2-pass, pass 2" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc - --end-usage=q --cq-level=32 --bit-depth=10 --passes=2 --pass=2 --fpf-log=aom-pass.log --ivf -o output.ivf ``` ### AVIF Encoding Using aomenc through avifenc is widely considered to be the best way to encode AVIF images, as SVT-AV1 only supports 4:2:0 chroma subsampling, rav1e isn't fast enough for still images, & the libaom team have put more effort into intra coding than the teams responsible for producing the other prominent open source AV1 encoders. A sample command for encoding AVIF looks like this: `avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=16 -a tune=ssim [input] output.avif` Where: - `-c aom` is the encoder - `-s 4` is the speed. Speeds 4 & below offer the best compression quality at the expense of longer encode times. - `-j 8` is the number of threads the encoder is allowed to use. Increasing this past 12 will sometimes hurt encode times, as AVIF encoding via aomenc doesn't parallelize perfectly. Test using a speed benchmark to verify which value works best for you. - `-d 10` is the bit depth. Specifying a value below 10 isn't recommended, as it will hurt coding efficiency even with an 8-bit source image. - `-y 444` is the chroma subsampling mode. 4:4:4 chroma subsampling tends to provide better compression than 4:2:0 with AVIF, though on some images 4:2:0 chroma subsampling might be the better choice. - `cq-level=16` is how you specify quality. Lower values correspond to higher quality & filesize, while higher values mean a smaller, lower-quality output is desired. This is preceded by `-a` because it is an aomenc option, not an avifenc one. - `tune=ssim` is how the encoder handles RDO (rate-distortion optimization). This may be redundant with the default aomenc parameters, but specifying doesn't hurt to avoid an unintended change if a default is modified sometime in the future. ## Recommendations aomenc unfortunately lacks the ability to take advantage of multiple threads, so therefore a tool like [Av1an](/tools/utilities/av1an/) will be needed for parallelization. The parameters shown will be biased towards Av1an and aom-av1-lavish usage, so if you plan on using standalone aomenc then adjust as needed. Here are some recommended parameters: `--bit-depth=10 --cpu-used=4 --end-usage=q --cq-level=24 --threads=2 --tile-columns=0 --tile-rows=0 --lag-in-frames=64 --tune-content=psy --tune=ssim --enable-keyframe-filtering=1 --disable-kf --kf-max-dist=9999 --enable-qm=1 --deltaq-mode=0 --aq-mode=0 --quant-b-adapt=1 --enable-fwd-kf=0 --arnr-strength=1 --sb-size=dynamic --enable-dnl-denoising=0 --denoise-noise-level=8` Now let's break it down. - `--bit-depth=10` We're using 10bit because weird linear algebra allows the video to become smaller and reduces banding. - `--cpu-used=4` This is the preset which ranges from 0-9, you can go to 3 if you want more efficiency, 2 if you have a lot of time, 4 is the sweet spot, and 6 if you want speed. Don't go above 6 (Worst efficiency) or even 0 (It would take WEEKS to finish). - `--end-usage=q --cq-level=24` This specifies that we are going to use a knockoff version of CRF level similar to x264/x265 encoders, in this case CRF 24. - `--tile-columns=0 --tile-rows=0` This is the tiles options, where the encoder splits the videos into tiles to encode faster. See the image below (Yellow lines): Tiling > **Note — Tile usage.** > > Do NOT use tiles for 1080p and below, use 1 ``tile-columns`` at 1440p (2K), 2 ``tile-columns`` and 1 ``tile-rows`` for 2160p (4K). > > If you would like an easy way to calculate the necessary number of tiles for your video, you can use [the AV1 Encoding Calculator](https://autocompressor.net/tools/av1-calculator) online or run [this local tile calculator](https://github.com/gianni-rosato/av1-tile-calc). - `--lag-in-frames=64` Similar to x264/x265 `rc-lookahead`. Sets a number of frames to look ahead for frametype and ratecontrol, allowing for better compression decision making. Setting to a value greater than 64 is generally not considered useful. - `--aq-mode=0` adaptive quantization mode, a mostly debatable area nowadays. 0 is better most of the time but some say 1 is also good. - `--tune-content=psy --tune=ssim` As the name suggests they are tunes that affect the video output, for the better, and for the worst. > **Note — Do not use `tune-content=psy` if you encode live action above `cq-level=30`..** > > > **Note — If you use any of the VMAF tunes, you need to specify `--vmaf-model-path=` to where you put VMAF models in..** > > - `--enable-keyframe-filtering=1` We're setting it to 1 because of compatibility reasons, 2 is more efficient but there are seeking issues and FFmpeg can't input it. - `--sb-size=dynamic` Allows the encoder to use 128x128 block partitioning besides 64x64 which gives an efficiency boost. - `--deltaq-mode=0` set to 0 b its better - `--arnr-strength=1` Controls how strong the filtering (smoothing) will be, always been a hot topic. Most agree on the default of 4. Others think 1 is good for 3D Pixar CGI-like and 2D animation and 4 for live action content, and a higher value for lower bitrate encodes. - `--disable-kf --enable-fwd-kf=0` We're disabling keyframes cause Av1an already did scene detection, so we wont have to. Plus it speeds things up. - `--kf-max-dist=9999` Maximum keyframe interval, we're setting it at the highest possible value since Av1an's scene detection keyframe interval is already 240 by default - `--enable-chroma-deltaq=1` --enable-qm=1 --quant-b-adapt=1`` Parameters that give you free efficiency boost, ignore it. - `--enable-dnl-denoising=0` Disables the encoder's built-in denoising technique when grain synthesis is enabled, you can optionally set it to 1 when you have a pretty noisy video since it works quite well (NLMeans is the denoiser used). - `--denoise-noise-level=8` AV1 grain synthesis, which is a technique where the encoder puts fake grain in so it looks more natural and potentially hiding video artifacts (cause grain is hard to encode and explodes bitrate usage because of their randomness). Don't attempt to use it at high values (>12) since it creates noticeable grain patterns. > **Note — You can use photon noise tables as an alternative via `--film-grain-table`, which is also conveniently available in Av1an as `--photon-noise=X`.** > > ## Tips & Tricks 1. Use ``--butteraugli-resize-factor=2`` if you use any of the butteraugli-based tunes to speed it up without much losses (lavish, butteraugli) and ``--butteraugli-intensity-target=250`` to match the content light level. 2. Use ``--arnr-maxframes`` to set max reference frames that will be used to filter the encode, higher values would make the video blurrier at high fidelity but look better at lower bitrates. --- # aom-psy101 Canonical URL: https://encode.wiki/tools/software-encoders/aom-psy101/ Collection: docs Status: archived License: CC BY-SA 4.0 Checksum: c7d342563ce1dbb2ee2d2efd324227df2696f9ef6c6fa2489b9978d7b4b9c1f9 # aom-psy101 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. > **Note — Community Fork.** > > This entry is about a fork of aomenc called aom-psy101. If you'd like to learn about the mainline aomenc encoder before reading, visit our [aomenc wiki entry](/tools/software-encoders/aomenc/). Mainline aomenc is unfortunately not perfect. It suffers from bad defaults, a heavy focus on the perceptually flawed [PSNR](/reference/metrics/psnr/) metric, misleading settings, and other issues. Fortunately, there are a couple of forks developed by the encoding community that were created to combat aomenc's underlying issues. - [aom-av1-psy](https://github.com/BlueSwordM/aom-av1-psy) *No longer maintained as of 13th January 2023* - [aom-av1-lavish](/tools/software-encoders/aom-av1-lavish/) *No longer maintained as of 4th June 2024* - [aom-psy101](https://gitlab.com/damian101/aom-psy101) - [aom-av1ador](https://github.com/porcino/aom-av1ador) These forks fix up the poor decisions made by the original AOM devs and most importantly introduce new parameters and tunes to help fine-tune the encoder even more. [aom-psy101](https://gitlab.com/damian101/aom-psy101) is a fork of aomenc that aims to improve the encoding quality and speed of AV1. It is developed by [damian101](https://gitlab.com/damian101), a talented AV1 community developer. ## FFmpeg aomenc is available in FFmpeg via ``libaom-av1``, check if you have it by running ``ffmpeg -h encoder=libaom-av1``. You can input non-FFmpeg standard aomenc parameters via ``-aom-params``. > **Caution — Mainline aomenc.** > > Unless you compile FFmpeg yourself with aom-psy101, you will be using the mainline aomenc. Compiling from source yourself with the aomenc libraries provided by aom-psy101 is the only way to use it with FFmpeg. ## Installation #### Linux & macOS 1. Clone the psy101 repo: ```bash title="Clone the psy101 repo" git clone https://gitlab.com/damian101/aom-psy101 cd aom-psy101 && mkdir aom_build && cd aom_build ``` 2. Configure compilation. The following flags are set to ensure the `aomenc` binary is build for optimal performance: ```bash title="Set CMake flags" cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -pipe -march=native" -DCMAKE_C_FLAGS="-flto -pipe -march=native" ``` 3. Compile: ```bash title="Compile" make -j$(nproc) ``` 4. Install to your system. This may require elevated privileges: ```bash title="Install" make install ``` #### Windows **MSYS2** is the best option for building in Windows, as it provides a Unix-like environment for compilation. 0. Make sure you have downloaded & installed MSYS2 from [the MSYS2 website](https://www.msys2.org/) before beginning the build process. 1. Close any MSYS2 Console that you have open, start the Clang64 console & install the required dependencies: ```bash pacman -S git perl mingw-w64-clang-x86_64-clang mingw-w64-clang-x86_64-ninja mingw-w64-clang-x86_64-cmake mingw-w64-clang-x86_64-nasm ``` 2. Clone the psy101 repo: ```bash title="Clone the psy101 repo" git clone https://gitlab.com/damian101/aom-psy101 cd aom-psy101 && mkdir aom_build && cd aom_build ``` 2. Configure compilation. The following flags are set to ensure the `aomenc` binary is build for optimal performance: ```bash title="Set CMake flags" LDFLAGS=-static cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -pipe -march=native" -DCMAKE_C_FLAGS="-flto -pipe -march=native" ``` 3. Compile: ```bash title="Compile" ninja ``` The resulting binary will be available within the home folder of the location where you installed MSYS2 (usually `C:`). Navigate there, and then to `aom-psy101\aom_build` folder; the binary should be there. --- # aomenc Canonical URL: https://encode.wiki/tools/software-encoders/aomenc/ Collection: docs Status: archived License: CC BY-SA 4.0 Checksum: 7196f48a50fd2f34e5b8257f7097db6c728fd86507c246dc7366c53b05aa9676 # aomenc [aomenc](https://aomedia.googlesource.com/aom/), AOM-AV1, or just **libaom** is a command line application for encoding AV1 written in C and Assembly developed by AOMedia, which is also the reference encoder for AV1. ## FFmpeg aomenc is available in FFmpeg via ``libaom-av1``, check if you have it by running ``ffmpeg -h encoder=libaom-av1``. You can input non-FFmpeg standard aomenc parameters via ``-aom-params``. ## Supported Color Space aomenc supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s) | |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUV422P | 4:2:2 | 8-bit | | YUV444P | 4:4:4 | 8-bit | | GBRP | - | 8-bit | | GRAY8 | - | 8-bit | | YUV420P10LE | 4:2:0 | 10-bit | | YUV422P10LE | 4:2:2 | 10-bit | | YUV444P10LE | 4:4:4 | 10-bit | | GBRP10LE | - | 10-bit | | GRAY10LE | - | 10-bit | | YUV420P12LE | 4:2:0 | 12-bit | | YUV422P12LE | 4:2:2 | 12-bit | | YUV444P12LE | 4:4:4 | 12-bit | | GBRP12LE | - | 12-bit | | GRAY12LE | - | 12-bit | ## Installation #### Linux & macOS 1. Clone the mainline aom repo: ```bash title="Clone the aom repo" git clone https://aomedia.googlesource.com/aom cd aom && mkdir aom_build && cd aom_build ``` 2. Configure compilation. The following flags are set to ensure the `aomenc` binary is build for optimal performance: ```bash title="Set CMake flags" cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native" ``` 3. Compile: ```bash title="Compile" make -j$(nproc) ``` 4. Install to your system. This may require elevated privileges: ```bash title="Install" make install ``` #### Windows **MSYS2** is the best option for building in Windows, as it provides a Unix-like environment for compilation. 0. Make sure you have downloaded & installed MSYS2 from [the MSYS2 website](https://www.msys2.org/) before beginning the build process. 1. Start the UCRT64 console & install the required dependencies: ```bash pacman -S cmake git perl yasm nasm python3 doxygen mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake base-devel ``` 2. Clone the mainline aom repo: ```bash title="Clone the aom repo" git clone https://aomedia.googlesource.com/aom cd aom && mkdir aom_build && cd aom_build ``` 2. Configure compilation. The following flags are set to ensure the `aomenc` binary is build for optimal performance: ```bash title="Set CMake flags" cmake .. -DBUILD_SHARED_LIBS=0 -DENABLE_DOCS=0 -DCONFIG_TUNE_BUTTERAUGLI=0 -DCONFIG_TUNE_VMAF=0 -DCONFIG_AV1_DECODER=0 -DENABLE_TESTS=0 -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_LD_FLAGS="-flto -O3 -march=native" ``` 3. Compile: ```bash title="Compile" make -j$(nproc) ``` The resulting binary will be available within the home folder of the location where you installed MSYS2 (usually `C:`). Navigate there, and then to the `aom` folder; the binary should be there. Built files should be in the "Debug" folder. > **Caution — Sharing Native Binaries.** > > Avoid sharing binaries compiled with native CPU optimizations unless the person you're sharing to has the same CPU architecture, as this can lead to incorrect encoder behavior. ## Usage ### AV1 Encoding > **Note — 2-Pass Encoding.** > > The way aomenc was developed requires 2-pass to take full advantage of its efficiency which include better rate controls and encoding features. So always specify the encoder to use 2 passes when encoding. ```bash title="Simple Y4M input with CQ 22, 1 pass, and raw ivf bitstream output" aomenc --end-usage=q --cq-level=32 --bit-depth=10 --passes=1 --ivf -o output.ivf input.y4m ``` ```bash title="Pipe from FFmpeg" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc - --end-usage=q --cq-level=32 --bit-depth=10 --passes=1 --ivf -o output.ivf ``` ```bash title="Pipe from FFmpeg, 2-pass, pass 1" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc - --end-usage=q --cq-level=32 --bit-depth=10 --passes=2 --pass=1 --fpf-log=aom-pass.log --ivf -o output.ivf ``` ```bash title="Pipe from FFmpeg, 2-pass, pass 2" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc - --end-usage=q --cq-level=32 --bit-depth=10 --passes=2 --pass=2 --fpf-log=aom-pass.log --ivf -o output.ivf ``` ### AVIF Encoding Using aomenc through avifenc is widely considered to be the best way to encode AVIF images, as SVT-AV1 only supports 4:2:0 chroma subsampling, rav1e isn't fast enough for still images, & the libaom team have put more effort into intra coding than the teams responsible for producing the other prominent open source AV1 encoders. A sample command for encoding AVIF looks like this: `avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=16 -a tune=ssim [input] output.avif` Where: - `-c aom` is the encoder - `-s 4` is the speed. Speeds 4 & below offer the best compression quality at the expense of longer encode times. - `-j 8` is the number of threads the encoder is allowed to use. Increasing this past 12 will sometimes hurt encode times, as AVIF encoding via aomenc doesn't parallelize perfectly. Test using a speed benchmark to verify which value works best for you. - `-d 10` is the bit depth. Specifying a value below 10 isn't recommended, as it will hurt coding efficiency even with an 8-bit source image. - `-y 444` is the chroma subsampling mode. 4:4:4 chroma subsampling tends to provide better compression than 4:2:0 with AVIF, though on some images 4:2:0 chroma subsampling might be the better choice. - `cq-level=16` is how you specify quality. Lower values correspond to higher quality & filesize, while higher values mean a smaller, lower-quality output is desired. This is preceded by `-a` because it is an aomenc option, not an avifenc one. - `tune=ssim` is how the encoder handles RDO (rate-distortion optimization). This may be redundant with the default aomenc parameters, but specifying doesn't hurt to avoid an unintended change if a default is modified sometime in the future. ## Recommendations aomenc is largely lacking in its ability to take advantage of multiple threads, so a tool like [Av1an](/tools/utilities/av1an/) should be utilized for effective parallelization. The parameters shown will be biased towards Av1an and [aom-av1-lavish](/tools/software-encoders/aom-av1-lavish/) usage, so if you plan on using standalone aomenc please adjust as needed. Here are some recommended parameters: `--bit-depth=10 --cpu-used=4 --end-usage=q --cq-level=24 --threads=2 --tile-columns=0 --tile-rows=0 --lag-in-frames=64 --tune=ssim --enable-keyframe-filtering=1 --disable-kf --kf-max-dist=9999 --enable-qm=1 --deltaq-mode=0 --aq-mode=0 --enable-fwd-kf=0 --arnr-strength=1 --sb-size=dynamic --enable-dnl-denoising=0 --denoise-noise-level=8` Now let's break it down. - `--bit-depth=10` We're using 10bit because weird linear algebra allows the video to become smaller and reduces banding. - `--cpu-used=4` This is the preset which ranges from 0-9, you can go to 3 if you want more efficiency, 2 if you have a lot of time, 4 is the sweet spot, and 6 if you want speed. Don't go above 6 (Worst efficiency) or even 0 (It would take WEEKS to finish). - `--end-usage=q --cq-level=24` This specifies that we are going to use a knockoff version of CRF level similar to x264/x265 encoders, in this case CRF 24. - `--tile-columns=0 --tile-rows=0` This is the tiles options, where the encoder splits the videos into tiles to encode faster. See the image below (Yellow lines): Tiling > **Note — Tile usage.** > > Do NOT use tiles for 1080p and below, use 1 ``tile-columns`` at 1440p (2K), 2 ``tile-columns`` and 1 ``tile-rows`` for 2160p (4K). > > If you would like an easy way to calculate the necessary number of tiles for your video, you can use [the AV1 Encoding Calculator](https://autocompressor.net/tools/av1-calculator) online or run [this local tile calculator](https://github.com/gianni-rosato/av1-tile-calc). - `--lag-in-frames=64` Similar to x264/x265 `rc-lookahead`. Sets a number of frames to look ahead for frametype and ratecontrol, allowing for better compression decision making. Setting to a value greater than 64 is generally not considered useful. - `--aq-mode=0` adaptive quantization mode, a mostly debatable area nowadays. 0 is better most of the time but some say 1 is also good. - `--enable-keyframe-filtering=1` We're setting it to 1 because of compatibility reasons, 2 is more efficient but there are seeking issues and FFmpeg can't input it. - `--sb-size=dynamic` Allows the encoder to use 128x128 block partitioning besides 64x64 which gives an efficiency boost. - `--deltaq-mode=0` This value has been tested to be more perceptually efficient. - `--arnr-strength=1` Controls how strong the filtering (smoothing) will be, always been a hot topic. Most agree on the default of 4. Others think 1 is good for 3D Pixar CGI-like and 2D animation and 4 for live action content, and a higher value for lower bitrate encodes. - `--disable-kf --enable-fwd-kf=0` We're disabling keyframes cause Av1an already did scene detection, so we wont have to. Plus it speeds things up. - `--kf-max-dist=9999` Maximum keyframe interval, we're setting it at the highest possible value since Av1an's scene detection keyframe interval is already 240 by default - `--enable-chroma-deltaq=1 --enable-qm=1` Parameters that give you free efficiency boost discovered via testing. - `--enable-dnl-denoising=0` Disables the encoder's built-in denoising technique when grain synthesis is enabled, you can optionally set it to 1 when you have a pretty noisy video since it works quite well (NLMeans is the denoiser used). - `--denoise-noise-level=8` AV1 grain synthesis, which is a technique where the encoder puts fake grain in so it looks more natural and potentially hiding video artifacts (cause grain is hard to encode and explodes bitrate usage because of their randomness). Don't attempt to use it at high values (>12) since it creates noticeable grain patterns. > **Note — You can use photon noise tables as an alternative via `--film-grain-table`, which is also conveniently available in Av1an as `--photon-noise=X`.** > > ## Community Forks Mainline aomenc is unfortunately not perfect. It suffers from bad defaults, a heavy focus on the perceptually flawed [PSNR](/reference/metrics/psnr/) metric, misleading settings, and other issues. Fortunately, there are a couple of forks developed by the encoding community that were created to combat aomenc's underlying issues. - [aom-av1-psy](https://github.com/BlueSwordM/aom-av1-psy) *No longer maintained as of 13th January 2023* - [aom-av1-lavish](/tools/software-encoders/aom-av1-lavish/) *No longer maintained as of 4th June 2024* - [aom-psy101](/tools/software-encoders/aom-psy101/) - [aom-av1ador](https://github.com/porcino/aom-av1ador) These forks fix up the poor decisions made by the original AOM devs and most importantly introduce new parameters and tunes to help fine-tune the encoder even more. --- # Aurora1 Canonical URL: https://encode.wiki/tools/software-encoders/aurora1/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: e0077099e1086c2213a61d75455f8249ed5ca2ef995948993c0dd4c3a3177a7f # Aurora1 AV1 [Aurora1 AV1](https://visionular.ai/aurora1-av1-encoder/) is a proprietary and paid software AV1 encoder developed by [Visionular](https://www.visionular.com/en). Although they do provide a contact form to get a free trial, not much is known about this encoder. --- # AVM Canonical URL: https://encode.wiki/tools/software-encoders/avm/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 29ed3434ebc772b215d139d342fde5d6da15665ba030dc2176add49887d951ac > AVM (AOM Video Model) is the reference software for next codec from Alliance for Open Media. [AVM](https://gitlab.com/AOMediaCodec/avm), or **A**OM **V**ideo **M**odel is the reference implementation for a future codec from the Alliance for Open Media, the organization behind [AV1](/codecs/video/av1/). The codebase is under the Clear BSD license and currently only produces `av01` bitstreams. The AVM codec is currently in development and is not yet ready for production use. Not much has been documented or tested. ## Rumors Some things about the new encoding implementation can be confirmed via the codebase, but none of those changes are final until the codec is standardized and officially released. Some rumors about the codec include: - The name of the codec is going to be AV2, superseding AV1 - The codec will be based on AV1, with certain backwards compatibility features available - Hardware decoding implementations could be implemented at no cost (no royalties) by utilizing GPU shaders and existing AV1 decoding hardware. - AOM's strategy will be to release codecs "mid-cycle" relative to ISO/ITU's release schedule, meaning it is likely that "AV2" will compete with [VVC](/codecs/video/vvc/), not [ECM](/codecs/video/ecm/). - A quantizer scale of 0-255 will be standard. - AVM tries to address some issues with high-fidelity AV1 encoding by introducing a better denoiser to mitigate [mosquito noise](/start-here/video-artifacts/#mosquito-noise). ## Installation ### Arch Linux AVM is available in the Arch User Repository (AUR) as `avm` and `avm-git`. ### Compiling Since this encoder is under heavy development, there are no pre-built binaries provided, so you will need to compile yourself. Windows users are recommended to compile via MinGW-W64 which comes with [MSYS2](https://msys2.org/). > **Caution — Compilation requires CMake, Nasm, and Perl..** > > ```bash git clone https://gitlab.com/AOMediaCodec/avm.git cd avm/build cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 make -j$(nproc) ``` Since this is a huge project, compiling will take a while depending on your CPU. The resulting binary will be called `aomenc`, the same name that encodes content to AV1. It will be available in the same folder (`build`), or you can run `make install` on Linux to install (May need elevated permissions). ## Usage > **Tip — To convert `cq-level` in aomenc and `crf` in SVT-AV1 to AVM's QP values, multiply by 4. For example, `--cq-level 20` equals to `--quantizer 60`..** > > Simple Y4M input with QP 65, and `ivf` output: ```bash aomenc --qp=65 -o output.ivf input.y4m ``` Preset level 6 (higher is faster), QP 65, Y4M input: ```bash aomenc --qp=65 --cpu-used=6 -o output.ivf input.y4m ``` FFmpeg piping: ```bash ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | aomenc --qp=65 --cpu-used=6 -o output.ivf - ``` ## aomdec You will need the `aomdec` binary you also compiled to be able to play your encoded video, as there are **zero** video players currently in the whole world that can play your encoded content. --- # Eve-AV1 Canonical URL: https://encode.wiki/tools/software-encoders/eve-av1/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 17147cd3f53a9eea2abd195472ad4d16e87d173fdd16574477d7014e7aa9bff0 Eve-AV1 is a proprietary [AV1](/codecs/video/av1/) video encoder developed by Two Orioles, LLC, an organization primarily known for their work on [dav1d](/tools/utilities/dav1d/), the most widely used AV1 software decoder. According to the Two Orioles [webpage on Eve-AV1](https://www.twoorioles.com/eve-av1): > Eve-AV1 gives you an unprecedented level of video quality for streaming on-demand video. It provides a 20% reduction in bitrate at the same visual quality compared to other AV1 encoders. Or you can choose a 3 to 5 times speedup in encoding time A single visual comparison is provided, alongside BD-rate graphs for [VMAF](/reference/metrics/vmaf/). ## Performance ![Eve-AV1 VMAF performance](/img/eve_av1_speed.webp) Eve-AV1 appears to perform quite well according to VMAF, and is able to do so at what appear to be impressive speeds. However, without more detailed visual comparisons, it is hard to take this data at face value. --- # Eve-VP9 Canonical URL: https://encode.wiki/tools/software-encoders/eve-vp9/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: acbddf571a76f69d19ad6b201169bf8c3c3395c4df77d23b622d9a3f4a6c6c01 Eve-VP9 is a proprietary [VP9](/codecs/video/vp9/) video encoder developed by Two Orioles, LLC, an organization primarily known for their work on [dav1d](/tools/utilities/dav1d/), the most widely used AV1 software decoder. According to the Two Orioles [webpage on Eve-VP9](https://www.twoorioles.com/eve-vp9): > At top quality, EVE-VP9 provides nearly 20% better compression than libvpx at the same speed; 10% better compression than x265 at 50% faster speed; and compared to x264, EVE-VP9 is nearly 40% better. Sparse visual comparisons and BD-rate graphs for VMAF are provided by the company, but no further details are available. ## Performance ![Eve-VP9 VMAF performance](/img/eve_vp9_speed.webp) Eve-VP9 appears to perform quite well according to VMAF, and is able to do so at what appear to be impressive speeds. However, without more detailed visual comparisons, it is hard to take this data at face value. --- # HM Canonical URL: https://encode.wiki/tools/software-encoders/hm/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 821e9f0a5b458ded20ebc4c203c0842eaa802b37dc2b488f9fcc437ed2b17d4c > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. HM is the original [H.265](/codecs/video/hevc/) reference encoder, predating alternatives like [x265](/tools/software-encoders/x265/). In the modern day, it joins other MPEG reference encoders such as [JM](/tools/software-encoders/jm/) & [VTM](/tools/software-encoders/vtm/) in their reputations for being highly niche offerings that are used rarely due to their usage complexity & speed disadvantages. x264 is more efficient than JM. HM is capable of producing higher quality streams than highly tuned x265, even at excruciatingly slow speeds. This is only a theoretical advantage, though, as HM is incapable of placing keyframes automatically with scene detection & would need a chunking too reminiscent of [Av1an](/tools/utilities/av1an/) to do this. For videos containing few enough frames where keyframe placement isn't a concern, HM is better in practice than x265 at the expense of a massive dropoff in speed. HM doesn't have any threading capabilities & is much slower than even x265 placebo. ## Installation These build instructions are valid for Linux & macOS. ```bash git clone https://vcgit.hhi.fraunhofer.de/jvet/HM cd HM/ mkdir build && cd build cmake .. -DCMAKE_BUILD_TYPE=Release make -j$(nproc) ``` The binary `TAppEncoderStatic` or `TAppEncoder` can be found within the cloned directories, & can be copied to `/usr/local/bin` for encoding. Decoding & other functions of the reference codec implementation aren't covered in this entry. ## Usage Here is a sample command: `TAppEncoderStatic -i input.yuv -b out.265 -c ~/HM/cfg/encoder_randomaccess_main10.cfg -wdt 1280 -hgt 720 -fr 50 -f 500 -q 27 -xPS 0` Make sure only to use only YUV input when encoding with HM. Each parameter does the following: - `-i input.yuv -b out.265` Specifies a raw YUV input file & an output raw h265 bitstream. To mux into an MP4 container, it is recommended that you use [mp4box](/tools/utilities/mp4box/) instead of muxing with [FFmpeg](/tools/utilities/ffmpeg/). - `-c [path/to/config]` Specifies the desired path to your HM configuration file. This makes it easier to encode without having to manually specify a plethora of settings. - `-wdt 1280 -hgt 720` Sets the input & output width & height. - `-fr 50 -f 500` Sets the framerate (FPS) & the number of frames to encode. In this case, we are encoding 500 frames of a video that is to be played back at 50fps. - `-q 27` Sets a quality target for the encoder. - `-xPS 0` Zero clue what this does. If someone has an idea, please contribute! --- # JM Canonical URL: https://encode.wiki/tools/software-encoders/jm/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 2ae51c90937d49e71de570ce924df9f01aa4a8d87ad4c90eb2c6989cc0cdbeb7 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! JM is the original [H.264](/codecs/video/avc/) reference encoder, predating alternatives like [x264](/tools/software-encoders/x264/). Since x264 became highly performant and perceptually driven, it joins other MPEG reference encoders such as [HM](/tools/software-encoders/hm/) & [VTM](/tools/software-encoders/vtm/) in their reputations for being highly niche offerings that are used rarely due to their usage complexity & speed disadvantages. x264 is generally more efficient than JM. --- # Kvazaar Canonical URL: https://encode.wiki/tools/software-encoders/kvazaar/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 37f1edcbeb3f1ef7b4d0dd605d0810b009495f525bb279343beada2d3c0de648 # Kvazaar [Kvazaar](https://github.com/ultravideo/kvazaar) is an open-source [H.265 / HEVC](/codecs/video/hevc/) software encoder Written in C, developed by [Ultra Video Group](https://ultravideo.fi/) and licensed under BSD 3-clause. [uvg266](/tools/software-encoders/uvg266/) (Developed by the same group) uses Kvazaar as a base for encoding to the [VVC](/codecs/video/vvc/) codec. [x265](/tools/software-encoders/x265/) is generally regarded as having better performance while producing better quality video streams. ## FFmpeg Kvazaar is available in [FFmpeg](/tools/utilities/ffmpeg/) via ``libkvazaar``, to check if you have it, run ``ffmpeg -h encoder=libkvazaar``. You can input non-FFmpeg standard Kvazaar parameters via ``-kvazaar-params``. You may need to download "Full" builds. As most of the time, this encoder is not included. ## Supported Color Space Kvazaar supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s)| |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUV420P10LE | 4:2:0 | 10-bit* | *10-bit support requires a flag to be set during compilation with CMake. ## Installation #### Linux & macOS For Arch Linux, Kvazaar is available as `kvazaar`. It is also available in the Arch User Repository (AUR) as `kvazaar-git`. Ultra Video Group does not ship any pre-built binaries of their encoders except for their [AppVeyor CI](https://ci.appveyor.com/project/Ultravideo/kvazaar), but AppVeyor deletes build artifacts after a month, so most of the time you'll have to compile Kvazaar yourself. Here are the instructions to do so: ### Autotools 0. Compilation requires GNU Automake, Autoconf, Libtool, and M4. Install them via your package manager. 1. Clone the repository and its submodules: ```bash git clone --recursive https://github.com/ultravideo/kvazaar.git cd kvazaar ./autogen.sh ./configure make -j$(nproc) ``` 3. Binaries will be available in `src`, or you can run `make install` on Linux to install (May need elevated permissions). ### CMake (10-bit support) You will need to use CMake to specify a flag to be able to encode 10-bit with the encoder; by default Kvazaar ships with only 8-bit. ```bash git clone --recursive https://github.com/ultravideo/kvazaar.git cd kvazaar/build cmake .. -DCMAKE_C_FLAGS="-DKVZ_BIT_DEPTH=10" # optional 10-bit flag make -j$(nproc) ``` Be aware that encoding 10-bit HEVC with Kvazaar is significantly slower, as the developers only prioritized SIMD optimizations for 8-bit encoding. Be aware that this implementation can be buggy in general. #### Windows Windows users are recommended to compile via MinGW-W64 which comes with [MSYS2](https://msys2.org/). Please be advised **the usage of Clang for compiling in this situation is heavily recommended** due to disabled AVX2 optimizations because of a known GCC issue from 2012 (MinGW environments-exclusive). To do this, run `CC=clang ./configure` during autoconf. ### MSYS2 0. Make sure you have downloaded & installed MSYS2 from [the MSYS2 website](https://www.msys2.org/) before beginning the build process. 1. Start the UCRT64 console & install the required dependencies with the `pacman` package manager 3. Resume the build process as you would on a Unix-like system. See the "Linux & macOS" tab for more information. ## Usage Here are some examples of how to use Kvazaar on its own: ```bash title="Simple Y4M input with QP 20 and raw 265 bitstream output" kvazaar -i input.y4m --input-file-format y4m --qp 20 -o output.265 ``` ```bash title="Preset slow, CRF 20, Y4M input" kvazaar -i input.y4m --input-file-format y4m --qp 20 --preset slow -o output.265 ``` The command below still uses the `kvazaar` binary, but reads from a YUV4MPEG pipe instead of a file. This is useful for piping FFmpeg output to Kvazaar. ```bash title="FFmpeg piping" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | kvazaar -i - --input-file-format y4m --qp 20 --preset slow -o output.265 ``` --- # rav1e Canonical URL: https://encode.wiki/tools/software-encoders/rav1e/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: e9430085a067ba29465ffb6d382f05b361950ade2cd1175e8ca571d5dfa4e37c # rav1e [rav1e](https://github.com/xiph/rav1e) is an open source command line application for encoding AV1 written in Assembly & Rust, co-developed by Xiph.org and Mozilla and licensed under BSD-2 Clause. ## FFmpeg rav1e is available in FFmpeg via `librav1e`, to check if you have it, run `ffmpeg -h encoder=librav1e`. You can input non-FFmpeg standard rav1e parameters via `-rav1e-params`. ## Supported Color Space rav1e supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s) | |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUVJ420P | 4:2:0 | 8-bit (Full range) | | YUV422P | 4:2:2 | 8-bit | | YUVJ422P | 4:2:2 | 8-bit (Full range) | | YUV444P | 4:4:4 | 8-bit | | YUVJ444P | 4:4:4 | 8-bit (Full range) | | YUV420P10LE | 4:2:0 | 10-bit | | YUV422P10LE | 4:2:2 | 10-bit | | YUV444P10LE | 4:4:4 | 10-bit | | YUV420P12LE | 4:2:0 | 12-bit | | YUV422P12LE | 4:2:2 | 12-bit | | YUV444P12LE | 4:4:4 | 12-bit | ## Installation #### Linux & macOS Official pre-built rav1e binaries can be found on the [releases page](https://github.com/xiph/rav1e/releases) in rav1e's Github repository. Unofficially, the [rAV1ator CLI](/tools/utilities/rav1ator-cli/) command line tool can automatically download and install rav1e to `/usr/local/bin`. rav1e can also be installed with Cargo by running `cargo install rav1e`. > **Info — Stable Release.** > > For stability & a proper version number, please reset the source to the correct release commit. In the releases page, click the icon to the right of the release tag & copy the commit in the url bar. Then, in the cloned rav1e directory, `git reset --hard [commit hash]` Here are instructions for resetting to release 0.7.1 (latest as of 19 Feb 2024) and building. Omit the `git reset` command to use the latest git, if you have a *specific* reason to use the latest git instead of an official tagged release. ```bash git clone https://github.com/xiph/rav1e.git cd rav1e git reset --hard a8d05d0c43826a465b60dbadd0ab7f1327d75371 RUSTFLAGS="-C target-cpu=native" cargo build --release ``` When done, the binary can be found in `/target/release`. You can then copy the binary wherever you desire it to go, like by doing `cp /target/release/rav1e /usr/local/bin` . ### Patched Installation with HDR10+ support rav1e currently has an [unmerged pull request](https://github.com/xiph/rav1e/pull/3000) by quietvoid, the person behind [hdr10plus_tool](/tools/utilities/hdr10plus_tool/) and [dovi_tool](/tools/utilities/dovi_tool/). The PR adds a new parameter called `--hdr10plus-json` for HDR10+ JSON dynamic metadata input. To merge it locally, do the following: ```bash git clone https://github.com/xiph/rav1e.git cd rav1e git reset --hard [release commit] git fetch origin pull/3000/head:HDR10+ ``` Now the patch should be applied, and you may build as usual. #### Windows If you would not like to build from source, official pre-built rav1e binaries can be found on the [releases page](https://github.com/xiph/rav1e/releases) in rav1e's Github repository. > **Info — Stable Release.** > > For stability & a proper version number, please reset the source to the correct release commit. In the releases page, click the icon to the right of the release tag & copy the commit in the url bar. Then, in the cloned rav1e directory, `git reset --hard [commit hash]` Here are instructions for resetting to release 0.7.1 (latest as of 19 Feb 2024) and building. Omit the `git reset` command to use the latest git, if you have a *specific* reason to use the latest git instead of an official tagged release. ```powershell git clone https://github.com/xiph/rav1e.git cd rav1e git reset --hard a8d05d0c43826a465b60dbadd0ab7f1327d75371 set RUSTFLAGS=-C target-cpu=native cargo build --release ``` When done, the binary can be found in `target/release` ## Usage For AV1 encoding, rav1e has very sane defaults. It is very hard to go wrong with parameters if you modify as few as possible. > **Tip — To convert `cq-level` in aomenc and `crf` in SVT-AV1 to rav1e's quantizer values, multiply by 4. For example, `--cq-level 20` equals to `--quantizer 80`..** > > ```bash title="Basic usage" rav1e -i input.y4m -o output.ivf --quantizer 60 --photon-noise 8 ``` ```bash title="Basic usage with FFmpeg piping, 10bit input" ffmpeg -i input.mkv -pix_fmt yuv420p10le -strict -2 -f yuv4mpegpipe - | rav1e - -o output.ivf --quantizer 80 --photon-noise 8 ``` ```bash title="Basic usage with FFmpeg piping, 10bit input and assuming 4K" ffmpeg -i input.mkv -pix_fmt yuv420p10le -strict -2 -f yuv4mpegpipe - | rav1e - -o output.ivf --quantizer 68 --tile-columns 2 --tile-rows 1 --photon-noise 8 ``` ## Tips & Tricks 1. Use 2x1 tiles or `tile-columns 2` and `tile-rows 1` for 4K (2160p) encoding, this will help with both encoding and decoding speeds. --- # SVT-AV1 Canonical URL: https://encode.wiki/tools/software-encoders/svt-av1/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: d2d4b2392cc3213904c6ad7e7c375355aa5c691bbce22b1f2ae16235ff25a5c7 # SVT-AV1 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. SVT-AV1 (Scalable Video Technology for AV1) is an AV1-compliant software encoder/decoder library. Jointly developed by Intel and Netflix, SVT-AV1 is written almost entirely in C with some parts written in C++ and Assembly. As the name suggests, it is part of the "[Scalable Video Technology](https://www.intel.com/content/www/us/en/developer/articles/technical/scalable-video-technology.html)" project lineup by Intel. This entry discusses the SVT-AV1 encoder, also known as the "Production" AV1 encoder (while [aomenc](/tools/software-encoders/aomenc/) is the "reference" AV1 encoder), & refers to SVT-AV1 as such. SVT-AV1 is known for its parallelization, high coding efficiency, & active development. SVT-AV1 scales across multiple CPU cores much more effectively than aomenc or [rav1e](/tools/software-encoders/rav1e/), so the use of tools like [Av1an](/tools/utilities/av1an/) is less helpful albeit still helpful for scene detection. ## FFmpeg SVT-AV1 is available in FFmpeg via `libsvtav1`, to check if you have it, run `ffmpeg -h encoder=libsvtav1`. You can input non-FFmpeg standard SVT-AV1 parameters via `-svtav1-params`. ## Supported Color Space SVT-AV1 supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s) | |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUV420P10LE | 4:2:0 | 10-bit | ## Installation #### Linux & macOS *A precompiled AVX2-optimized binary of SVT-AV1-PSY can be installed for x86_64 Linux via [rAV1ator CLI](/tools/utilities/rav1ator-cli/). However, it is always recommended to build from source.* To build SVT-AV1 from source, first clone the desired SVT-AV1 repository & enter the build directory. ```bash title="Clone mainline SVT-AV1" git clone https://gitlab.com/AOMediaCodec/SVT-AV1/ git reset --hard bbcff785881b320f7e1b1f77a2f5ed025f8bfd75 # Reset to release 2.1.0 cd SVT-AV1/Build/linux ``` ```bash title="Clone SVT-AV1-PSY" git clone https://github.com/gianni-rosato/svt-av1-psy cd SVT-AV1/Build/linux ``` In the directory, simply run `./build.sh [flags]` to build. Be aware that building requires CMake version 3.16 or higher and either GCC or Clang. It is recommended to use clang when building SVT-AV1. ```bash title="Build release" ./build.sh release ``` ```bash title="Statically build just the encoder with clang and enable link-time optimization" ./build.sh jobs=8 all cc=clang cxx=clang++ no-dec enable-lto static native ``` The compiled binaries will be in the `Bin/Release` directory, including SvtAv1EncApp. If you just want the encoder, adding the `no-dec` flag will skip building SvtAv1DecApp and save on compilation time. If you want extra performance, it is possible to build SVT-AV1 using PGO (Profile-guided Optimization). **Be aware that this particular script infers that you have a .y4m file (or multiple) in `/dev/shm` for transcoding**. You can compile statically linked SVT-AV1 with PGO (and LTO, or link-time optimization) by following this script: ```bash title="Bulding SVT-AV1 with profile guided optimization" git clone https://gitlab.com/AOMediaCodec/SVT-AV1/ cd SVT-AV1/Build/linux ./build.sh cc=gcc cxx=g++ enable-lto enable-pgo static native jobs=$(nproc) pgo-dir=/dev/shm pgo-videos=/dev/shm release ``` If you wish to store videos elsewhere or provide custom parameters to the SvtAv1EncApp binary, try this script: ```bash git clone https://gitlab.com/AOMediaCodec/SVT-AV1/ cd SVT-AV1/Build/linux ./build.sh cc=gcc cxx=g++ enable-lto enable-pgo static native jobs=$(nproc) pgo-dir=/dev/shm pgo-compile-gen release ../../Bin/Release/SvtAv1EncApp # Run this binary as many times as you'd like with arguments of your choice to collect data ./build.sh cc=gcc cxx=g++ enable-lto enable-pgo static native jobs=$(nproc) pgo-dir=/dev/shm pgo-compile-use release ``` #### Windows To be filled. If you believe you can help, see our [Contribution Guide](/reference/legacy/contribution-guide/). ## Encoding ### Strengths SVT-AV1's greatest strength is its parallelization capability, where it outclasses other AV1 encoders by a significant margin. SVT-AV1's parallelization techniques do not involve tiling & don't harm video quality, & can comfortably utilize up to 16 cores given 1080p source video. This is while maintaining competitive coding efficiency to mainline aomenc. Perceptually, mainline SVT-AV1 is outperformed by well-tuned community forks of aomenc, but according to many the gap has begun to close with the introduction of SVT-AV1-PSY. ### Weaknesses SVT-AV1 is strongest on x86 CPUs, & while ARM NEON assembly is [available](https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/ba13fac241f1b54954935f2cb200efc07f3de13a) and has been slowly improving since its introduction in version 1.8.0, SVT-AV1 still underperforms on ARM. For this reason, it is not a good cross-architecture CPU benchmark. SVT-AV1's support for various AV1 features is also limited; it only supports up to 4:2:0 chroma subsampling with no support for 12-bit color, and it does not support scene change detection (there are no plans to implement this, either). The smallest possible video that SVT-AV1 can produce is 64x64. ### Encoder Optimization Aside from build optimizations for speed, there is further tweaking to be done to the `SvtAv1EncApp` binary parameters when encoding. The following applies to mainline SVT-AV1, but does not apply to SVT-AV1-PSY. - `--film-grain` & `--film-grain-denoise` Most live-action sources feature hard-to-compress digital noise that is easily smoothed out by AV1 compression. To add this grain back, or even denoise through the encoder and then add grain, it is possible to use the `--film-grain` parameter to specify an amount of film grain to add to the encode (& `--film-grain-denoise` to specify how to denoise the input video before encoding for potentially better appeal). Denoising a video always removes fine details, so sticking with just `--film-grain` is recommended in most cases. According to [SVT-AV1 documentation](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/CommonQuestions.mdx#practical-advice-on-grain-synthesis), a level of 8 should be used for live-action content with a normal amount of grain while a level of 4 works well for hand-drawn animation or other smoother-looking sources that still stand to benefit from some grain synthesis. - `--input-depth 10` 10-bit output from AV1 encoding is always desirable for coding efficiency, even if your source is 8-bit. This option only produces a 10-bit AV1 bitstream if the source provided to the encoder is 10-bit. - `--tune 2` There are three tunes in mainline SVT-AV1: Tune 1 is for [PSNR](/reference/metrics/psnr/) [RDO](/start-here/psychovisual/), Tune 2 is for [SSIM](/reference/metrics/ssim/) RDO, & Tune 0 is a [psychovisual](/start-here/psychovisual/) tune labeled VQ. It has been common practice to lean away from the PSNR tune, as it is not designed for visual quality but rather to perform better on the PSNR metric which is widely considered to be inconsistent with our human perception of fidelity. Using the VQ tune is a safe bet for now, but many believe the newer SSIM tune provides better visual fidelity. Using SVT-AV1-PSY, the custom Subjective SSIM tune (Tune 3) provides the best of both Tune 2 & Tune 0 with additional improvements as well. - `--enable-qm 1` Enables quantization matrices, disabled by default. Improves coding efficiency mainly by improving encoding speed while producing similar quality video. - `--qm-min 0` Sets the minimum flatness of quantization matrices to 0, down from the default 8. This is recommended unless you are dealing with extremely heavy grain. The maximum quantization matrix flatness is 15 by default, and should be left alone - `--keyint [FPS*10]` Similar to `--kf-max-dist` in [vpxenc](/tools/software-encoders/vpxenc/), this tells the encoder when to place keyframes. Because SVT-AV1 doesn't have scene detection, this isn't the maximum distance between keyframes, but rather a fixed interval for placing keyframes. If using Av1an, set to -1 to disable keyframe insertion as Av1an handles that instead. - `--irefresh-type 2` Intra refresh is specified through this option, & lets the user decide between Closed GOP & Open GOP. GOP stands for Group of Pictures. Open GOP allows GOPs to [reference one another](https://ottverse.com/closed-gop-open-gop-idr/), but support for this feature is currently incomplete. Therefore, it is recommended to use Closed GOP for the time being via `--irefresh-type 2` until this is rectified. - `--preset X` SVT-AV1 can be used in 14 different presets, labeled -1 through 13. Preset -1 is the slowest, but provides the best coding efficiency; it is also dubbed a research preset that is not recommended for regular use. Preset 13 is the fastest, and is also not recommended for regular use as it makes serious trade-offs to achieve unrealistically fast speeds at the cost of the encoder's coding efficiency. Using presets 2 through 8 is the best course of action for non-realtime applications if you desire reasonable speed, while 9 through 12 are useful for real-time encoding at 1080p or lower, even on low-end consumer computer hardware. - `--crf X` CRF is the best way to target quality for optimal visual fidelity. VBR & CBR lose efficiency due to their inherently limited rate control capabilities. ## Community Forks Currently, there is only one noteworthy community fork of SVT-AV1 called [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/). ### SVT-AV1-PSY SVT-AV1-PSY is a community fork of SVT-AV1 that strives to improve the perceptual fidelity and quality of life provided by the encoder. The goal of this project is to create the best encoding implementation for perceptual quality with AV1, and it aims to surpass previous community forks of [aomenc](/tools/software-encoders/aomenc/#community-forks) in speed and visual quality. SVT-AV1-PSY has a number of feature additions to the mainline SVT-AV1 encoder as well as modified defaults that aim to make it easier to produce a more perceptually optimal bistream. For a full list of the encoder's feature additions and modifications to defaults, see the [project's README](https://github.com/gianni-rosato/svt-av1-psy/blob/master/README.md#feature-additions). --- # SVT-AV1-PSY Canonical URL: https://encode.wiki/tools/software-encoders/svt-av1-psy/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 679da4e950a293c667f9377a99ba83de73e1a6b0ce835f0affd7ed8c8046900c # SVT-AV1-PSY > **Note — Community Fork.** > > This entry is about a fork of SVT-AV1 called SVT-AV1-PSY. If you'd like to learn about the mainline SVT-AV1 encoder before reading, visit our [SVT-AV1 wiki entry](/tools/software-encoders/svt-av1/). SVT-AV1-PSY is a project that aims to enhance the Scalable Video Technology for AV1 Encoder with perceptual enhancements for psychovisually optimal AV1 encoding. The ultimate goal is to create the best encoding implementation for perceptual quality with AV1. The development of this project involves a collaborative effort from a team of dedicated developers and contributors who are committed to improving the encoder's [psychovisual](/start-here/psychovisual/) quality. The SVT-AV1-PSY project is maintained by [Gianni Rosato](https://github.com/gianni-rosato), [Julio Barba](https://github.com/juliobbv), & [Clybius](https://github.com/Clybius), as well as a number of community contributors including [BlueSwordM](https://github.com/BlueSwordM), the maintainer of the [SVT-AV1-PSY AUR package](https://aur.archlinux.org/packages/svt-av1-psy-git) & the original author of [aom-av1-psy](https://github.com/BlueSwordM/aom-av1-psy). The development process involves community testing and optimization to ensure that the encoder and decoder deliver optimal performance. The team uses a variety of tools and methodologies to analyze and improve the performance of the encoder and decoder, including subjective analyses. [SSIMULACRA2](/reference/metrics/ssimulacra2/) and [XPSNR](/reference/metrics/xpsnr/) are used extensively for metrics testing, and the team is committed to improving the overall quality and performance of the encoder using these two metrics as general guidelines and benchmarks. However, the stated goal is not to improve metric scores but to improve the overall perceptual quality of the encoder; naturally, changes are often made to SVT-AV1-PSY that end up degrading metric performance in favor of perceptual fidelity per bit. If you are interested in reading more about the project, you can visit [the project's website](https://svt-av1-psy.com/) for more information. The repository was archived by the owner on Apr 30, 2025. It is now read-only and the author has officially [announced](https://github.com/psy-ex/svt-av1-psy/blob/master/README.md) the end of SVT-AV1-PSY. A fork of the project, [svt-av1-psyex](https://github.com/BlueSwordM/svt-av1-psyex) developed by previous contributor BlueSwordM, was also mentioned by the author in the same announcement. SVT-AV1-PSY contributors are not in any way affiliated with the Alliance for Open Media or any upstream SVT-AV1 project contributors who have not also contributed to the SVT-AV1-PSY project. ## Micro-Release Framework SVT-AV1-PSY features additions outside of mainline SVT-AV1 that are often considered to be significant, but don't line up with a major mainline release. Therefore, the SVT-AV1-PSY release framework is different, and the encoder is currently being developed around micro-releases to make this clear. Micro-releases are tagged with letters starting at `A` to indicate that they contain significant new features and aren't equivalent to mainline releases; for example, `v2.0.0-A` was a micro-release with significant new features exclusive to SVT-AV1-PSY that followed the prior `v2.0.0` release (which came out with PSY + mainline features in tandem with mainline `v2.0.0`). Releases without letters are in-line with mainline SVT-AV1 releases, and may contain significant PSY feature additions as well. ## Feature Additions SVT-AV1-PSY includes a number of new features that are not present in mainline SVT-AV1. These features are designed to improve the visual quality of AV1 encodes, and offer more flexibility when configuring the encoder for a wide range of encoding scenarios. Many of these changes are being integrated into mainline SVT-AV1 as the project matures. The full suite of changes can be found in the [SVT-AV1-PSY README](https://github.com/gianni-rosato/svt-av1-psy/blob/master/README.md). ## Installation Building & installing SVT-AV1-PSY is the same as building & installing mainline SVT-AV1. #### Linux & macOS *A precompiled AVX2-optimized binary of SVT-AV1-PSY can be installed for x86_64 Linux via [rAV1ator CLI](/tools/utilities/rav1ator-cli/). However, it is always recommended to build from source.* To build SVT-AV1 from source, first clone the desired SVT-AV1 repository & enter the build directory. ```bash title="Clone SVT-AV1-PSY" git clone https://github.com/gianni-rosato/svt-av1-psy cd svt-av1-psy/Build/linux ``` In the directory, simply run `./build.sh [flags]` to build. Be aware that building requires CMake version 3.16 or higher and either GCC or Clang. It is recommended to use Clang. ```bash title="Build release" ./build.sh release ``` ```bash title="Statically build just the encoder with clang and enable link-time optimization" ./build.sh jobs=8 all cc=clang cxx=clang++ no-dec enable-lto static native ``` The compiled binaries will be in the `Bin/Release` directory, including SvtAv1EncApp. If you just want the encoder, adding the `no-dec` flag will skip building SvtAv1DecApp and save on compilation time. If you want extra performance, it is possible to build SVT-AV1 using PGO (Profile-guided Optimization). **Be aware that this particular script infers that you have a .y4m file (or multiple) in `/dev/shm` for transcoding**. You can compile statically linked SVT-AV1 with PGO (and LTO, or link-time optimization) by following this script: ```bash title="Bulding SVT-AV1 with profile guided optimization" git clone https://gitlab.com/AOMediaCodec/SVT-AV1/ cd SVT-AV1/Build/linux ./build.sh cc=gcc cxx=g++ enable-lto enable-pgo static native jobs=$(nproc) pgo-dir=/dev/shm pgo-videos=/dev/shm release ``` If you wish to store videos elsewhere or provide custom parameters to the SvtAv1EncApp binary, try this script: ```bash git clone https://gitlab.com/AOMediaCodec/SVT-AV1/ cd SVT-AV1/Build/linux ./build.sh cc=gcc cxx=g++ enable-lto enable-pgo static native jobs=$(nproc) pgo-dir=/dev/shm pgo-compile-gen release ../../Bin/Release/SvtAv1EncApp # Run this binary as many times as you'd like with arguments of your choice to collect data ./build.sh cc=gcc cxx=g++ enable-lto enable-pgo static native jobs=$(nproc) pgo-dir=/dev/shm pgo-compile-use release ``` #### Windows **Clang** is the best option for building in Windows, as it results in faster binaries than building with MSYS2 (either GCC/Clang) and MSVC. > **Important**: All steps below are for **PowerShell**, **not** Command Prompt, but you can use Command Prompt as long as you change the `$env` command to the Command Prompt equivalent. 0. Make sure you have the following prerequisites before beginning the build process: - [LLVM](https://github.com/llvm/llvm-project/releases/latest) - [Microsoft C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools): Inside Build Tools, select "Desktop development with C++" and under optional, only MSVC and the Windows 11 SDK is needed, feel free to unselect other optionals. - [CMake](https://cmake.org/download) - [NASM](https://nasm.us) - [Git](https://git-scm.com/downloads) - [Ninja](https://github.com/ninja-build/ninja/releases/latest) 1. Clone the SVT-AV1-PSY repository: ```bash title="Clone SVT-AV1-PSY" git clone https://github.com/gianni-rosato/svt-av1-psy.git cd svt-av1-psy ``` 2. Configure compilation: ```bash title="Configure SVT-AV1-PSY" $env:Path = "C:\Program Files\LLVM\bin;" + $env:Path $env:CC = "clang" $env:CXX = "clang" cmake --fresh -B svt_build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DSVT_AV1_LTO=OFF -DENABLE_AVX512=ON -DCMAKE_CXX_FLAGS_RELEASE="-flto -DNDEBUG -O2 -march=znver2" -DCMAKE_C_FLAGS_RELEASE="-flto -DNDEBUG -O2 -march=znver2" ``` **Note**: `-march=znver2` has found to be the fastest on Windows, even on newer Ryzen CPUs. This is the case even for Intel CPUs, please try `znver2` first, if it doesn't run, then try `-march=native`, `-march=x86-64-v3`, or potentially older Intel architecture, such as `-march=haswell` or `-march=skylake`. 3. Compile: ```bash title="Build release" ninja -C svt_build ``` The compiled binary will be in the `Bin/Release` directory. ## Projects Featuring SVT-AV1-PSY Some projects that feature SVT-AV1-PSY include: - [Aviator](/tools/utilities/aviator/) ~ an AV1 encoding GUI by @gianni-rosato - [rAV1ator CLI](/tools/utilities/rav1ator-cli/) ~ a TUI for video encoding with Av1an by @gianni-rosato - [SVT-AV1-PSY on the AUR](https://aur.archlinux.org/packages/svt-av1-psy-git) ~ by @BlueSwordM - [SVT-AV1-PSY in CachyOS](https://github.com/CachyOS/CachyOS-PKGBUILDS/pull/144) ~ by @BlueSwordM - [Custom Handbrake Builds](https://github.com/vincejv/docker-handbrake) ~ by @vincejv - [Staxrip](https://github.com/staxrip/staxrip) ~ a video & audio encoding GUI for Windows by @Dendraspis ## License Up to v0.8.7, SVT-AV1 is licensed under the BSD-2-clause license and the Alliance for Open Media Patent License 1.0. Starting from v0.9, SVT-AV1 is licensed under the BSD-3-clause clear license and the Alliance for Open Media Patent License 1.0. SVT-AV1-PSY does not feature license modifications from mainline SVT-AV1. --- # SVT-HEVC Canonical URL: https://encode.wiki/tools/software-encoders/svt-hevc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 98af54156d9e5947fcd9c5f0b61e6cbc7b895f8c3cbae1af8ea0377b3580831b # SVT-HEVC [SVT-HEVC](https://github.com/OpenVisualCloud/SVT-HEVC) (Scalable Video Technology for HEVC) is an open source [H.265 / HEVC](/codecs/video/hevc/) software encoder developed by Intel made specifically to **only support x86**. As the name suggests, it is part of the "[Scalable Video Technology](https://www.intel.com/content/www/us/en/developer/articles/technical/scalable-video-technology.html)" project lineup by Intel. The encoder is written in C with some parts in Assembly and licensed under **BSD+Patent**. > **Note — It is recommended to use [x265](/tools/software-encoders/x265/) instead as it performs much better in quality..** > > ## FFmpeg SVT-HEVC is only available in FFmpeg when it is compiled with their provided [plugin](https://github.com/OpenVisualCloud/SVT-HEVC/tree/master/ffmpeg_plugin). Otherwise it is via `libsvt_hevc`, to check if you have it, run `ffmpeg -h encoder=libsvt_hevc`. ## Supported Color Space SVT-HEVC supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s) | |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUV420P10LE | 4:2:0 | 10-bit | ## Installation #### Linux & macOS To build SVT-AV1 from source, first clone the SVT-HEVC repository & enter the build directory. ```bash title="Clone SVT-HEVC and cd" git clone https://github.com/OpenVisualCloud/SVT-HEVC cd SVT-HEVC/Build/linux ``` In the directory, simply run `./build.sh [flags]` to build. Be aware that building requires CMake version 3.5.1 or higher and either GCC or Clang. It is recommended to use Clang when building SVT-HEVC. ```bash title="Build release" ./build.sh release ``` ```bash title="Statically build release" ./build.sh static release ``` The compiled binaries will be in the `Bin/Release` directory. #### Windows To be filled. If you believe you can help, see our [Contribution Guide](/reference/legacy/contribution-guide/). ## Usage ### Normal usage ```bash title="Simple Y4M input with QP 20, and raw 265 bitstream output" SvtHevcEncApp -i input.y4m -q 20 -b output.265 ``` ```bash title="Preset 4, CRF 20, Y4M input" SvtHevcEncApp -i input.y4m -q 20 -encMode 4 -b output.265 ``` ```bash title="FFmpeg piping" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | SvtHevcEncApp -i stdin -q 20 -encMode 4 -b output.265 ``` > **Danger — SVT-HEVC currently does not have the ability to detect EOF (End of File) within Y4M pipes ([issue](https://github.com/OpenVisualCloud/SVT-HEVC/issues/634#issuecomment-1434911798)), so the encode process will go on indefinitely until your drive is full..** > > The only known solution is to manually set the number of frames to encode via `-n`. ### Encoding HDR ```bash SvtHevcEncApp -i input.y4m -q 20 -encMode 4 -hdr -max-cll XXX -max-fall XXX -master-display G(0.265,0.690)B(0.150,0.060)R(0.680,0.320)WP(0.3127,0.3290)L(1000,0.0100) -b output.265 ``` Remember sure to adjust the HDR metadata accordingly. ### Encoding with Dolby Vision The ability to encode with Dolby Vision via RPU file is surprisingly present within SVT-HEVC, although it is limited to Profile 8.1. ```bash SvtHevcEncApp -i input.y4m -q 20 -encMode 4 -hdr -dolby-vision-rpu RPUFile.bin -dolby-vision-profile 81 -max-cll XXX -max-fall XXX -master-display G(0.265,0.690)B(0.150,0.060)R(0.680,0.320)WP(0.3127,0.3290)L(1000,0.0100) -b output.265 ``` Remember sure to adjust the HDR metadata accordingly. ## List of all configuration parameters As with every SVT line-up, they redirect you to read their "User Guide" instead of presenting what each and every parameter do in the `--help` page. The table below is an exact copy from their user guide taken from [GitHub](https://github.com/OpenVisualCloud/SVT-HEVC/blob/master/Docs/svt-hevc_encoder_user_guide.md) for backup purposes. Feel free to read them. | **Encoder Parameter as shown in the configuration file** | **Command Line parameter** | **Range** | **Default** | **Description** | | --- | --- | --- | --- | --- | | **Channel Number** | -nch | [1 - 6] | 1 | Number of encode instances | | **ConfigFile** | -c | any string | null | Configuration file path | | **InputFile** | -i | any string | null | Input file path and name | | **StreamFile** | -b | any string | null | Output bitstream file path and name | | **ErrorFile** | -errlog | any string | stderr | Error log displaying configuration or encode errors | | **ReconFile** | -o | any string | null | Output reconstructed yuv used for debug purposes. **Note:** using this feature will affect the speed of the encoder significantly. This should only be used for debugging purposes. | | **UseQpFile** | -use-q-file | [0, 1] | 0 | When set to 1, overwrite the picture qp assignment using qp values in QpFile | | **QpFile** | -qp-file | any string | null | Path to qp file | | **SegmentOvFile** | -segment-ov-file | any string | null | Path to segment override file which will allow for sharpness improvement and bit rate reduction on a per segment basis. Refer to config/SVTSegmentOvFile.txt for details. | | **EncoderMode** | -encMode | [0 - 11] | 7 | A preset defining the quality vs density tradeoff point that the encoding is to be performed at. (e.g. 0 is the highest quality mode, 11 is the highest density mode). Section 3.4 outlines the preset availability per resolution | | **EncoderBitDepth** | -bit-depth | [8, 10] | 8 | Specifies the bit depth of input video | | **EncoderColorFormat** | -color-format | [1, 2, 3] | 1 | Specifies the chroma subsampling of input video(1: 420, 2: 422, 3: 444) | | **CompressedTenBitFormat** | -compressed-ten-bit-format | [0, 1] | 0 | Offline packing of the 2bits: requires two bits packed input (0: OFF, 1: ON) | | **SourceWidth** | -w | [64 - 8192] | 0 | Input source width | | **SourceHeight** | -h | [64 - 4320] | 0 | Input source height | | **FrameToBeEncoded** | -n | [0 - 2^31 -1] | 0 | Number of frames to be encoded, if number of frames is > number of frames in file, the encoder will loop to the beginning and continue the encode. 0 encodes the full clip. | | **BufferedInput** | -nb | [-1, 1 to 2^31 -1] | -1 | number of frames to preload to the RAM before the start of the encode. If -nb = 100 and –n 1000 --> the encoder will encode the first 100 frames of the video 10 times. Use -1 to not preload any frames. This parameter is best used to eliminate the impact of disk reading on encoding speed and is most noticeable when frames sizes are 4k or 8k. Because frames are repeated when value specified (-nb) is less than the total frame count (-n), you should expect bitstreams to be different. | | **Profile** | -profile | [1,2] | 2 | 1: Main, 2: Main 10 | | **Tier** | -tier | [0, 1] | 0 | 0: Main, 1: High | | **Level** | -level | [1, 2, 2.1,3, 3.1, 4, 4.1, 5, 5.1, 5.2, 6, 6.1, 6.2] | 0 | 0 to 6.2 [0 for auto determine Level] | | **FrameRate** | -fps | [0 - 2^64 -1] | 60 | If the number is less than 1000, the input frame rate is an integer number between 1 and 60, else the input number is in Q16 format (shifted by 16 bits) [Max allowed is 240 fps]. If FrameRateNumerator and FrameRateDenominator are both !=0 the encoder will ignore this parameter | | **FrameRateNumerator** | -fps-num | [0 - 2^64 -1] | 0 | Frame rate numerator e.g. 6000 When zero, the encoder will use –fps if FrameRateDenominator is also zero, otherwise an error is returned | | **FrameRateDenominator** | -fps-denom | [0 - 2^64 -1] | 0 | Frame rate denominator e.g. 100 When zero, the encoder will use –fps if FrameRateNumerator is also zero, otherwise an error is returned | | **Injector** | -inj | [0,1] | 0 | Enable injection of input frames at the specified framerate (0: OFF, 1: ON) | | **InjectorFrameRate** | -inj-frm-rt | [1 - 240] | 60 | Frame Rate used for the injector. Recommended to match the encoder speed. | | **SpeedControlFlag** | -speed-ctrl | [0,1] | 0 | Enables the Speed Control functionality to achieve the real-time encoding speed defined by –fps. When this parameter is set to 1 it forces –inj to be 1 and -inj-frm-rt to be set to –fps. | | **InterlacedVideo** | -interlaced-video | [0,1] | 0 | 1 : encoder will signal interlaced signal in the stream \
0 : assumes progressive signal | | **SeparateFields** | -separate-fields | [0,1] | 0 | 1 : Interlaced input, application will separate top and bottom fields and encode it as progressive. \
0 : Treat video as progressive video | | **HierarchicalLevels** | -hierarchical-levels | [0 – 3] | 3 | 0 : Flat\
1: 2-Level Hierarchy\
2: 3-Level Hierarchy\
3: 4-Level Hierarchy\
Minigop Size = (2^HierarchicalLevels) \
(e.g. 3 == > 7B pyramid, 2 ==> 3B Pyramid)\
Refer to Appendix A.1 | | **BaseLayerSwitchMode** | -base-layer-switch-mode | [0,1] | 0 | 0 : Use B-frames in the base layer pointing to the same past picture\
1 : Use P-frames in the base layer\
Refer to Appendix A.1 | | **PredStructure** | -pred-struct | [0 – 2] | 2 | 0: Low Delay P\
1: Low Delay B\
2: Random Access\
Refer to Appendix A.1 | | **IntraPeriod** | -intra-period | [-2 - 255] | -2 | Distance between Intra Frame inserted. \
-1 denotes no intra update. \
-2 denotes auto. | | **IntraRefreshType** | -irefresh-type | [-1,N] | -1 | -1: CRA (Open GOP)\
>=0: IDR (Closed GOP, N is headers insertion interval, 0 supported if CQP, >=0 supported if VBR)\
Refer to Appendix A.3 | | **QP** | -q | [0 - 51] | 32 | Initial quantization parameter for the Intra pictures used when RateControlMode 0 (CQP) | | **LoopFilterDisable** | -dlf | [0, 1] | 0 | When set to 1 disables the Deblocking Loop Filtering | | **SAO** | -sao | [0,1] | 1 | When set to 0 the encoder will not use the Sample Adaptive Filter | | **UseDefaultMeHme** | -use-default-me-hme | [0, 1] | 1 | 0 : Overwrite Default ME HME parameters\
1 : Use default ME HME parameters, dependent on width and height | | **HME** | -hme | [0,1] | 1 | Enable HME, 0 = OFF, 1 = ON | | **SearchAreaWidth** | -search-w | [1 - 256] | Depends on input resolution | Motion vector search area width | | **SearchAreaHeight** | -search-h | [1 - 256] | Depends on input resolution | Motion vector search area height | | **ConstrainedIntra** | -constrd-intra | [0,1] | 0 | Allow the use of Constrained Intra, when enabled, this features yields to sending two PPSs in the HEVC Elementary streams \
0 = OFF, 1 = ON | | **RateControlMode** | -rc | [0,1] | 0 | 0 : CQP , 1 : VBR | | **TargetBitRate** | -tbr | Any Number | 7000000 | Target bitrate in bits / second. Only used when RateControlMode is set to 1 | | **vbvMaxrate** | -vbv-maxrate | Any Number | 0 | VBVMaxrate in bits / second. Only used when RateControlMode is set to 1 | | **vbvBufsize** | -vbv-bufsize | Any Number | 0 | VBV BufferSize in bits / second. Only used when RateControlMode is set to 1 | | **vbvBufInit** | -vbv-init | [0 - 100] | 90 | Sets the initial percentage size that the VBV buffer is filled to | | **hrdFlag** | -hrd | [0,1] | 0 | Sets the HRD (Hypothetical Reference Decoder) Flag in the encoded stream, 0 = OFF, 1 = ON When
hrdFlag is set to 1, vbvMaxrate and vbvBufsize must be greater than 0 | | **MaxQpAllowed** | -max-qp | [0 - 51] | 48 | Maximum QP value allowed for rate control use. Only used when RateControlMode is set to 1. Has to be >= MinQpAllowed | | **MinQpAllowed** | -min-qp | [0 - 50] | 10 | Minimum QP value allowed for rate control use. Only used when RateControlMode is set to 1. Has to be < MaxQpAllowed | | **LookAheadDistance** | -lad | [0 - 250] | Depending on BRC mode | When RateControlMode is set to 1 it's best to set this parameter to be equal to the Intra period value (such is the default set by the encoder). When CQP is chosen, then a (2 \* minigopsize +1) look ahead is recommended. | | **SceneChangeDetection** | -scd | [0,1] | 1 | Enables or disables the scene change detection algorithm \
0 = OFF, 1 = ON | | **BitRateReduction** | -brr | [0,1] | 0 | Enables visual quality algorithms to reduce the output bitrate with minimal or no subjective visual quality impact. \
0 = OFF, 1 = ON | | **ImproveSharpness** | -sharp | [0,1] | 0 | This is a visual quality knob that allows the use of adaptive quantization within the picture and enables visual quality algorithms that improve the sharpness of the background. This feature is only available for 4k and 8k resolutions \
0 = OFF, 1 = ON | | **VideoUsabilityInfo** | -vid-info | [0,1] | 0 | Enables or disables sending a vui structure in the HEVC Elementary bitstream. 0 = OFF, 1 = ON | | **HighDynamicRangeInput** | -hdr | [0,1] | 0 | When set to 1, signals HDR10 input in the output HEVC elementary bitstream and forces VideoUsabilityInfo to 1. \
0 = OFF, 1 = ON | | **AccessUnitDelimiter** | -ua-delm | [0,1] | 0 | SEI message, 0 = OFF, 1 = ON | | **BufferingPeriod** | -pbuff | [0,1] | 0 | SEI message, 0 = OFF, 1 = ON | | **PictureTiming** | -tpic | [0,1] | 0 | SEI message, 0 = OFF, 1 = ON. \
If 1, VideoUsabilityInfo should be also set to 1. | | **RegisteredUserData** | -reg-user-data | [0,1] | 0 | SEI message, 0 = OFF, 1 = ON | | **UnregisteredUserData** | -unreg-user-data | [0,1] | 0 | SEI message, 0 = OFF, 1 = ON | | **RecoveryPoint** | -recovery-point | [0,1] | 0 | SEI message, 0 = OFF, 1 = ON | | **TemporalId** | -temporal-id | [0,1] | 1 | 0 = OFF\
1 = Insert temporal ID in NAL units | | **AsmType** | -asm | [0,1] | 1 | Assembly instruction set \
(0: C Only, 1: Automatically select highest assembly instruction set supported) | | **LogicalProcessors** | -lp | [0, total number of logical processor] | 0 | The number of logical processor which encoder threads run on.Refer to Appendix A.2 | | **FirstLogicalProcessor** | -flp | [0, the index of last logical processor] | 0 | The index of first logical processor which encoder threads run on.Refer to Appendix A.2 | | **TargetSocket** | -ss | [-1,1] | -1 | For dual socket systems, this can specify which socket the encoder runs on. Refer to Appendix A.2 | | **ThreadCount** | -thread-count | [0,N] | 0 | The number of threads to get created and run, 0 = AUTO | | **SwitchThreadsToRtPriority** | -rt | [0,1] | 1 | Enables or disables threads to real time priority, 0 = OFF, 1 = ON (only works on Linux) | | **FPSInVPS** | -fpsinvps | [0,1] | 1 | Enables or disables the VPS timing info, 0 = OFF, 1 = ON | | **TileRowCount** | -tile_row_cnt | [1,22] | 1 | Tile count in the Row | | **TileColumnCount** | -tile_col_cnt | [1,20] | 1 | Tile count in the column | | **TileSliceMode** | -tile_slice_mode | [0,1] | 0 | Per slice per tile, only valid for multi-tile | | **UnrestrictedMotionVector** | -umv | [0,1] | 1 | Enables or disables unrestricted motion vectors\
0 = OFF(motion vectors are constrained within frame or tile boundary)\
1 = ON.\
For MCTS support, set -umv 0 with valid TileRowCount and TileColumnCount | | **MaxCLL** | -max-cll | [0 , 2^16-1] | 0 | Maximum content light level (MaxCLL) as required by the Consumer Electronics Association 861.3 specification. Applicable for HDR content. If specified, signaled only when HighDynamicRangeInput is set to 1 | | **MaxFALL** | -max-fall | [0 , 2^16-1] | 0 | Maximum Frame Average light level (MaxFALL) as required by the Consumer Electronics Association 861.3 specification. Applicable for HDR content. If specified, signaled only when HighDynamicRangeInput is set to 1 | | **UseMasterDisplay** | -use-master-display | [0,1] | 0 | Enables or disables the MasterDisplayColorVolume\
0 = OFF\
1 = ON | | **MasterDisplay** | -master-display | For R, G, B and whitepoint [0, 2^16-1]. For max, min luminance [0, 2^32-1] | 0 | SMPTE ST 2086 mastering display color volume SEI info, specified as a string. The string format is “G(%hu,%hu)B(%hu,%hu)R(%hu,% hu)WP(%hu,%hu)L(%u,%u)” where %hu are unsigned 16bit integers and %u are unsigned 32bit integers. The SEI includes X, Y display primaries for RGB channels and white point (WP) in units of 0.00002 and max, min luminance (L) values in units of 0.0001 candela per meter square. Applicable for HDR content. Example for a P3D65 1000-nits monitor,G(13250,34500)B(7500,3 000)R(34000,16000)WP(15635,16 450)L(10000000,1) | | **DolbyVisionRpuFile** | -dolby-vision-rpu | any string | null | Path to the file containing Dolby Vision RPU metadata | | **DolbyVisionProfile** | -dolby-vision-profile | 8.1 or 81 | 0 | Generate bitstreams confirming to the specified Dolby Vision profile 8.1. When specified, enables HighDynamicRangeInput automatically. Applicable only for 10-bit input content. MasterDisplay should be set for using dolby vision profile 81. Pass the dynamic metadata through DolbyVisionRpuFile option | | **NaluFile** | -nalu-file | any string | null | Path to the file containing CEA 608/708 metadata. Text file should contain the userSEI in POC order as per below format: `/`. Currently only PREFIX_SEI messages are supported | --- # SVT-VP9 Canonical URL: https://encode.wiki/tools/software-encoders/svt-vp9/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: cceacd7e7b5b7c2db1563e74f45e207cae817473dd0c9ff9e39c37948cfd09d3 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. SVT-VP9 is a [VP9](/codecs/video/vp9/) encoder developed by Intel.
Like [its](/tools/software-encoders/svt-hevc/) [siblings](/tools/software-encoders/svt-av1/) in the SVT encoder family, it scales very well on multicore processors by default. The [reference encoder](/tools/software-encoders/vpxenc/) beats it in rate control flexibility and supports 10-bit color,
but SVT-VP9 is much faster out of the box. ## FFmpeg There are patches bundled in the SVT-VP9 source code for an FFmpeg plugin that adds the `libsvt_vp9` encoder.
One must recompile FFmpeg with the plugin patch applied to take advantage of it.
(An easy way to do so on Windows is using [media-autobuild_suite](https://github.com/m-ab-s/media-autobuild_suite).) Operation is not too different from the SVT-AV1 FFmpeg integration. Your commands will generally look like this:
`ffmpeg -i video.mp4 -c:v libsvt_vp9 -qp 38 -tune ssim -preset 7 -g 255 video_vp9.webm`
Parameters | Parameter | Description | |------------------|-------------| | -qp | Quantizer value, higher = lower quality. Range is 1..51 in current patches, but -qmin/-qmax can be set as high as 69 for extreme low bitrates. | | -preset | Speed preset. Range is 0..9, with 9 being fastest and default. | | -tune | Quality metric. Can be "vq" (default), "ssim" or "vmaf". | | -g | Size of the Group of Pictures. Range is -2..255, with -1 = no intraframe updates ever, -2 = "auto".
We recommend you set it as high as possible for encode efficiency. | | -rc | Rate control mode.
Can be "cqp" (Constant Quantizer, default), "vbr" (Variable Bitrate) or "cbr" (Constant Bitrate).
(Consider using **vpxenc's two-pass mode** if you really need to match a target bitrate.) | | -level | Encoder level. Range is 1..6. Generally better not to set it. | | -socket | Index of the CPU socket to use. By default it's -1, which uses "all available processors". |
## Supported Color Space SVT-VP9 only supports 8-bit yuv420p. ## Usage ### Standalone To be filled. If you believe you can help, see our [Contribution Guide](/reference/legacy/contribution-guide/). --- # uavs3e Canonical URL: https://encode.wiki/tools/software-encoders/uavs3e/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: df163b79b8b135c0b52100368e8ea0770240ee42f087343f9e9e81717efb8a32 # uavs3e [uavs3e](https://github.com/uavs3/uavs3e/) is an open-source encoder for the [AVS3](/codecs/video/avs3/) codec, developed by Chinese research institutions including Peking University Shenzhen Graduate School, Peng Cheng Laboratory, and Guangdong Bohua UHD Innovation Corporation. The encoder receives infrequent updates, which may result in bugs and compilation issues. It supports AMD64 with AVX2, ARM with NEON, and LoongArch CPUs. ## Installation #### Linux & macOS The developers do not provide pre-built binaries, so you'll need to compile it yourself. Compilation requires GNU Make and CMake. As of August 2024, `uavs3e` does not compile with GCC 14, using Clang 18 instead resolves this issue. ```bash git clone https://github.com/uavs3/uavs3e.git cd uavs3e mkdir build/linux && cd build/linux cmake ../.. -DCOMPILE_10BIT=0 -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ make -j 8 ``` If you encounter undefined calls to `close` or `lseek64` functions, add the following two lines at the top of the `test/utest.c` file: ```c #define _LARGEFILE64_SOURCE #include ``` Binaries will be available in the `build/linux` folder. On Linux, you can run `make install` to install the encoder (may require elevated permissions). #### Windows 1. Ensure you have the following prerequisites installed before starting the build process: - [Microsoft C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/): Select "Desktop development with C++". - [Git](https://git-scm.com/download/win) 2. Open Developer PowerShell for VS 2022. 3. Run the following commands: ```bash git clone https://github.com/uavs3/uavs3e.git cd uavs3e .\version.bat cd build\x86_windows devenv uavs3e.sln /Upgrade msbuild uavs3e.sln /p:Configuration=Release /p:WindowsTargetPlatformVersion=10.0 ``` Binaries will be available in the `bin` folder. ### 10-bit Support #### Linux & macOS To enable 10-bit support, set `-DCOMPILE_10BIT=1` in the CMake command. However, the encoder compiled with this flag has been reported to cause [segmentation faults](https://github.com/uavs3/uavs3e/issues/53) on some systems. #### Windows To enable 10-bit support, change the line `#define COMPILE_10BIT 0` to `#define COMPILE_10BIT 1` in the `inc/com_api.h` file. ## Usage The encoder cannot parse `.y4m` files, they need to be converted to raw video (`.yuv`) format. ```bash title="Simple 8-bit FHD 23.976 fps input with QP 20 and raw avs3 bitstream output" uavs3enc -i input.yuv -w 1920 -h 1080 -d 8 --fps_num 24000 --fps_den 1001 -q 20 -o output.avs3 ``` ```bash title="Speed 2, CRF 20, intra period 120, multithreaded" uavs3enc -i input.yuv -w 1920 -h 1080 -d 8 --fps_num 24000 --fps_den 1001 -p 120 --wpp_threads 8 --frm_threads 8 --speed_level 2 --rc_type 1 -q 20 -o output.avs3 ``` Currently, uavs3e [does not support piping](https://github.com/uavs3/uavs3e/issues/2). ## Notes 1. Usable speed presets range from 0 to 4, where 0 is the slowest and 4 is the fastest. 2. The encoder can be quite competitive, even compared with the newest [AV1](/codecs/video/av1/) and [VVC](/codecs/video/vvc/) encoders in terms of visual fidelity. 3. `uavs3d` can be used to decode the output bitstream. For real-time playback, you need to have [FFmpeg](/tools/utilities/ffmpeg/) compiled with `--enable-libuavs3d`. --- # uvg266 Canonical URL: https://encode.wiki/tools/software-encoders/uvg266/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 37bd284d97d43a5e09fa3a9977cca1321efd993abe47ff32f29742e28493a555 [uvg266](https://github.com/ultravideo/uvg266/) is an open-source software encoder for encoding to the [H.266 / VVC](/codecs/video/vvc/) codec. Developed by the [Ultra Video Group](https://ultravideo.fi/), written in C and licensed under BSD 3-clause. The encoder is based on [Kvazaar](/tools/software-encoders/kvazaar/), their open source [HEVC](/codecs/video/hevc/) encoder solution. ## Installation ### Arch Linux uvg266 is available in the Arch User Repository (AUR) as `uvg266` and `uvg266-git`. ### Compiling Ultra Video Group does not ship any pre-built binaries of their encoders so you'll have to compile them yourself. Windows users are recommended to compile via MinGW-W64 which comes with [MSYS2](https://msys2.org/). > **Caution — Compilation requires GNU Make and CMake.** > > > **Tip — The following build procedure should work across all common operating systems. Using Clang instead of GCC works..** > > You may add `-DCMAKE_CXX_FLAGS="-flto -O3 -march=native" -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt" -DCMAKE_C_FLAGS_INIT="-flto=8 -static"` in CMake for better performance. ```bash git clone https://github.com/ultravideo/uvg266.git cd uvg266/build cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 make -j 8 ``` Binaries will be available in the same folder (`build`), or you can run `make install` on Linux to install (May need elevated permissions). ### 10-bit Support You need to compile with `-DUVG_BIT_DEPTH=10` in the CMake `-DCMAKE_C_FLAGS` option to enable support for encoding 10-bit videos. For example: ```bash -DCMAKE_C_FLAGS="-DUVG_BIT_DEPTH=10" ``` With native optimizations: ```bash -DCMAKE_C_FLAGS="-flto -O3 -march=native -pipe -fno-plt -DUVG_BIT_DEPTH=10" ``` > **Caution — Encoding 10-bit with uvg266 is significantly slower as the developers only prioritized SIMD optimizations for 8-bit, and can get really buggy..** > > ## Usage ```bash title="Simple Y4M input with QP 20 and raw 266 bitstream output" uvg266 -i input.y4m --input-file-format y4m --qp 20 -o output.266 ``` ```bash title="Preset slow, CRF 20, Y4M input" uvg266 -i input.y4m --input-file-format y4m --qp 20 --preset slow -o output.266 ``` ```bash title="FFmpeg piping" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | uvg266 -i - --input-file-format y4m --qp 20 --preset slow -o output.266 ``` ## Troubleshooting 1. `Could not find a strategy for crc32c_8x8!` - You're out of luck, uvg266 failed to initialize its block partitioning strategy for your specific CPU instruction set, so you **can't use the encoder or encode that specific video**. --- # vpxenc Canonical URL: https://encode.wiki/tools/software-encoders/vpxenc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e0c93b984a6b9e43908c8cbeeaac6f0623e98448f3b40f817b3af8f312cb3cf5 # vpxenc > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. vpxenc is part of the libvpx library for working with the [VP9](/codecs/video/vp9/) & [VP8](/codecs/video/vp8/) video codecs. It is capable of encoding & decoding both formats, where vpxenc is the multipurpose encoder. VP9 competes with [HEVC](/codecs/video/hevc/) (h265) & [AVC](/codecs/video/avc/) (h264) in coding efficiency, and has been superseded by [AV1](/codecs/video/av1/). VP8 competes with AVC. By default, vpxenc isn't as competitive as it could be, but even when used properly, most tests show that h265 offers slightly better quality per bit with efficient encoders like [x265](/tools/software-encoders/x265/). ## FFmpeg vpxenc is available in FFmpeg via `libvpx` for **VP8** and `libvpx-vp9` for **VP9**, to check if you have it, run `ffmpeg -h encoder=libvpx` or `ffmpeg -h encoder=libvpx-vp9`. Non-FFmpeg standard VP8/VP9 parameters are **not** supported. ## Supported Color Space vpxenc supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s) | |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUVA420P | 4:2:0 | 8-bit (Alpha Channel) | | YUV422P | 4:2:2 | 8-bit | | YUV440P | 4:4:0 | 8-bit | | YUV444P | 4:4:4 | 8-bit | | GBRP | - | 8-bit | | YUV420P10LE | 4:2:0 | 10-bit | | YUV422P10LE | 4:2:2 | 10-bit | | YUV440P10LE | 4:4:0 | 10-bit | | YUV444P10LE | 4:4:4 | 10-bit | | GBRP10LE | - | 10-bit | | YUV420P12LE | 4:2:0 | 12-bit | | YUV422P12LE | 4:2:2 | 12-bit | | YUV440P12LE | 4:4:0 | 12-bit | | YUV444P12LE | 4:4:4 | 12-bit | | GBRP12LE | - | 12-bit | ## Installing (Binary) Windows builds are available on Lastrosade's [website](https://jeremylee.sh/bins/) and can be downloaded [here](https://jeremylee.sh/bins/vpx.7z). For Linux and MacOS, it may be be available when searching "vpxenc" or "libvpx" in their respective package managers. ## Compiling (Windows/MacOS/Linux) Windows users are recommended to compile via MinGW-W64 which comes with [MSYS2](https://msys2.org/). nasm/yasm, and the GNU build tools (make, configure) are required for this operation. ### Cloning First, cloning ```bash git clone https://chromium.googlesource.com/webm/libvpx cd libvpx mkdir libvpx_build && cd libvpx_build ``` ### ./configure file Now here comes the annoying part, the configure file have really bad defaults. So you will need to adjust them, here are some recommended options you should use: ```bash ../configure --cpu=native --extra-cxxflags="-ffat-lto-objects -flto" --extra-cflags="-ffat-lto-objects -flto" --as=auto --enable-vp9-highbitdepth --enable-libyuv --enable-webm-io --enable-vp9 --enable-runtime-cpu-detect --enable-internal-stats --enable-postproc --enable-vp9-postproc --enable-static --disable-shared --enable-vp9-temporal-denoising --disable-unit-tests --disable-docs --enable-multithread ``` Now let's break down what each of them do. - `--cpu=native` Native CPU optimizations. - `--extra-cxxflags="-ffat-lto-objects -flto" --extra-cflags="-ffat-lto-objects -flto"` More CPU optimizations for faster encoding. - `--as=auto` Set the assembler to auto, so it can choose between `yasm` and `nasm`. - `--enable-vp9-highbitdepth` Enables high bit depth (>=10 bits) when encoding VP9. - `--enable-libyuv` Enables YUV4MPEG input support (IMPORTANT), otherwise it will only accept RAW. - `--enable-webm-io` Enables input and output support for WebM container. - `--enable-vp9` Enables VP9 encoding support. - `--enable-runtime-cpu-detect` Enables runtime CPU detection. - `--enable-internal-stats` Enables internal statistics for the encoder for debug purposes. - `--enable-postproc` Enables postprocessing stuff for better video quality. - `--enable-vp9-postproc` Enables VP9-specific postprocessing stuff for better video quality. - `--enable-static` Enables static builds. - `--disable-shared` Disables shared builds. - `--enable-vp9-temporal-denoising` Disables spatial denoising for VP9 and enables temporal instead. - `--disable-unit-tests` Disables unit tests, unless you want to test the encoder as a developer. This should be disabled. - `--disable-docs` Disables documentation, as enabling this also requires doxygen. - `--enable-multithread` Enables the usage of multiple CPU threads for encoding and decoding. ### Other ./configure options There are other options you may want use to either speed up compiliation or drop unwanted features. 1. `--disable-vp8 --disable-vp9-decoder --disable-vp8-decoder` Disables VP8 encoding and `vpxdec` (decoder) to be compiled. 2. `--enable-small` Prioritizes smaller encoder binary size over encoding speed. 3. `--target=` Enables target compilation for a specific operating system or CPU architecture. There's a lot of them. Here's an exhaustive list of all of them based on the configure file: ```bash arm64-android-gcc arm64-darwin-gcc arm64-darwin20-gcc arm64-darwin21-gcc arm64-darwin22-gcc arm64-darwin23-gcc arm64-linux-gcc arm64-win64-gcc arm64-win64-vs15 arm64-win64-vs16 arm64-win64-vs16-clangcl arm64-win64-vs17 arm64-win64-vs17-clangcl armv7-android-gcc armv7-darwin-gcc armv7-linux-rvct armv7-linux-gcc armv7-none-rvct armv7-win32-gcc armv7-win32-vs14 armv7-win32-vs15 armv7-win32-vs16 armv7-win32-vs17 armv7s-darwin-gcc armv8-linux-gcc loongarch32-linux-gcc loongarch64-linux-gcc mips32-linux-gcc mips64-linux-gcc ppc64le-linux-gcc sparc-solaris-gcc x86-android-gcc x86-darwin8-gcc x86-darwin8-icc x86-darwin9-gcc x86-darwin9-icc x86-darwin10-gcc x86-darwin11-gcc x86-darwin12-gcc x86-darwin13-gcc x86-darwin14-gcc x86-darwin15-gcc x86-darwin16-gcc x86-darwin17-gcc x86-iphonesimulator-gcc x86-linux-gcc x86-linux-icc x86-os2-gcc x86-solaris-gcc x86-win32-gcc x86-win32-vs14 x86-win32-vs15 x86-win32-vs16 x86-win32-vs17 x86_64-android-gcc x86_64-darwin9-gcc x86_64-darwin10-gcc x86_64-darwin11-gcc x86_64-darwin12-gcc x86_64-darwin13-gcc x86_64-darwin14-gcc x86_64-darwin15-gcc x86_64-darwin16-gcc x86_64-darwin17-gcc x86_64-darwin18-gcc x86_64-darwin19-gcc x86_64-darwin20-gcc x86_64-darwin21-gcc x86_64-darwin22-gcc x86_64-darwin23-gcc x86_64-iphonesimulator-gcc x86_64-linux-gcc x86_64-linux-icc x86_64-solaris-gcc x86_64-win64-gcc x86_64-win64-vs14 x86_64-win64-vs15 x86_64-win64-vs16 x86_64-win64-vs17 generic-gnu ``` For **Windows** compilation with MinGW you may need to use `--target=x86_64-win64-gcc` and `--target=arm64-darwin22-gcc` for **MacOS**. ### Running GNU make After successfully running the configure command above, run `make -j $(nproc)` to start compiling with your CPU count. The resulting binary will be called `vpxenc` and you can copy it wherever you like. ## VP8 *Incomplete* ## VP9 For encoding VP9, vpxenc's default parameters are not considered optimal. There are a lot of options that are either disabled without reason or are simply misconfigured, hurting coding efficiency at little cost otherwise. As of mid-2021, some parameters (the TPL-model, lag-in-frames and auto-alt-ref frames) were changed (since libvpx 1.9.0 and libvpx 1.10.0) which means that there's not much use of setting these three parameters unless you're in [FFmpeg](/tools/utilities/ffmpeg/). This section covers the most important options libvpx-vp9 has to offer, recommended settings, & what they do. It is important to note that the vpxenc parameters provided below are considered optimal because they are efficient, but VP9 Profile 2 isn't compatible with many hardware-accelerated VP9 decoding implementations. ### Encoding - `--codec=vp9` Self-explanatory. - `--passes=2` vpxenc's 2-pass mode is quite fast compared to 2-pass in [x264](/tools/software-encoders/x264/) and x265. Only use 1-pass mode for real-time applications, which won't be covered here yet. It is the default in the standalone vpxenc libvpx-vp9 encoder. - `--webm` Enables WebM output for the encoder, and passes the encoder flags set. It is not necessary to enable it, but since it passes the encoder flags, I would use it. Can be changed to `--ivf` for an ivf video stream. - `--good` This is a sort of quality deadline, the minimum speed the encoder is allowed to go to. It isn't recommended to use `--best` as it is slow for the quality uplift you get. Do not use RT for anything but real-time encoding. - `--threads=8` Dictates the number of threads the encoder should spawn. It doesn’t mean it’ll scale all that well over those 8 threads. On a 16 thread CPU with a single encoder instance, I would use 8 threads. With multiple encoder instance encoding(with qencoder/av1an/neav1e), I would set it to 2 threads. - `--profile=2` VP9 profile 2 is obligatory if you want 10-bit & 12-bit support for HDR, and improved quality from 8-bit. - `--lag-in-frames=25` Lag-in-frames is the libvpx equivalent of lookahead in x264. The higher the number, the slower the encoder will be, but at the upside of making it more efficient. Going above –lag-in-frames=12 also activates another setting called alternate reference frames. 25 is the maximum you can get in libvpx-vp9. It is the default in the standalone vpxenc libvpx-vp9 encoder. - `--end-usage=q` Q mode is the closest equivalent to CRF that libvpx-vp9 offers, so use it if maximum quality is desired. - `--cq-level=25` For 1080p30 8-bit content, it is recommended to go with a Q of 25; you can go lower if you value higher quality over pure efficiency. For 1080p60 8-bit content, I would recommend going with a higher Q value with a delta of around 15. So, a Q of 30 to 40 is usually recommended. Depending on the content, you may have to tune this value, so this advice is only useful in choosing a starting point. - `--kf-max-dist=[input FPS * 10]` This tells the encoder to have a maximum number of frames between keyframes. It will usually place a lower number of keyframes in content like movies, TV shows, or animated shows, so you can set it to a very high number or not set it at all if you want maximum efficiency for this kind of content. Otherwise, I would go with the 10-second rule: `--kf-max-dist=240` for 24FPS content, 300 for 30FPS content, 600 for 60FPS content, and so on. - `--cpu-used=3` This is where the biggest balance of quality to speed is with libvpx-vp9. This is similar to presets in x264 and x265, except the lower the number, the slower the encoder takes. Using `--cpu-used=3` & below enables RDO, which increases quality at the expense of speed. > **Note — `--cpu-used=5` and above are *slower* in the 1st pass, so it isn't recommended to use them anyway..** > > - `--auto-alt-ref=6` Activates alternate reference frames. Alternate reference frames are "invisible" frames which are used as references when creating the final display frames. More alternate reference frames is typically more efficient. Setting this greater than 1 activates overlay frames and isn't compatible with the 8-bit color profiles. - `--arnr-maxframes=7` This is the maximum number of alternate reference frames the encoder is allowed to use. For most content, 7 is usually a good bet, and it is the default. With animated content, going with a value of 12 or to the max is a good bet, as animated content benefits from more additional alt-ref frames than other content. Be aware that increasing this value will impact encode speed. - `--arnr-strength=4` This setting dictates how much denoising will occur in the alt-ref frames. Lowering it to 2 or 3 is usually a good bet for noisier/grainy content to try and retain more detail, but 4 is a sane starting place. The default setting is 5, which is fine for most content, but it can be beneficial going a bit lower. For animation, keeping the default of 5 is likely a better option. - `--aq-mode=0` Adaptive quantization is the way for an encoder to spend more bits in certain areas to improve [psychovisual fidelity](/start-here/psychovisual/). `--aq-mode=0` works well on clean content (animation, video games, screen content). `--aq-mode=2` is recommended when you want to give more detail to more complex parts of a video. - `--frame-boost=1` This flag lets the encoder periodically boost the bitrate of a scene/frame if it needs it. Leaving it at the default `--frame-boost=0` is usually a good bet, & this isn't a particularly salient change. - `--tune-content=default` This determines how the encoder is tuned. In libvpx-vp9, there are three options: `default`, `screen`, and `film`. Default is for most scenarios, screen is for screen content(video games, live-streaming content like web pages & your screen), and film is for heavily dithered/grainy video. Leaving it at the default for about everything but screen content as described above is probably the best option. `--tune-content=screen` with `--aq-mode=2` is not recommended, as it creates some odd artifacts. It is advised to use `--aq-mode=0` if `--tune-content=screen` is activated, or if you want better perceptual quality, `--aq-mode=1`. - `--row-mt=1` Enables row multi-threading in libvpx-vp9. *Always* enable it no matter what, as it does not hurt efficiency, but boosts speed considerably. This feature is disabled by default. - `--bit-depth=10` Always use 10-bit for maximum efficiency & minimal banding, even with an 8-bit source. Make sure to enable `--profile=2` as mentioned above. - `--tile-columns=1` This setting divides the video into tile columns for easier parallelization when encoding & decoding. Setting `--tile-columns=1`, you will get 2¹ tile columns. Setting it higher is a trade-off between parallelization & coding efficiency, as more tiles means less information your encoder can work with, and this will result in decreased efficiency. Do note there is an upper threshold in regards to the number of tile columns you can get due to the fixed minimum tile width of 256 pixels. So, this means 4 tile columns (2²) for 720p and 1080p, 8 tile columns (2⁴) for 1440p/4k, and so on. If you set a tile column number that is too high, it will drop down to the lowest supported number of tile columns at the input resolution. - `--tile-rows=0` This setting divides the video into tile rows. This option is different from columns because although it also makes decoding performance higher, it does not scale as well as tile columns & doesn’t increase encoder threading nearly as much. Always use more tile-columns than rows, or leave the number of tile rows at default (0). Leaving the encoder defaults at `--tile-rows=0` & `--tile-columns=0` will result in the highest overall coding efficiency possible with these options. - `--enable-tpl=1` This option enables a temporal layer model, which helps with coding efficiency. It is the default in the standalone vpxenc libvpx-vp9 encoder. All of these options are only available for the standalone vpxenc program. Here is a sample FFmpeg command line interpretation of the commands above, with some options missing: ```bash ffmpeg -i input.mkv -c:v libvpx-vp9 -pix_fmt yuv420p10le -pass 1 -quality good -threads 4 -profile:v 2 -lag-in-frames 25 -crf 25 -b:v 0 -g 240 -cpu-used 3 -auto-alt-ref 6 -arnr-maxframes 7 -arnr-strength 4 -aq-mode 0 -tune-content default -tile-rows 0 -tile-columns 1 -enable-tpl 1 -row-mt 1 -f null - ffmpeg -i input.mkv -c:v libvpx-vp9 -pix_fmt yuv420p10le -pass 2 -quality good -threads 4 -profile:v 2 -lag-in-frames 25 -crf 25 -b:v 0 -g 240 -cpu-used 3 -auto-alt-ref 6 -arnr-maxframes 7 -arnr-strength 4 -aq-mode 0 -tune-content default -tile-rows 0 -tile-columns 1 -enable-tpl 1 -row-mt 1 output.mkv ``` Alternatively, you can pass a raw .y4m stream to standalone vpxenc & encode that way. *VP9 section written based on work by BlueSwordM, who has granted written permission for this wiki page to exist in its current fashion* --- # VTM Canonical URL: https://encode.wiki/tools/software-encoders/vtm/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 99d94000fb94ec867cd8a27ff968a752c6308c6384791abf26e9fc5bd250864d > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! VTM is the original [H.266](/codecs/video/vvc/) (better known as VVC) reference encoder, in competition with alternatives like [VVenC](/tools/software-encoders/vvenc/). In the modern day, it joins other MPEG reference encoders such as [HM](/tools/software-encoders/hm/) & [JM](/tools/software-encoders/jm/) in their reputations for being highly niche offerings that are used rarely due to their usage complexity & speed disadvantages; however, VTM may be more useful due to the current difficulty facing VVC encoding regardless of the encoding implementation one chooses to use. x264 is more efficient than JM. --- # VVenC Canonical URL: https://encode.wiki/tools/software-encoders/vvenc/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: f80795252f74f1c45870e662331fa867b0733a5ad6622e9bb7afa28939c69a3e # VVenC vvencapp spamming the terminal output with progress per-picture-order-count cause god knows why VVenC is an open source command line application for encoding [H.266/VVC](/codecs/video/vvc/) written in C++ and developed by Fraunhofer Heinrich-Hertz-Institute (HHI). ## Installation #### Linux & macOS Arch Linux users may check the AUR for the `vvenc` and `vvenc-git` packages. Compilation from source as shown belowe requires CMake. ```bash git clone https://github.com/fraunhoferhhi/vvenc.git cd vvenc mkdir build cd build cmake .. -DCMAKE_BUILD_TYPE=Release cmake --build . ``` Binaries will be available in `bin/release-static/` ### FFmpeg Integration Since 15th June 2024, you can officially compile your own FFmpeg binary with `libvvenc`. If you use earlier FFmpeg revisions you will need to apply a custom patch. A comprehensive tutorial can be found in the [official VVenC wiki](https://github.com/fraunhoferhhi/vvenc/wiki/FFmpeg-Integration) ([archive](https://web.archive.org/web/20230509115800/https://github.com/fraunhoferhhi/vvenc/wiki/FFmpeg-Integration)). A VVC decoder has been [integrated into FFmpeg](https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/301ed950d1c3c500d0c2eee8472587dc5e691c04) in early 2024, FFmpeg 7.0 is the first release to support it. However if you still want to compile FFmpeg with the `vvdec` decoder, you can refer to the [official VVenC wiki](https://github.com/fraunhoferhhi/vvdec/wiki/FFmpeg-Integration) ([archive](https://web.archive.org/web/20240809090834/https://github.com/fraunhoferhhi/vvdec/wiki/FFmpeg-Integration)). There is an FFmpeg fork called [FFmpeg-VVC](https://github.com/MartinEesmaa/FFmpeg-VVC). It is maintained by Martin Eesmaa, who is the author of the [VVCEasy](https://github.com/MartinEesmaa/VVCEasy) GitHub repo featuring guides and more general information about working with VVC. Please be aware the script below produces a build of FFmpeg licensed under *LGPL version 2.1 or later*. It is legal to distribute the resulting binary, given you do not interfere with the flags provided below. ```bash title="Build FFmpeg with libvvdec, libvvenc, and fdk-aac" brew install libxml2 ffmpeg nasm # macOS-only; if on Linux, use your native package manager. Package names may differ. git clone https://github.com/fraunhoferhhi/vvenc git clone https://github.com/fraunhoferhhi/vvdec git clone https://github.com/mstorsjo/fdk-aac cd vvenc && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local .. sudo cmake --build . --target install -j $nproc cd ../../ cd vvdec && mkdir build && cd build cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local .. sudo cmake --build . --target install -j $nproc cd ../../ cd fdk-aac && ./autogen.sh && ./configure make -j sudo make install cd ../ git clone --depth=1 https://github.com/MartinEesmaa/FFmpeg-VVC cd FFmpeg-VVC export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --enable-libfdk-aac --enable-libvvenc --enable-libvvdec --enable-static --enable-pic --enable-libxml2 --pkg-config-flags="--static" --enable-sdl2 make -j ``` Binaries will be available in the final directory you end up in after the build process is complete. #### Windows To be filled. If you believe you can help, see our [Contribution Guide](/reference/legacy/contribution-guide/). ## Usage There are two encoders, the simple encoder (`vvencapp`) and the full-featured expert mode encoder (`vvencFFapp`) which is based on the [VTM](/tools/software-encoders/vtm/) configuration scheme. VVenC used to only accept YUV files input until support was added for Y4M. Here are some examples: ```bash title="Standard VVenC input" vvencapp -i input.y4m --qp 20 -o output.266 ``` ```bash title="Preset slow + qpa (already default) + YUV420P10" vvencapp -i input.y4m --preset slow --qpa on --qp 20 -c yuv420_10 -o output.266 ``` ```bash title="Piping with FFmpeg" ffmpeg -hide_banner -loglevel error -i input.mkv -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | vvencapp -i - --y4m --preset medium --qpa on --qp 20 -c yuv420_10 -o output.266 ``` ```bash title="FFmpeg preset fast + qp 32 muxing to mp4" ffmpeg -i input.mkv -c:v libvvenc -qp 32 -preset fast out.mp4 ``` > **Note — FFmpeg vvenc plugin only supports yuv420p10le output pixel format which means resulting video will always have 10 bit color depth..** > > > **Note — QPA.** > > VVenC by default operates with QP (Quantization Parameter), which is basically fixed quality. For "CRF-like" rate control, QPA is enabled by default > QPA (provided by --qpa) enables perceptually motivated QP adaptation based on [XPSNR](/reference/metrics/xpsnr/). QPA modifies the QP value on the fly spatially and temporally as well as enabling temporal RDO. --- # x264 Canonical URL: https://encode.wiki/tools/software-encoders/x264/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 8ee97cb87b1d835c1f1bbda048082b323c24cba29a9cf72f2f634bba01308157 x264 is a software library and command line application for encoding H.264 / AVC developed by VideoLAN, the people behind the ever-popular VLC Media Player and released under GNU GPL. It is written in C and Assembly with almost two decades worth of development and threading optimizations which makes it the fastest software video encoder available, which also happens to be extremely popular. x264 has great fine detail retention which makes it perfect for high fidelity content. ## FFmpeg x264 is available in FFmpeg via `libx264`, to check if you have it, run `ffmpeg -h encoder=libx264`. You can input non-FFmpeg standard x264 parameters via `-x264-params`. ## Supported Color Space x264 supports the following color spaces: | Format | Chroma Subsampling | Supported Bit Depth(s) | |------------------|:------------------:|-----------------------| | YUV420P | 4:2:0 | 8-bit | | YUVJ420P | 4:2:0 | 8-bit (Full range) | | YUV422P | 4:2:2 | 8-bit | | YUVJ422P | 4:2:2 | 8-bit (Full range) | | YUV444P | 4:4:4 | 8-bit | | YUVJ444P | 4:4:4 | 8-bit (Full range) | | NV12 | Semi-planar | 8-bit | | NV16 | Semi-planar | 8-bit | | NV21 | Semi-planar | 8-bit (reversed) | | GRAY8 | - | 8-bit | | YUV420P10LE | 4:2:0 | 10-bit | | YUV422P10LE | 4:2:2 | 10-bit | | YUV444P10LE | 4:4:4 | 10-bit | | GBRP10LE | - | 10-bit | | GRAY10LE | - | 10-bit | ## Usage x264 has been praised for its simple, no-fuss settings. Here are some examples: ```bash title="Simple raw Y4M input with CRF 20 and raw 264 bitstream output" x264 --crf 20 -o output.264 input.y4m ``` ```bash title="Preset slow, CRF 20, Y4M input" x264 --preset slow --crf 20 -o output.264 input.y4m ``` These next couple of examples utilize [FFmpeg](/tools/utilities/ffmpeg/) to pipe video into x264. ```bash title="FFmpeg piping" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | x264 --preset slow --crf 20 --demux y4m - -o output.264 ``` ```bash title="FFmpeg piping, MKV output" ffmpeg -v error -i input.mkv -f yuv4mpegpipe -strict -1 - | x264 --preset slow --crf 20 --demux y4m - -o output.mkv ``` > **Note — Output Containers.** > > x264 can output ``264``, MKV, FLV (Flash Video), and MP4 (If compiled with GPAC or L-SMASH support). For more information about what containers are, see the ["Terminology" section on containers](/start-here/terminology/#container). > **Caution — x264 will use Haali Matroska Muxer for MKV outputs, which has seeking issues. It is recommended to remux back using FFmpeg or mkvmerge/MKVToolNix..** > > ## Recommendations As x264 is made to "just work", there aren't many advanced parameters to modify. The general guideline is to encode as slowly as you can tolerate. ### Preset ``--preset veryslow`` The most obvious way to increase fidelity per bit is to allow the encoder to spend more effort, and therefore time, encoding. This preset is decently slow, but preset ``placebo`` is even slower. ### Threads ``--threads X`` This is set to 1.5x the number of available cores by default, and shouldn't need to be adjusted unless you need to reduce the number of threads for some reason. x264's threading is very efficient, and in most cases, x264 should be able to completely saturate most consumer CPUs. ### Open GOP ``--open-gop`` Enables Open GOP (Group of Pictures), where each GOP can [reference one another](https://ottverse.com/closed-gop-open-gop-idr/), thus improving compression with little speed loss. For unknown reasons it is disabled by default in x264. ### AQ Mode ``--aq-mode 3`` In short, will make x264 bias to dark areas and spend more bitrate there, thus dark scenes will look less bad. Basically no speed loss. ### Reference Frames ``--bframes 8 --ref 12`` These parameters are responsible for the amount of reference frames x264 will use for compression, the more the better. Maximum of 16, will definitely increase compute time the higher you go. ### MB Tree ``--no-mbtree`` This option disables mb-tree rate control. mb-tree rate control is intended to redistribute bitrate to give more bits to frames that have less motion, and less bits to frames that have more motion (because artifacts in those frames will be less noticeable when the video is playing). However, many users claim this can have negative effects on video quality, especially when encoding videos that have significant amounts of grain. ### Lossless Encoding x264 can also encode lossless video, allowing it to compete with lossless video codecs like [FFV1](/codecs/video/ffv1/) and [UT Video](/codecs/video/utvideo/). To encode lossless video, use `--qp 0`. Slower presets will decrease the size even further while the video remains lossless. However, when space is not a concern, such as when encoding an intermediate lossless for later encoding to a lossy format, x264 can become the fastest lossless codec available, providing super fast encoding and decoding. For the absolute fastest encoding and decoding, one can use both `--preset ultrafast` and `--tune fastdecode`. However, `--preset superfast` still provides incredibly fast encoding and decoding speed, with 20-30% space savings for lossless compared to `ultrafast`, so it may be the ideal choice for many users. > **Note — Why QP 0 instead of CRF 0?.** > > CRF automatically adjusts a number of quantization parameters to achieve a desired quality output. QP stands for *Quantization Parameter*, and allows full control over the resulting video quality. In this case, all-intra or lossless, using QP is necessary, but for lossy encoding CRF will produce better visual fidelity per bit. --- # x265 Canonical URL: https://encode.wiki/tools/software-encoders/x265/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 5d78e5c870c51c8f5ef3c69fb33a87f0fceebaa7c50606d819c580e4dda8914a x265 is a software library and command line application for encoding [H.265 / HEVC](/codecs/video/hevc/) developed by MulticoreWare, written in C++ and x86 assembly, and released in 2013. By default, x265 is tuned for low-bitrate content due to the blurring filters it applies, but it can be tuned for high fidelity as well and excels at it. x265 is currently not recommended for lossless encoding. For that niche, x264 is considerably faster without meaningful efficiency loss. # FFmpeg x265 is available in FFmpeg via `libx265`, to check if you have it, run `ffmpeg -h encoder=libx265`. # Parameters This section will overview the most important parameters for controlling output and quality in x265. The parameters will be listed in the format used by the standalone x265 binary, but all the parameters should also be usable in FFmpeg in the format e.g. `-x265-params pass=1`. ## Preset `--preset` In x265, we have a preset system to easily control how much effort and therefore time x265 puts into compressing video. This system possesses the following presets: ultrafast, superfast, faster, fast, medium (default), slow, slower, veryslow and placebo Generally speaking the further left of medium the preset is, the faster and therefore less efficient the encode is, and the opposite is true for the right side. It is recommended to pick the slowest preset you can bear to use on your hardware before messing with any of the following settings on this page, as this will be your baseline for encoder performance. It is not recommended to use the two extremes as ultrafast is terrible in its consistency and you would be better off using something like x264. placebo on the other hand performs slightly worse than veryslow for reasons that will be explained later. ## Rate Control ### `--CRF` CRF or Constant Rate Factor is the closest thing in x265 to a quality slider for the final encode where a smaller value is higher quality and a larger value is lower quality. While CRF offers a good overall consistency, the exact quality you get at a specific CRF does still vary by source, although not as much as other methods. Changing specific settings (like psy-rd) or the preset you're using can change the quality at a given CRF. Usually the slower the preset, the higher the quality given a CRF. Below is a list of rough CRF values to experiment with when targeting a specific quality. This is only rough as it is source and settings dependant as explained earlier. | Quality | 720 | 1080 | 4k | |---------------- |----- |------ |---- | | Transparent | 14 | 16 | 18 | | High Quality | 18 | 22 | 24 | | Medium Quality | 24 | 26 | 28 | | Low Quality | 28 | 30 | 32 | While not always consistent, x265 aims for an increase of 6 CRF or a decrease of 6 CRF to be a half or doubling of file size, respectively. CRF is also preferable to other rate control methods due to its consistency in quality and being the most efficient mode of them all. ### `--bitrate` `--bitrate` as the name implies, is used to specify the target ABR for 1-pass encoding or target bitrate for VBR in 2-pass+. It is highly discouraged to use this rate control unless required, as it is less efficient than CRF and does not keep a consistent quality like CRF either. ## Threading `--wpp` and `--frame-threads` x265 possesses several threading features that each come with their own upsides and downsides when used. ### `--wpp` `--wpp` or Wavefront Parallel Processing is the default method x265 uses to parallelize encoding first and foremost. With only an efficiency hit of 1-3% of the final encode, it increases threading of x265 by 3-5x. Unless you're using a tool like Av1an for maximum efficiency encoding, it is always recommended to have this setting on (default). WPP works by splitting the video frame up into rows where the row is 2 CU, or super blocks in libvpx terms, behind the row above it. This allows the encoder to reference everything allowed by the H.265 specification. Due to it not being single threaded, some optimizations cannot be done, resulting in a small efficiency loss. ### `--frame-threads` `--frame-threads` is an extension of the idea of wpp across multiple frames being encoded concurrently. Similarly, this results in a further 1-3% efficiency loss, however it can boost threading 5-7x on top of wpp. According to reports by some users in older versions of x265, the larger the number of frame threads, the worse the efficiency impact. However, in the latest version of x265 4.1, there is no difference between 2 frame threads and the max 16 threads. ## B-frames `--bframes`, `--b-adapt`, `--bframe-bias` and `--b-pyramid` B-frames are bi-directional predictive frames. This means they can reference either past, future or both types of frames at the same time, making them one of the most efficient types. B-frames however are usually the most compressed frame types. Referencing other frames rather than storing their own detail (typically) makes them often the worse quality and blurry frames. In total, we have 4 different settings to control how they are used by the encoder. Usually, you will never touch these. ### `--bframes` `--bframes` is used to control the max number of consecutive B-frames the encoder is allowed to use and search for. Generally, the larger this value is to a max of `16` the slower but more efficient the encoder becomes. However, in higher fidelity encodes around CRF 18 or lower, beyond 8-12 B-frames can start to hurt detail for the reasons stated above. Likewise, the benefits of more B-frames vs the time it takes makes it not typically worth it beyond a similar point. In general, it is best to leave this setting at the default for your preset. However, for content like anime or otherwise flat non IRL, adding +2 B-frames to the current preset generally gives a small efficiency boost. You can also set it to the max value of 16 for micro encodes to try and squeeze the most efficient possible out of the encoder, if time is no object to you. ### `--b-adapt` `--b-adapt` controls the method that the encoder uses to decide where to put B-frames. It has 3 different modes. | mode | feature | |--------|----------------------------| | 1 | Fixed | | 2 | Light lookahead | | 3 | Viterbi trellis (default) | It is always recommended to use 3 as it comes at a negligible speed penalty and is the smartest at placing B-frames for maximum efficiency without harming visual fidelity. ### `--bframe-bias` `--bframe-bias` as the name implies, is used to control the bias for x265 to use B-frames over other frame types. Normally,you should never touch this setting as the default of `0` is already optimal for most cases, however for high fidelity encodes or micro encodes, adjusting this bias can help retain detail or sacrifice spacial detail for better temporal information. ### `--b-pyramid` `--b-pyramid` or `--no-b-pyramid` decides if B-frames can be used as reference frames for other B-frames or other frame types. Normally, this should always be on and is the default. However, for high fidelity encodes, it might be worthwhile experimenting with turning it off only in limited use cases. ## Restoration Filtering `--sao`, `--limit-sao`, `--no-sao` and `--deblock` ### `--sao`, `--limit-sao` and `--no-sao` SAO or Sample Adaptive Offset is a restoration filter in H.265 used to prevent obvious blocking and ringing artifacts, especially around sharp edges. However, this does sometimes come at the cost of some finer details in the video, like for example human skin and surface details, to be erased. Generally speaking, at CRF values at or above 20, you can leave this option on, which is default as it does a good job of making the overall video more appealing. x265 however has a primitive implementation of SAO which tends to be too aggressive at high quality or fidelity ranges, leading to blurring around CRF 19-16. While it does "limit" how much the encoder uses SAO, `--limit-sao` is more of an early termination for the encoder deciding where to use it rather than limiting its strength. However, it generally does a good job of preserving more detail than normal, even if it makes some artifacts more noticeable. For below CRF 16, depending on your content, it might be preferable to outright disable sao with `--no-sao`, as usually at such high quality it's not usually needed. ### `--deblock` Word of caution, the documentation around the deblock setting is esoteric. Deblock is another loop filter, this one intended to reduce blocking in videos, but may have a blurring effect at high strengths. For most encodes, it is fine to leave this at the default value. At lower CRF values, it may be desirable to lower this to `--deblock -1:-1` for anime or `--deblock -2:-2` for live action, in order to preserve more grain and detail. ## Psycho-visual options `--psy-rd`, `--psy-rdoq`, `--aq-mode` and `--aq-strength` You can read more about the importance of perceptual optimization in video encoders on the [psychovisual](/start-here/psychovisual/) page. ### `--psy-rd` and `--psy-rdoq` To make a long story short, `--psy-rd` and `--psy-rdoq` together are psychovisual optimization tools used to control the encoder's willingness to retain finer detail and noise in the final encode. However, the way the two settings achieve this are very different. `--psy-rd` retains detail by affecting how the encoder weights sections of the frame based off the amount of "energy" or high frequency information it contains and boosts them accordingly. `--psy-rdoq` retains detail by affecting how the encoder quantizes coefficients after transformations. It has no reference to the source in its calculation and only prefers retaining overall visual energy and nothing specific. Both settings are highly source dependent and ideally would be tweaked per scene in a video. Unless you know what you're doing and are willing to test thoroughly that the settings you are using are beneficial, it's almost always recommended to never touch either setting, as the defaults for both are good general purpose settings. However, as a general rule of thumb, `--psy-rd` is better at retaining specific detail and overall sharpness while `--psy-rdoq` is better at retaining overall noise. ### `--aq-mode` and `--aq-strength` Adaptive quantization (`--aq-mode`), shortened to AQ, is a mechanism to redistribute bitrate within a frame to improve perceptual quality consistency. In x265, we have 4 modes. | mode | feature | |------|----------------------------------------------| | 1 | AQ enabled | | 2 | AQ with auto variance (default) | | 3 | AQ 2 with a bias for dark scenes | | 4 | AQ 2 with edge information | Generally speaking, we always want AQ 2 or AQ with auto variance as this will bias both parts of the frame that are smooth and textured. Normally, these parts of the frame are bitrate starved and have the most noticeable artifacting. Some people will use AQ 3 to, as the name implies, preserve detail in dark scenes and parts of the frame. However, in metric analysis and visual comparisons, AQ 3 can sometimes bloat bitrates for minor to no gains. While this is not an entirely useless AQ mode, thorough testing should be done before using it. The relative strength of an AQ mode can also be controlled with `--aq-strength`. While the default is `1.0`, many people do lower it to `0.7` or `0.8` for flat anime or compressed IRL content. In general, like other psychovisual optimization tools in x265, these settings are highly source dependant and are best left at their defaults, unless you know what you are doing and have data to back it up. ## CU-Tree `--cutree` and `--no-cutree` CU-Tree similar to MB-Tree in x264 is a method for the encoder to keep track of what parts of the frame are used or referenced by future frames. In a sense, it's a temporal motion quantizer. It has been very common for a long time, all the way back when only x264 existed, for people to disable MB-Tree due to the idea that it removed too much detail or blurred the video. However, a lot of that blur and detail loss would occur in parts of the frame that viewers would not be able to see clearly due to motion, or in areas that didn't significantly contribute to the overall image quality during playback. Both MB-Tree and CU-tree have also gotten a lot better since their original implementation making it almost always worse to disable them rather than to keep them enabled. Anyone telling you to disable either must provide evidence of it being actually better. --- # x266 Canonical URL: https://encode.wiki/tools/software-encoders/x266/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 8e653ad4c2754cbf3862f8c0b406f61037ea74a07ce743c86d62182b2aeb4113 x266 is an upcoming software encoder for the [H.266 / VVC](/codecs/video/vvc/) codec. Very little is currently known about the encoder other than the fact it is still being developed by [MulticoreWare](https://multicorewareinc.com) and it's dedicated FAQ page after the [31st January 2023 Webinar FAQ](https://multicorewareinc.com/faq-webinar/x266-faq.html). According to their FAQ, H2 2023 is their "very rough and approximate" ETA for a v1.0 public release, but so far there have been no updates as of 14th October, 2024. --- # autocompressor Canonical URL: https://encode.wiki/tools/utilities/autocompressor/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: fe5392ceb4f26e6c17872de07adca0e9dc9188c833c47f515b29c4e525553d51 [Autocompressor](https://autocompressor.net/) is a free online media compression tool by Auto-Rez Media Technologies that compresses videos, images, audio, and GIFs to a target file size. Depending on the file type you upload, it will be handled differently such that the result serves a similar function to what you uploaded. For example, videos are transcoded to videos, audio files to audio files, images to images, and animations to animations. From their site: > Reduce the file size of your video, image, audio, or GIF quickly and easily with our free online compression tool. Whether you need to shrink your file to 25MB or any other specific size, our tool offers a simple and intuitive solution. We'll also create perfectly compressed emotes and stickers for Discord. Using the link mode, you can instantly compress YouTube videos and other sites to 25MB. Our advanced AUTO-REZ™ technology achieves higher quality at a given filesize than any other website. Autocompressor will automatically choose the output format based on the input format and the compatibility category you select. Autocompressor also plans to offer a premium service in the future where the cost is determined by the CPU minutes used during the operation as well as the compression effort tier. For instance, encoding a typical 4-minute YouTube video to 8MB with MP4-Normal will cost about 5 CPU minutes. Users can create an account to get 1800 CPU minutes free. The site ensures high security for user data by using BCrypt hashes to store passwords. You can read more on their [About page](https://autocompressor.net/about). --- # Av1an Canonical URL: https://encode.wiki/tools/utilities/av1an/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: f459f20699a58792b591fc75cd70ceafc37e09323f2ba74ff2f9a2e236ff9c41 # Av1an Av1an with 96 workers

Av1an is a video encoding framework. It can increase your encoding speed and improve CPU utilization by running multiple encoder processes in parallel. Target quality, VMAF plotting, and more, available to take advantage for video encoding. The only downside to this, is that RAM usage increases significantly with each "worker" you add, so if you have a low amount of RAM (< 8GB), this might not be that useful to you. > **Caution — Av1an does not ship with any dependencies except for it's Docker image, so you will have to provide them on your own..** > > ## Prerequisites You will need Python, [FFmpeg](/tools/utilities/ffmpeg/) and [Vapoursynth](/tools/filtering/vapoursynth/) to be installed regardless of your current operating system. ## Installation ### Windows #### Script Windows users can use this [script](https://github.com/Hishiro64/av1an-win-script) which installs everything needed in a single portable folder. Do note that you will need to update the dependencies manually once installed. #### Pre-built Binary Av1an is available as a pre-built binary under the "[latest](https://github.com/master-of-zen/Av1an/releases/tag/latest)" tag on GitHub releases, you can simply download that and place it wherever you want. #### Compile from Source To compile from source, it is easier to use mingw-w64 which comes with [MSYS2](https://msys2.org). Once installed, open MinGW64 and run the following: ```bash pacman -Syuu && pacman -S cmake git nasm python3 mingw-w64-x86_64-gcc mingw-w64-x86_64-cmake base-devel mingw-w64-x86_64-ffmpeg mingw-w64-x86_64-rust mingw-w64-x86_64-lld mingw-w64-x86_64-clang mingw-w64-x86_64-make ``` Then, before you do anything further. Download Vapoursynth **portable** (`VapourSynth64-Portable-RXX.7z`) from its GitHub [release page](https://github.com/vapoursynth/vapoursynth/releases). Make sure the version you chose is compatible with the current MinGW64 Python version. For example, R65 supports 3.8 and 3.11, the version from Pacman (currently) is 3.11, so you should choose that. After downloading, open the 7z file and head to `\sdk\lib64`, copy all the libraries from there to MinGW64's `lib` directory, this usually resides in `C:\msys64\mingw64\lib`. Now finally, resume your session and run the following: ```bash git clone https://github.com/master-of-zen/Av1an cd Av1an RUSTFLAGS="-C target-cpu=native" cargo build --release ``` The binary will be available in `C:\msys64\home\USER\Av1an\target\release`. The `USER` part is your Windows username. ### macOS #### Using MacPorts Av1an is available in the MacPorts repository, to install just simply run: ```bash port install av1an ``` #### Compile from Source You need to have Git, Nasm, and Rust to be installed first. You can achieve this by using [Homebrew](https://brew.sh/) or similar. ```bash git clone https://github.com/master-of-zen/Av1an cd Av1an RUSTFLAGS="-C target-cpu=native" cargo build --release ``` Binary is located in `Av1an/target/release`, you can optionally place it somewhere like `/usr/local/bin`. ### Linux #### Package Managers Av1an is available in the package manager of some distributions such as Arch Linux. If not, you can skip to the next part every Linux user is familiar with. #### Compile from Source Just like macOS, you need to have Git, Nasm, and Rust to be installed. Can be achieved by either using your package manager or Rustup for Rust. ```bash git clone https://github.com/master-of-zen/Av1an.git cd Av1an RUSTFLAGS="-C target-cpu=native" cargo build --release ``` Binary is located in `Av1an/target/release`, you can optionally place it somewhere like `/usr/local/bin` or `/usr/bin`. ### Docker The following examples assume the file you want to encode is in your current working directory. #### Windows ```bash docker run --privileged -v "$(pwd):/videos" --user $(id -u):$(id -g) -it --rm masterofzen/av1an:latest -i S01E01.mkv {options} ``` #### Linux ```bash docker run --privileged -v "${PWD}:/videos" -it --rm masterofzen/av1an:latest -i S01E01.mkv {options} ``` #### Manual Build ```bash docker build -t "av1an" . ``` Run in the root directory of the repository. The dependencies will automatically be installed into the image, no manual installations necessary. > **Note — To specify a different directory to use you would replace $(pwd) with the directory.** > > ```bash > docker run --privileged -v "/c/Users/masterofzen/Videos":/videos --user $(id -u):$(id -g) -it --rm masterofzen/av1an:latest -i S01E01.mkv {options} > ``` > > The ``--user`` flag is required on Linux to avoid permission issues with the docker container not being able to write to the location, if you get permission issues ensure your user has access to the folder that you are using to encode. > **Caution — The Docker image ships with the default dependencies..** > > ## Installing Dependencies You will need a chunk method installed as the built-in ones are very slow. ## Troubleshooting ### "Error: The file 'XXXXX.ivf' could not be opened for reading: open file error." with mkvmerge on Linux Since mkvmerge opens all the encoded chunks at once, this obviously hits the default open file descriptor limit of 1024 on longer videos. You can temporarily increase it with `ulimit -n 20000` to allow opening 20,000 files at once in that session. ### Gray screen flashing for a single frame in output This is a unique problem with Av1an. Since Av1an does random seeking for chunks, traditional keyframe-based decoding methods such as L-SMASH and FFMS2 may fail and result in a gray frame. There are currently only two known methods to solve this: 1. Create a lossless intermediary with x264 `-qp 0`, this always fixes all related seeking issues but will result in a huge file due to it's lossless nature. 2. Using a linear-decoding chunk method such as `bestsource`, the downside to this is it's mega slow. Tests shown encode speeds drop by a whopping 10-15x slower. 3. Using `dgdecnv` chunk method. `dgdecnv` and `DGIndexNV` is a proprietary CUVID-accelerated Vapoursynth "Source" decoder created by [Donald Alan Graft](https://www.rationalqm.us/) where `DGIndexNV` is the standalone program for seeking and `dgdecnv` is the Vapoursynth plugin itself. It is especially made with random seeking in mind which makes it perfect for Av1an. Only downside is you need an NVIDIA GPU and `DGIndexNV` only supports x86_64 platforms. --- # av1an-command-gen Canonical URL: https://encode.wiki/tools/utilities/av1an-command-gen/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 44c7a5296c1d134f9022ac6d1f978b1d88fa7ef956be80d590b0d936f7945c4a # Av1an Command Generator Av1an Command Generator is a rudimentary tool for easily generating Av1an commands for AV1 encoding. It is written in the Zig programming language. It is very similar to [rAV1ator CLI](/tools/utilities/rav1ator-cli/) in the sense that it can produce Av1an commands based on user input. [GitHub Link](https://github.com/gianni-rosato/av1an-command-gen/) ## Description Av1an Command Generator is designed for beginners, although experienced encoders can use it to speed up their Av1an scripting. The program generates an AV1 video encoding command for use with [Av1an](/tools/utilities/av1an/), a chunked video encoding tool that can be used with [aomenc](/tools/software-encoders/aomenc/), [SVT-AV1](/tools/software-encoders/svt-av1/), and [rav1e](/tools/software-encoders/rav1e/). This tool takes in the video resolution, frame rate, desired encoder, speed preset, and target bitrate range as command line arguments. Based on these parameters, it calculates settings like tile columns/rows, lag-in-frames, CRF, and encoder speed preset. Then, it injects these into a generated encoding command string. The output is a full `av1an` command that can be run to encode a video based on the specified settings. ## Usage ```bash av1an-command-gen [width] [height] [fps] [encoder] [speed] [bitrate_target] ``` - `width` - Input video width in pixels - `height` - Input video height in pixels - `fps` - Input video frame rate - `encoder` - `aom`, `svt`, or `rav1e` - `speed` - `slower`, `slow`, `med`, `fast`, `faster` - `bitrate_target` - `lowest`, `low`, `med`, `high` ## Examples Generate a command for encoding a 1280x720 video at 24 fps using rav1e at 'med' speed and 'low' bitrate target: ```bash av1an-command-gen 1280 720 24 rav1e med low ``` Generate a command for encoding a 1920x1080 video at 30 fps using svt-av1 at 'fast' speed and 'high' bitrate target: ```bash av1an-command-gen 1920 1080 30 svt fast high ``` ## Installation This program requires the [Zig](https://ziglang.org/) v0.11.0 programming language. To build: ```bash zig build ``` This will produce a standalone binary `av1an-command-gen` in `zig-out/bin/`. ## License This project is licensed under the BSD 3-Clause License. --- # Aviator Canonical URL: https://encode.wiki/tools/utilities/aviator/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: ff7d9aa32c1b90fa3eb02fd730538e2e086fdbf6eb9ed3e3adf29f597f77ebba # Aviator Aviator is a GUI application designed for encoding [AV1](/codecs/video/av1/) video & Opus audio with [SVT-AV1](/tools/software-encoders/svt-av1/), [libopus](/codecs/audio/opus/), & [ffmpeg](/tools/utilities/ffmpeg/) on Linux systems in a user-friendly, intuitive manner. Aviator's primary focus is ease-of-use, while still striving to offer optimal quality per bit through a smart default encoding configuration implemented via the SVT-AV1-PSY encoder. Aviator Splash ## Installation #### Linux Aviator is available on Linux through Flathub as a Flatpak. You can learn how to set up Flatpak on your distro of choice [here](https://flatpak.org/setup/). Please do not use Aviator through the AUR. Download on Flathub If Flatpak is already set up and you'd prefer to install Aviator from the terminal, you can run the following commmand: ```bash flatpak install flathub net.natesales.Aviator ``` Flatpak was chosen for Aviator because it allows Aviator to ship its own sandboxed dependencies, ensuring every Linux user has a cohesive experience. One major benefit is the latest stable SVT-AV1-PSY implementation is always included, and Flatpak Aviator won't use your system's SVT-AV1 implementation which may be incompatible with Aviator's custom parameters. If you would like to build Aviator from source, you may do so via the following commands. Ensure you have the right dependencies; if not, `make` will warn you and you can install them as you see what you're missing. ```bash git clone https://github.com/gianni-rosato/aviator cd aviator make ``` #### Windows > **Caution — Partial Support.** > > Windows support is provided on a "best-effort" basis via **WSL2**. Compatibility hiccups will almost certainly be encountered should you decide to run Aviator on Windows. Aviator is supported on a "best-effort" basis on Windows via WSL2. In order to set up a WSL2 Arch Linux environment, see the [rAV1ator CLI](/tools/utilities/rav1ator-cli/#windows) WSL2 tutorial for Windows 11. Once WSL2 is set up, install Flatpak in your Linux environment by follwing the steps below. **Always be careful when running commands with `sudo` copied from the Internet, including the commands below.** ```bash sudo pacman -Syu # Update your system sudo pacman -S flatpak # Install flatpak sudo systemctl enable --now dbus # Start dbus flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo # Add flathub repo if it hasn't already been added ``` Finally, you can verify that Flatpak has actually been installed by running `flatpak --version` and checking that the version string is valid. Now, Aviator requires a couple of special Flatpak dependencies most commonly found on GNOME Linux distros. ```bash sudo flatpak install org.gnome.Sdk # Select option 15 sudo flatpak install org.gnome.Platform # Select option 14 ``` Finally, you can install Aviator. ```bash sudo flatpak install flathub net.natesales.Aviator ``` You can run Aviator by executing the following command: ```bash flatpak run net.natesales.Aviator ``` Now, you should be all set! Happy encoding! ## Aviator's Defaults Hovering over most user configurable options in Aviator will produce a helpful tooltip that you can look at to make things more clear. ### Perceptual Optimization Aviator doesn't use mainline SVT-AV1, but rather uses [a fork](https://github.com/gianni-rosato/svt-av1-psy) dubbed SVT-AV1-PSY maintained for perceptual quality. It includes several unique changes, including a custom [SSIM](/reference/metrics/ssim/)-based [RDO](/start-here/psychovisual/) tune that isn't included in mainline SVT-AV1. You can read more about SVT-AV1-PSY on the [SVT-AV1](/tools/software-encoders/svt-av1/#community-forks) entry. Aviator's default FFmpeg command uses the following encoding parameters, some of which are redundant with defaults: ```bash -c:v libsvtav1 -crf X -preset X -pix_fmt yuv420p10le -svtav1-params film-grain=X:irefresh-type=2:input-depth=10:tune=3:sharpness=1:qm-min=0:keyint=300:aq-mode=2:enable-qm=1:film-grain-denoise=X ``` ### Video Aviator Video Settings By default, output resolution will match your source's resolution. Manually changing one resolution value will automatically calculate the other based on the video's aspect ratio. Aviator's SVT-AV1 speed preset is set to 6 by default, with a CRF (Constant Rate Factor) level of 32. You can set CRF from 0 to 63 using the slider, with larger numerical values indicating smaller filesize at the expense of visual quality. You can look at the detailed specifications behind each speed preset [here](https://gitlab.com/AOMediaCodec/SVT-AV1/-/blob/master/Docs/CommonQuestions.mdx#what-presets-do). Speed 7 offers a good balance between speed & compression efficiency at any CRF level. Setting values that don't correspond with the source video's aspect ratio means the output will either stretch or crop based on if the "crop" option is checked. The Grain Synth slider allows you to add artificial grain to your video to mimic its natural grain. This option applies the artificial grain at decode time as a filter, which makes it easier to encode grainy videos at high fidelity. The Denoise switch removes noise from the video before applying artificial grain. ### Audio Aviator Audio Settings The default bitrate for Opus audio is 80kb/s. The audio source can be copied to the output media via that "Copy Audio" switch, & audio with >2 channels can be downmixed to stereo via the appropriately labelled switch. Volume adjustment & normalization are also offered in this section as well; the Volume slider allows you to increase or decrease the output's volume & is measured in decibels, & negative values decrease the volume. The Normalize toggle allows you to normalize your audio's perceived loudness. The "Copy Audio" switch disables WebM output due to potential compatibility hiccups & overrides every option on the Audio page to keep the source audio untouched. This option, when enabled, ensures the source audio isn't reencoded. The Volume slider allows you to increase or decrease the output's volume. It is measured in decibels, and negative values decrease the volume. The Normalize switch allows you to normalize the audio's perceived loudness. ### Output Aviator Output UI The container your video is stored in is associated with the file extension. Aviator offers two options for video output: the [Matroska](/start-here/terminology/#mkv--mka--mks--mk3d) video container & the [WebM](/start-here/terminology/#webm) container. The open-source Matroska container (.MKV) is used by default in Aviator & is a universal multimedia container with broad video & audio support. WebM is designed for web compatibility. Aviator won't copy subtitles to WebM outputs because WebM is only officially compatible with [WebVTT](/codecs/subtitles/webvtt/) subtitles. Both containers work out of the box with Aviator's AV1 video & Opus audio, but WebM output will be disabled if the Copy Audio switch is enabled because then we lose this format compliance assurance. ## Credits Aviator is actively developed by [Gianni Rosato](https://github.com/gianni-rosato/). --- # dav1d Canonical URL: https://encode.wiki/tools/utilities/dav1d/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 890b7be571cd5749cbce055f3afb6c3201ea06c0f9d56d04d869eed5a8e09323 The dav1d (which stands for "dav1d is an AV1 decoder") AV1 decoder is a high-performance, lightweight, and open-source software decoder for the AV1 video codec. It is primarily developed by [Two Orioles, LLC](https://www.twoorioles.com) on behalf of Videolan. It is known for being extremely lightweight and very fast. ## Features - **Design**: Written in C99 with assembly optimizations (NASM/GAS syntax), dav1d is designed for speed and low resource consumption - **Platform Support**: Compatible with x86, x64, ARMv7, ARMv8, & more. It runs on Windows, Linux, macOS, Android, and iOS. - **Adoption**: dav1d is used across all Android devices as well as major browsers like Chrome, Safari, Edge, and Firefox. It is believed that the AVIF decoding on Apple devices uses a version of dav1d that may be an internal fork. ## Binary Size dav1d's binary size is approximately one-third that of libaom's decoder, weighing in at around 0.9 MB. It also has a much smaller codebase that libaom, about one-tenth the lines of code; making it lightweight and easier to integrate into applications. ## Performance Comparison ### libgav1 - dav1d significantly outperforms Google's libgav1 in decoding speed. For example: - On a Google Pixel 3 XL (without hardware acceleration), dav1d achieves 50 fps for 4K60 video decoding, whereas libgav1 achieves less than 10 fps. - dav1d is more efficient in CPU resource usage and power consumption, making it ideal for low-end devices. ### libaom - dav1d offers better decoding performance than libaom (the AV1 reference implementation), particularly after optimizations with assembly code. While libaom is slower due to its focus on demonstrating codec functionality rather than performance, dav1d is optimized for real-world use cases. - Memory usage is also significantly lower with dav1d—about one-fourth that of libaom. ### ffhevc dav1d's decoding efficiency rivals or exceeds that of ffhevc. ## Recent Developments As of April 2024: - Android officially adopted dav1d as its default AV1 software decoder via a Play System update. This change significantly improved video playback on devices without hardware AV1 decoders - dav1d supports smooth playback of 720p30 videos on most devices and even higher resolutions on capable hardware ## Conclusion dav1d stands out as the most efficient and widely adopted software-based AV1 decoder. Its small binary size, high performance across platforms, and low resource consumption make it a preferred choice over alternatives like libgav1 and libaom. Additionally, its adoption by major platforms such as Android underscores its effectiveness in real-world applications. --- # Discord Canonical URL: https://encode.wiki/tools/utilities/discord/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: c07e25135ed4498f6f605bdfd152f2a7a632757f5a4b9a0b2d9b74f716b8aa5a > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. > **Caution — Pending Review.** > > The content in this entry may not be entirely accurate, & is pending further review to assess the quality of the information. This entry is a bit different from others, and may be moved to a blog post in the future. Below are a number of tables that enumerate the codecs Discord supports for local playback (*not* livestreaming). These tables are based on community testing across a number of platforms, and are not exhaustive. ## Key The key has a number of emojis corresponding to full support, partial support, support with the [Discord embed workaround](https://codecs.wiki/blog/embedding-the-un-embeddable), and no support. - ✅ Full support - ⚠️ Partial support - 🛠️ Support with the Discord embed workaround - ❔ Untested or unknown - ❌ No support If the browser isn't compatible with a particular platform, it will be marked as N/A. ## Video Codecs & Containers **[H.264](/codecs/video/avc/)** 4:2:0 8-bit lossy | .mp4 | Browser | macOS | Windows | iOS | Android | Linux | |---------|:-----:|:-------:|:---:|:-------:|:-----:| | Chrome | ✅ | ✅ | ❔ | ❔ | ✅ | | Firefox | ✅ | ✅ | ❔ | ❔ | ✅ | | Safari | ✅ | N/A | ✅ | N/A | N/A | | App | ✅ | ✅ | ✅ | ✅ | ✅ | **[H.265](/codecs/video/hevc/)** 4:2:0 8-bit lossy | .mp4 | Browser | macOS | Windows | iOS | Android | Linux | |---------|:-----:|:-------:|:---:|:-------:|:-----:| | Chrome | ✅ | ⚠️** | ❔ | ❔ | ❌ | | Firefox | ❌ | ⚠️*** | ❔ | ❔ | ❌ | | Safari | ⚠️* | N/A | ⚠️* | N/A | N/A | | App | ✅ | ⚠️** | ⚠️* | ✅ | ❌ | \* Only supports H.265 with the `hvcC` box in the MP4 container. These streams can be encoded by adding `-tag:v hvc1` to an FFmpeg encoding command. \** Requires you have hardware accelerated HEVC decoding support. \*** Firefox on Windows requires you have hardware accelerated HEVC decoding support and the `media.wmf.hevc.enabled` flag enabled in `about:config`. **[H.265](/codecs/video/hevc/)** 4:2:0 8-bit lossy | .mov | Browser | macOS | Windows | iOS | Android | Linux | |---------|:-----:|:-------:|:---:|:-------:|:-----:| | Chrome | ✅ | ⚠️** | ❔ | ❔ | ❌ | | Firefox | ❌ | ⚠️*** | ❔ | ❔ | ❌ | | Safari | ⚠️* | N/A | ⚠️* | N/A | N/A | | App | ✅ | ⚠️** | ⚠️* | ✅ | ❌ | \* Only supports H.265 with the `hvcC` box in the MOV container. These streams can be encoded by adding `-tag:v hvc1` to an FFmpeg encoding command. \** Requires you have hardware accelerated HEVC decoding support. \*** Firefox on Windows requires you have hardware accelerated HEVC decoding support and the `media.wmf.hevc.enabled` flag enabled in `about:config`. **[H.265](/codecs/video/hevc/)** 4:2:0 10-bit lossy | .mp4 | Browser | macOS | Windows | iOS | Android | Linux | |---------|:-----:|:-------:|:---:|:-------:|:-----:| | Chrome | ✅ | ❌ | ❔ | ❔ | ❌ | | Firefox | ❌ | ❌ | ❔ | ❔ | ❌ | | Safari | ⚠️* | N/A | ⚠️* | N/A | N/A | | App | ✅ | ❌ | ⚠️* | ✅ | ❌ | \* Only supports H.265 with the `hvcC` box in the MP4 container. These streams can be encoded by adding `-tag:v hvc1` to an FFmpeg encoding command. **[H.265](/codecs/video/hevc/)** 4:2:0 10-bit lossy | .mov | Browser | macOS | Windows | iOS | Android | Linux | |---------|:-----:|:-------:|:---:|:-------:|:-----:| | Chrome | ✅ | ❌ | ❔ | ❔ | ❌ | | Firefox | ❌ | ❌ | ❔ | ❔ | ❌ | | Safari | ⚠️* | N/A | ⚠️* | N/A | N/A | | App | ✅ | ❌ | ⚠️* | ✅ | ❌ | \* Only supports H.265 with the `hvcC` box in the MOV container. These streams can be encoded by adding `-tag:v hvc1` to an FFmpeg encoding command. --- # dovi_tool Canonical URL: https://encode.wiki/tools/utilities/dovi_tool/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: eadae74eb66277a1539b61f915a476648b3ce0d818d119740da88acb87667694 # dovi_tool **dovi_tool** is a command line tool written in Rust combining multiple utilities for working with Dolby Vision. ## Installation #### Linux & macOS You can download official pre-built binaries for Linux & macOS from [the GitHub Releases](https://github.com/quietvoid/dovi_tool/releases), or you may compile the tool yourself via the instructions below. Ensure you have Rust installed. ```bash git clone https://github.com/quietvoid/dovi_tool.git cd dovi_tool RUSTFLAGS="-C target-cpu=native" cargo build --release ``` #### Windows You can download official pre-built binaries for Windows from [the GitHub Releases](https://github.com/quietvoid/dovi_tool/releases), or you may compile the tool yourself via the instructions below. Ensure you have Rust installed. ```pwsh git clone https://github.com/quietvoid/dovi_tool.git cd dovi_tool set RUSTFLAGS=-C target-cpu=native cargo build --release ``` ## Usage ```bash title="General usage" dovi_tool [OPTIONS] ``` ```bash title="Get more options for a subcommand" dovi_tool --help ``` --- # eac3to Canonical URL: https://encode.wiki/tools/utilities/eac3to/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 7e0a35a2dcf8c1685ae20b26454b6605683cb4b551c3372ddecf4bcc0da96395 eac3to is a command line tool written by madshi to mostly work with audios (conversion) and raw, unencrypted Blu-rays (BDMV). > **Note — Compatibility.** > > This is a Windows only software, although usage with Wine is possible. > **Caution — Avoid using when possible.** > > There is absolutely no reason whatsoever to use this software unless required by paleolithic [private trackers](https://wiki.installgentoo.com/wiki/Private_trackers), due to a bajillion dependencies needed that aren't bundled with the download itself. Use something like FFmpeg instead. ## Installation - Download from [VideoHelp](https://www.videohelp.com/software/eac3to), drag and drop into your designated folder, add to PATH when necessary. ## Usage ### Audio conversion Examples: ```bash title="Convert PCM audio to FLAC" eac3to source.pcm destination.flac ``` ```bash title="Convert TrueHD to FLAC while also extracting the AC-3 compatibility layer" eac3to source.thd destination.flac destination.ac3 ``` ### Get BDMV info ```bash eac3to Movie.2024.Bluray/BDMV ``` ### Demux ```bash eac3to Movie.2024.Bluray/BDMV -demux ``` ### Delay audio ```bash eac3to input.eac3 output.eac3 1000ms ``` --- # FFMetrics Canonical URL: https://encode.wiki/tools/utilities/ffmetrics/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 2cb307fc6240d3a30ecf28e718b1648929b90fc6106b54d42d05f74f4a3ffb8b FFmetrics Window

FFMetrics is a proprietary Windows-only graphical user interface (GUI) for FFmpeg that allows you to calculate and visualize video quality metric results with graphs. PSNR, SSIM, and VMAF are the only options. ## Installation > **Caution — You need to have FFmpeg in your system PATH in order to use this. and `--enable-libvmaf` is required for VMAF calculation support..** > > Download the program from their GitHub homepage [releases](https://github.com/fifonik/FFMetrics/releases), pick whichever version you'd like to download and extract to your preferred folder. Run the exe file to launch. ## Usage Use the "Browse" button to insert your reference (source) video and "Add files" to add your distorted (encoded) video(s). Tick the checkboxes to choose the metric calculations you'd like to run. Custom VMAF models can be chosen via "VMAF options" and once you're done just click "Start", the program will automatically generate graphs live for each score. Depending on your CPU. This process might take a while. --- # ffmpeg Canonical URL: https://encode.wiki/tools/utilities/ffmpeg/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 3921caa7c74844f8a9cb5080f86e96e726d1236856cdabf12270fb405b407940 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! [FFmpeg](https://ffmpeg.org/) is a multimedia framework that has utilities for transcoding, transmuxing, and filtering audio and video. It provides the `ffmpeg`, `ffprobe`, and `ffplay` command-line utilities. It also features the libav\* libraries, which allow you to use the functionality of FFmpeg without the programs. # Installation There are a number of ways to install FFmpeg depending on the operating system you're using. ### Linux & macOS **Package Manager** The easiest way to obtain FFmpeg is through your package manager. On most package managers, the package is simply named `ffmpeg`, however `ffprobe` and `ffplay` may have their own packages. Note that the packages may be outdated. **Compiling from source** A more complete guide is available at the [FFmpeg Compilation Guide](https://trac.ffmpeg.org/wiki/CompilationGuide). Simplifying things a bit, what you need to do is: - grab [the sources](https://ffmpeg.org/download.html) or clone from FFmpeg's git: `git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg` - Enter the directory & run `./configure --help` to see a list of features and libraries you can choose to build with. - Install all libraries you want to build FFmpeg with. - Run `./configure` with `--enable-` flags as desired. - Run `make`, or `make -j $(nproc)` on Linux to properly make use of multiple cores. on macOS, this would be `make -j $(sysctl -n hw.ncpu)`. - Run `make install`. May require root. ### Windows There are no official binaries for FFmpeg on Windows, but you can download third-party binaries: - by [gyan.dev](https://www.gyan.dev/ffmpeg/builds/) - by [BtbN](https://github.com/BtbN/FFmpeg-Builds/releases) # Using FFmpeg `ffmpeg` is the primary command-line tool of FFmpeg. It takes 0 or more files as inputs & outputs. `ffmpeg`'s command-line arguments are positional, meaning it matters where you put each option. Each input and output has its own arguments. For example, `ffmpeg -r 24 -i file1 file2` applies the `-r 24` option to the input `file1`, interpreting the video as having that frame rate, while `ffmpeg -i file1 -r 24 file2` applies the `-r 24` option to `file2`. To get a list of options, refer to the more verbose [FFmpeg documentation](//ffmpeg.org/ffmpeg-all.html). #### Transcode a video ```bash ffmpeg -i [input] -c:v [video_codec] -b:v [video_bitrate] -c:a [audio_codec] -b:a [audio_bitrate] output ``` | Option | Meaning | |----------------------|-------------------------------------------------------------| | `-c:v video_encoder` | **c**odec for the automatically selected **v**ideo stream | | `-b:v video_bitrate` | **b**itrate for the automatically selected **v**ideo stream | | `-c:a audio_encoder` | **c**odec for the automatically selected **a**udio stream | | `-b:a audio_bitrate` | **b**itrate for the automatically selected **a**udio stream | #### Transmux a video ```bash ffmpeg -i [input] -c copy [output] ``` | Option | Meaning | |-----------|-------------------------------| | `-c copy` | set the **c**odec to **copy** | #### Filter a video ```bash ffmpeg -i [input] -c:v [video_encoder] -c:a [audio_codec] (...) -vf [filter_name] output ``` | Option | Meaning | |-------------------|-------------------------------------------------| | `-vf filter_name` | set the **v**ideo **f**ilter to **filter_name** | #### Container selection FFmpeg will usually select the appropriate container based on the file extension of the output. If it doesn't detect the correct container, you can specify it with `-f`. | Option | Meaning | |------------------|---------------------------------------| | `-f format_name` | set the **f**ormat to **format_name** | *References:* *[^multimediawiki-howtos]: [HOWTO Search Results - MultimediaWiki](//wiki.multimedia.cx/index.php?search=HOWTO&title=Special%3ASearch&go=Go)* *Special thanks to [bluefalcon's encoding guide](https://encoding.bluefalcon.cc/) for this material, licensed under CC BY-SA 4.0. Our adaptation features formatting changes & content changes, specifically regarding the titles of some headings.* --- # hdr10plus_tool Canonical URL: https://encode.wiki/tools/utilities/hdr10plus_tool/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 97531a767ea9aba4049a3bc40cd4548279e88007f161f97576c15c472c731279 # hdr10plus_tool **hdr10plus_tool** is a command line tool written in Rust for working with HDR10+ in HEVC files. It was previously named *hdr10plus_parser*. ## Installation #### Linux & macOS You can download official pre-built binaries for Linux & macOS from [the GitHub Releases](https://github.com/quietvoid/hdr10plus_tool/releases), or you may compile the tool yourself via the instructions below. Ensure you have Rust installed. ```bash git clone https://github.com/quietvoid/hdr10plus_tool.git cd hdr10plus_tool RUSTFLAGS="-C target-cpu=native" cargo build --release ``` #### Windows You can download official pre-built binaries for Windows from [the GitHub Releases](https://github.com/quietvoid/hdr10plus_tool/releases), or you may compile the tool yourself via the instructions below. Ensure you have Rust installed. ```pwsh git clone https://github.com/quietvoid/hdr10plus_tool.git cd hdr10plus_tool set RUSTFLAGS=-C target-cpu=native cargo build --release ``` ## Usage ```bash title="General usage" hdr10plus_tool [OPTIONS] ``` ```bash title="Get more options for a subcommand" hdr10plus_tool --help ``` ### Extracting ```bash title="Extract using FFmpeg pipe (Recommended)" ffmpeg -hide_banner -strict -2 -i input.mkv -map 0:v:0 -c copy -vbsf hevc_mp4toannexb -f hevc - | hdr10plus_tool extract -o metadata.json - ``` ```bash title="Extract from raw bitstream" hdr10plus_tool extract video.hevc -o metadata.json ``` ### Injecting ```bash title="Inject to raw bitstream" hdr10plus_tool inject -i video.hevc -j metadata.json -o injected_output.hevc ``` ### Removing HDR10+ Metadata ```bash title="Remove using FFmpeg pipe (Recommended)" ffmpeg -hide_banner -strict -2 -i input.mkv -map 0:v:0 -c copy -vbsf hevc_mp4toannexb -f hevc - | hdr10plus_tool remove - ``` ```bash title="Remove from raw bitstream" hdr10plus_tool remove video.hevc -o hdr10plus_removed_output.hevc ``` --- # MKVToolNix Canonical URL: https://encode.wiki/tools/utilities/mkvtoolnix/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: e5dfbf8418b0e406d65a7469c9f85e94a01d2c480321ad88a2d8f08c7db5350a MKVToolNix Workload

MKVToolNix is free and open source GUI frontend to a set of tools (mkvmerge, mkvinfo, mkvpropedit, mkvextract) to create, alter and inspect Matroska files under Linux, other \*NIXes and Windows. MKVToolNix is built with Qt and written in C++. ## Installation Fortunately, Moritz Bunkus (Creator) is sane enough to provide downloads for pretty much all popular desktop operating systems. - Windows & Mac: [MKVToolNix site](https://mkvtoolnix.download/downloads.html). - Linux: [Install as a Flatpak](https://flathub.org/apps/org.bunkus.mkvtoolnix-gui) or check your package manager. ## Usage Drag and drop media files (or use "Add source files") to the application and modify, remux, demux, add streams, drop streams, anything and when done set your output destination and click "Start multiplexing" to start writing the output. ## Tips and tricks 1. You can import unencrypted BDMV (``index.bdmv``) or Blu-ray playlist files into MKVToolNix and it will prompt you to select which stream you would want to import. 2. You can set a WebM output in Output > Miscellaneous > "Create WebM compliant file" 3. You can split videos in multiple ways in Output > Splitting. Best of all? No freeze frames unlike in FFmpeg and no re-encoding required. 4. You can edit metadata without remuxing (Albeit rather limited) in the "Header Editor". --- # MP4Box Canonical URL: https://encode.wiki/tools/utilities/mp4box/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: 011ac3b12d79e9d153554b79129df76470376d1b6f1e6a51a470a9743440f420 > **Danger — Help Wanted.** > > This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor! MP4Box is a software for multi-purpose MP4 file manipulation. Part of the free and open-source multimedia framework called [GPAC](https://github.com/gpac/gpac) for working with the MP4/ISOBMFF standard. --- # NMKODER Canonical URL: https://encode.wiki/tools/utilities/nmkoder/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: 5dd3f6ab943bf0a1b8dc13e8c27facb6779d97261a9d8fcd9ae2448748e25f23 NMKODER Window

[NMKODER](https://github.com/n00mkrad/nmkoder) is a free and open source, **Windows-only** GUI software developed by [N00MKRAD](https://github.com/n00mkrad). Built around FFmpeg, FFprobe, and Av1an for video encoding, muxing, and analysis such as concatenation, metrics calculation, and bitrate plotting. Though most people use this for the [Av1an](/tools/utilities/av1an/) functionality. The software is written in C# and built with WinForms. > **Danger — Abandonware.** > > NMKODER have not been updated for years, it is not recommended to use this. But you can attempt to replace the provided binaries (aomenc.exe, etc) with new ones to technically "update" this. ## Installation You can download it from the GitHub [releases](https://github.com/n00mkrad/nmkoder/releases) page (the 7z file). Extract it to wherever after it's finished downloading, open the folder, and click the exe file. The entire software is portable, so you can share the folder with anyone and they'll have no problem with running it. ## Usage NMKODER is really straightforward to use and consists of multiple tabs for each functionality. There is a drop zone on the left for drag-and-drop video files. - "File List" is for imported files. - "Track List" is for A/V tracks. - "Quick Convert" tab is for conversion with FFmpeg. - "AV1AN" is what it exactly says. - "Utilities" is for helpful tools such as colorspace/HDR metadata transferring, metric calculation, etc. After configuring everything, click on the "Start" button to start encoding. --- # rAV1ator Canonical URL: https://encode.wiki/tools/utilities/rav1ator/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: e549adf11b0e6195fc2f45ebfc652007a7b09a5bf2f9f38592c77acea88106b9 > **Note — Under Maintenance.** > > The content in this entry is incomplete & is in the process of being completed. rAV1ator is a fork of [Aviator](/tools/utilities/aviator/) designed to use [Av1an](/tools/utilities/av1an/) & [rav1e](/tools/software-encoders/rav1e/) instead of [SVT-AV1](/tools/software-encoders/svt-av1/). It is also distributed as a Flatpak with bundled dependencies outside of Flathub; running the following command will allow you to install it: ```bash flatpak --user remote-add --no-gpg-verify project-volo https://giannirosato.com/repo && flatpak --user install project-volo net.natesales.rAV1ator ``` Initially, Aviator was supposed to switch to using the Av1an + rav1e backend that currently serves rAV1ator, but the decision was made to split the two projects due to their fundamentally different goals & Flathub's trouble with Rust dependencies. Aviator is designed to allow easy, painless AV1 encoding, while rAV1ator is designed to make accessing *specifically* Av1an & rav1e easier for interested codec enthusiasts. rAV1ator is maintained by Gianni Rosato & Trix. --- # rav1ator-cli Canonical URL: https://encode.wiki/tools/utilities/rav1ator-cli/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: ee93e43f301f443be745a32a2e5b03c420bec41d868c2b9571218433a7ce00db # rAV1ator CLI rAV1ator CLI, or just `rav1ator-cli`, is a TUI tool that provides an interactive command line interface for encoding videos with [Av1an](/tools/utilities/av1an/) using various different encoders including [rav1e](/tools/software-encoders/rav1e/), [aomenc](/tools/software-encoders/aomenc/) (specifically aom-av1-lavish, as mentioned in the aomenc page), [SVT-AV1](/tools/software-encoders/svt-av1/), [x265](/tools/software-encoders/x265/), and [x264](/tools/software-encoders/x264/). rAV1ator CLI was maintained by Gianni Rosato at https://github.com/gianni-rosato/rav1ator-cli, but is now maintained by Ultimaxx at https://github.com/ultimaxx/rav1ator-cli as of January 2025. The original repository has since been archived. ```bash ~ > rav1ator-cli -h rAV1ator: CLI Edition_ v0.2.4 Usage: rav1ator-cli [input] [output] [--offline] Dependencies (Arch): rust ffmpeg python mkvtoolnix-cli vapoursynth gum numactl vapoursynth-plugin-bestsource-git av1an ffms2 Options: (Currently, only one option is useful at a time) -h, --help Print this help section -l, --last-used Print last used encode settings from history -f, --full-history Print full history from ".rav1ator-cli-history" file -b, --binaries Just install binaries, then exit -x, --offline Don't check for updates. -a, --batch Batch encode. All video files in a directory specified after this flag are encoded. ``` rAV1ator CLI can: - Check if it is installed & up to date on its own without a package manager - Download AVX2-optimized encoder binaries compiled with -O3 -flto in most cases & allow the user to install them with detailed instructions - Remember encoding history and let you view your whole history or your most recent command - Allow you to encode an entire directory of video files with the same settings - Encode with x264, x265, aomenc, SVT-AV1, or rav1e, set a speed preset, CRF/quality value, FFmpeg parameters, and encoder parameters - Generate Av1an encoding commands with the user's chosen settings & run them to encode a provided input video to an MKV output. - Encode from scratch, or resume a previous rAV1ator CLI encode - Engage with rich interactivity features like spinners, prompts, & dropdowns - Automatically error check binaries with SHA256 hashes for security & convenience Overall, it aims to provide an easy way to encode videos on the command line with helpful visual feedback. The interactive prompts help users pick encoding settings without needing deep encoding knowledge. rAV1ator-cli: Demo 1 rAV1ator-cli: Demo 2 ## Installation rAV1ator CLI is natively supported on Linux, & is supported on Windows via WSL2. A tutorial for setting up WSL2 is provided below. macOS is not supported. ## Linux (Arch) These instructions are for Arch Linux specifically. Other distros should be very similar, and packages that are Arch-specific will be labelled. If you're on Ubuntu, you should see the relevant section of the [AV1 for Dummies](https://codecs.wiki/blog/av1-encoding-for-dummies) blog post on this site. If you're on another distro and you want to be able to follow these instructions specifically, see the [Linux (Other)](#linux-other) section. 0. Update your system before doing anything. On Arch: ```bash sudo pacman -Syu ``` 1. Install `yay` (Arch only) by running the following commands: ```bash sudo pacman -S --needed base-devel git git clone https://aur.archlinux.org/yay.git cd yay && makepkg -si ``` 2. Next, you'll want to install all of rav1ator-cli's dependencies. You can do that by running: ```bash yay -Syu openssl ffmpeg python mkvtoolnix-cli vapoursynth gum numactl vapoursynth-plugin-bestsource-git av1an ffms2 ``` 3. Install rav1ator-cli: ```bash curl -sOJ https://raw.githubusercontent.com/ultimaxx/rav1ator-cli/main/rav1ator-cli && chmod +x rav1ator-cli sudo cp rav1ator-cli /usr/local/bin ``` You're done! Run `rav1ator-cli -h` to get some help getting started. Happy encoding! ## Linux (Other) If you're on Ubuntu, you can see the relevant section of the [AV1 for Dummies](https://codecs.wiki/blog/av1-encoding-for-dummies) blog post on this site for more information about doing this *without* a distrobox. However, a distrobox is valuable because you can use Arch's fast-paced package management on other distros. Distroboxes are generally easier than Docker for beginners, and use Docker or Podman behind the scenes anyway. This tutorial will focus on using Distrobox with Podman. 0. Install Distrobox and Podman on your distro of choice. Please look up how to do this for your respective distro, and how to get everything set up properly. 1. If you already use Podman, are re-creating a Distrobox, or you already have other Distroboxes running from long enough ago where your Arch image is outdated, you might want to run `podman image rm docker.io/archlinux/archlinux:latest`. It won't hurt to run it anyway if you're not sure. **Do this every time you make a new Distrobox if you're experiencing issues with Distrobox creation**. 2. Now, run `distrobox-create --name rvcli-box --image archlinux:latest`. You do not need to name yours "rvcli-box", but that's what we're going to call the box in this tutorial. 3. `distrobox enter rvcli-box` to go inside. You are now using Arch Linux from within your existing distro! 4. First, run `sudo pacman -Syu` to update your system. 5. Next, run `sudo pacman -S --needed base-devel git && git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si`. 6. Finally, `yay -Syu openssl ffmpeg python mkvtoolnix-cli vapoursynth gum numactl l-smash vapoursynth-plugin-lsmashsource av1an ffms2` to install the necessary dependencies. 7. Download rav1ator-cli: `curl -sOJ https://raw.githubusercontent.com/ultimaxx/rav1ator-cli/main/rav1ator-cli && chmod +x rav1ator-cli` 8. Copy rav1ator-cli to your /usr/local/bin: `sudo cp rav1ator-cli /usr/local/bin` 9. If you'd like to exit the distrobox, just do `Ctrl` + `D`. You're done! Run `rav1ator-cli -h` to get some help getting started. Happy encoding! ## Windows > **Note — Windows.** > > The content in this entry was written by pat-e, or `pate` on Discord. This tutorial focuses on Windows 11. **Enable WSL2 on Windows 11 and Install "ArchWSL2"** by pat-e ### Basic installtion 1. Open the Terminal as Administrator: ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/01_wsl_rv-cli.avif) 2. In the Command-Prompt, enter the following to install WSL: ```pwsh wsl.exe --install --no-distribution ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/02_wsl_rv-cli.avif) If the "Host Process for Windows Services" asks for allowing changes, approve it (Press "Yes"): ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/03_wsl_rv-cli.avif) 3. Once the installation of WSL is finished, restart your Computer: ```pwsh shutdown -r -f -t 1 ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/04_wsl_rv-cli.avif) 4. From [This GitHub link](https://github.com/sileshn/ArchWSL2), download the latest release of ArchWSL2 in the Releases section: ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/06_wsl_rv-cli.avif) 5. On your SSD, create a folder where we will place the ArchWSL2 files. This folder **must be kept and never deleted** as this will contain the base files for this distribution. It is advised to use an SSD instead of an HDD. The storage must be local storage, not a network-share or a removable disk. In our example, we will create a folder located at C:\\Stuff\\ArchWSL2; ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/07_wsl_rv-cli.avif) ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/08_wsl_rv-cli.avif) 6. Extract the downloaded ZIP of "ArchWSL2" into your newly created folder: ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/09_wsl_rv-cli.avif) 7. Once extracted, start the extracted "Arch.exe": ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/10_wsl_rv-cli.avif) 8. On first start, the program will create a virtual disk (VHDx). Once finished, press "Enter" to continue (it will close the window) ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/11_wsl_rv-cli.avif) 9. Once the virtual hard disk is created, restart the "arch.exe" again. It will finish some steps and ask for creating a new user-account. **This account is completely separate from your Windows user.** Please remember the username and password you use. ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/12_wsl_rv-cli.avif) ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/13_wsl_rv-cli.avif) 10. The Window will close and reopen again. As first step, update all packages. As your account you created earlier is in the sudo-group, you have to enter your password again as confirmation. ```bash sudo pacman -Syu ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/14_wsl_rv-cli.avif) 11. Approve the installation of all the updates (Type "y"): ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/15_wsl_rv-cli.avif) 12. Done... You can now use ArchWSL2. ### After the Installation and Cleanup, How Do I Start Arch? For the start of Arch, there are 2 steps possible: 1. You can just start "wsl" form the command prompt, terminal or "Windows Search": ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/20_wsl_rv-cli.avif) ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/21_wsl_rv-cli.avif) 2. As another option, you can create a Shortcut to the "arch.exe" in the Installation-Folder: ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/22_wsl_rv-cli.avif) ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/23_wsl_rv-cli.avif) ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/24_wsl_rv-cli.avif) **Hint**: When you open the "arch.exe", you will be placed into the directory where also the "arch.exe" is located. To change to your home directory, just enter the command below: ```bash cd ~ ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/25_wsl_rv-cli.avif) ### Unlock WSL RAM Usage (Optional) **Hint**: WSL caps RAM usage at 50% of the total RAM available on your system. When you start the VM, you will see that you only will only have 50% of your total memory available: ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/26_wsl_rv-cli.avif) To allow more memory, you need to place a config-file in your profile-folder in Windows. See the instructions below: 1. Open "notepad.exe" and enter the following: ```txt [wsl2] memory=12GB ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/27_wsl_rv-cli.avif) 2. The amount of memory should never be more than current memory. Set it to total memory minus 4GB to leave enough left over for Windows. In my example, Windows has 16GB of RAM available, so I select the memory to be 12GB. 3. When saving the file, enter the following as filename: `%userprofile%` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/28_wsl_rv-cli.avif) 4. When pressing enter, the directory will switch to your user-profile folder: ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/29_wsl_rv-cli.avif) 5. Select the "Save as type" to "All files (\*.\*)" and save as the following filename: `.wslconfig` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/30_wsl_rv-cli.avif) Make sure the file is saved as `.wslconfig`. Then exit notepad. 6. Exit any current running WSL / Arch (exit): ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/31_wsl_rv-cli.avif) 7. Open the Terminal as Administrator and "shutdown" any running WSL: ```pwsh wsl --shutdown ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/32_wsl_rv-cli.avif) 8. Restart WSL again and check the memory settings from within Arch by running the following command: ```bash free -h ``` ![](https://raw.githubusercontent.com/av1-community-contributors/images/main/33_wsl_rv-cli.avif) If you've made it this far, you should be more than ready to encode. We hope you enjoy rAV1ator CLI! ## macOS No macOS support is provided at this time. It is definitely technically feasible, and I may produce a separate tool in the future with proper macOS binaries, though this would be a burden to keep up considering I don't currently see any demand for a port. ### Troubleshooting - If your encode features a grey screen flashing occaisonally in the output, create a lossless intermediary of your source with x264 `-qp 0`. This happens because of VC-1 decoding errors, and is not something I can fix. - If you have any more questions, please join the [AV1 for Dummies Discord server](https://discord.gg/bbQD5MjDr3). There is a rAV1ator CLI channel over there, and I am always happy to talk! --- # YUView Canonical URL: https://encode.wiki/tools/utilities/yuview/ Collection: docs Status: current License: CC BY-SA 4.0 Checksum: b427f22c73189e794ca48c38696254d801e0caa1b5a265ee5b5a9a58e1fc6322 YUView Window

[YUView](https://github.com/IENT/YUView) is a free and open source, cross-platform software developed by [IENT](https://github.com/IENT/) (Institut für Nachrichtentechnik) for working, playing, and analyzing YUV files. Though it supports more than YUV thanks to FFmpeg and other third party decoders while providing a lot of options for [HEVC](/codecs/video/hevc/) internal visualizations such as slice index and transform depth. The software is built with QT and written in C++. ## Installation IENT provides the complete set of binaries for all major desktop operating systems on their GitHub [releases](https://github.com/IENT/YUView/releases/). So pick and choose your OS there. For Linux, YUView is in the official repo on Ubuntu and the AUR (Arch User Repository) for Arch Linux. But they do provide an AppImage in their releases and a [Flatpak](https://flathub.org/apps/de.rwth_aachen.ient.YUView). For those who want to compile from source instead, they provided a tutorial [here](https://github.com/IENT/YUView/wiki/Compile-YUView). ## Usage Open a file via the "File" > "Open File" tab on the top left or by pressing Ctrl + O, you may import multiple files. After that, you can start analyzing the file such as viewing motion vectors, isolating chroma to Cb for YCbCr, etc. --- # Video Players Canonical URL: https://encode.wiki/tools/video-players/ Collection: docs Status: needs-review License: CC BY-SA 4.0 Checksum: bf363cea7a876210f7a4e3c39e3fc1465d2ccb5bc66b98d86212c149ce51e403 Many different players exist for video, but here are a few recommended ones: ## MPV **MPV** is a open source lightweight media player. It is intended as a command-line application, making it's user interface extremely minimal, however, many frontends exist for mpv, giving a more complete GUI, such as Celluloid and IINA. It has wide codec and container support. TODO: MPV keyboard shorcuts ## VLC **VLC** is an open source media player and toolkit. It supports almost all video and audio codecs in common use. Download instructions for VLC on all relevant platforms can be found at [VideoLAN's website](https://www.videolan.org/vlc/#download) ## MPC-HC **Media Player Classic - Home Cinema (MPC-HC)** is an open source media player designed exclusively for Windows 7, 8, 8.1, 10, 11 operating systems. Recognized for its simplicity and efficiency, MPC-HC provides users with a streamlined and user-friendly platform for playing a diverse range of audio and video file formats. Its lightweight design ensures smooth playback without taxing system resources. While the official development ceased in 2017, the community-driven MPC-HC on GitHub continues to provide updates and support. The latest un/stable builds of MPC-HC, maintained by the community, can be found on [clsid2's MPC-HC repository](https://github.com/clsid2/mpc-hc/releases).