title: zstd

text: Zstandard

anchor: zstandard

text: Usage

anchor: usage

text: Compress a file

anchor: compress-a-file

text: Decompress a file

anchor: decompress-a-file

markdown:
# Zstandard

> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
Zstandard is a compression algorithm developed by Facebook known for its extremely fast decompression speeds. It was released in early 2015 and is used in a variety of different contexts. It was designed to perform similarly to older Deflate-based compression algorithms like [ZIP](/codecs/data/zip/) or [gzip](/codecs/data/gzip/) while being faster overall. In practice, it is said to compress similarly to pure LZMA (part of [XZ](/codecs/data/xz/) & [7-zip](/codecs/data/7z/)) while being much faster.

While `.tar.zstd` archives aren't as popular as `.tar.xz` or `.tar.gz`, Zstandard is already a very popular tool for compression in the world of open-source software. It has been integrated into both the FreeBSD kernel & the Linux kernel and is available as a filesystem compression method for the btrfs, squashfs, bcachefs, & OpenZFS filesystems. Filesystem compression refers to a compression scheme that transparently compresses files stored on a filesystem at all times, leading to an overall reduction in storage used across the filesystem.

The command line `zstd` utility can compress to Zstandard at compression levels 1 through 19 by default. The upper bound is raised to 22 when passing the `--ultra` flag. All Arch Linux packages are compressed at zstd level 20, allowing Arch packages to be decompressed 14 times faster compared to XZ at the cost of an average 0.8% filesize increase across all packages. It is popular in the game emulation scene as well, as many game file formats for emulating console games support zstd compression. The ZIP file format standard actually supports Zstandard in compression level 93 since version 6.3.8, published in 2020. Content encoding using zstd is supported in chromium since Chromium 118 behind an experimental flag, meaning it might compete with [Brotli](/codecs/data/brotli/) on the web in the future. Apple's LZFSE algorithm is purportedly similar to Zstandard compression level 6.

Zstandard has the potential to effectively compete with nearly every modern compression method available across most modern use cases. In certain scenarios, if it takes off as a content delivery format, it could replace Brotli if the benefits of super-fast & super-light decode improve the responsiveness of web pages & are worth sacrificing a bit of compression ratio. When using the much higher effort settings, it often outcompetes Brotli for the archive size as well. In the future, `.tar.zst` could replace 7-zip, ZIP, or other archiving formats, making speedy decode a reality on systems featuring varying levels of compute horsepower.

## Usage
> **Note — This guide has been written for the `zstd` command-line utility, however GUI archivers such as peazip and 7zip have growing support for zstd..**
>
>
## Compress a file
Like many other compressing utilities, in order to compress mutliple files, one should probably archive them with [tar](/codecs/data/tar/).
```bash
zstd -# {file} -o {file}.zstd
```
`-#` is actually a number that represents the desired compression level, for example `-3`, `-6`. By default you can specify 1-19. By also passing `-ultra`, you can go up to compression level 22.

## Decompress a file
```bash
zstd -d {file}.zstd -o file
```


collection: docs

title: AVIF

description: AVIF, which stands for AV1 Image File Format, is a newer image codec that is based on the AV1 video codec. AVIF supersedes HEIC, & uses the same HEIF container as HEIC. AVIF is designed to have a better featureset & bett

status: needs-review

license: CC BY-SA 4.0

text: Performance Checklist

anchor: performance-checklist

text: Format Breakdown

anchor: format-breakdown

text: Advantages

anchor: advantages

text: Limitations

anchor: limitations

text: Encoders

anchor: encoders

text: AV1 Encoders

text: tinyavif

anchor: tinyavif

text: Conclusion

anchor: conclusion

contentFormat: markdown

markdown:
> **Note — Under Maintenance.**
>
> The content in this entry is incomplete & is in the process of being completed.
AVIF, which stands for AV1 Image File Format, is a newer image codec that is based on the [AV1](/codecs/video/av1/) video codec. AVIF supersedes [HEIC](/codecs/images/heic/), & uses the same HEIF container as HEIC. AVIF is designed to have a better featureset & better general lossy compression than older image codecs, including [WebP](/codecs/images/webp/), HEIC, & [JPEG](/codecs/images/jpeg/). AVIF is often compared to [JPEG-XL](/codecs/images/jxl/), though in practice, the two have very different strengths.

There are two AVIF profiles available for encoding: Baseline & Advanced, which are based on AV1's Main & High profiles respectively. The AVIF Baseline profile supports up to 8,192\*4,352 resolution specified by the requirement of using AV1 Level 5.1 or lower. Using tiling, it is possible to increase the maximum resolution of the AVIF Baseline profile to 65536\*65536, although this hurts coding efficiency as visual anomalies may be encountered along the edges of the tile boundaries. AVIF is also limited to 10 bit color precision in its Baseline profile.

In the AVIF Advanced profile, the maximum image dimensions extend to 16,384*8,704. Tiling may be used in the Advanced profile to create larger images, but the same limitations regarding visual artifacts apply. The AVIF Advanced profile extends the allowed AV1 Level to 6.0 or lower, & the highest bit depth offered by this profile is 12 BPC. It is worth noting that while it is currently a near certainty that AVIF implementations will support both the Baseline & Advanced profiles, this may not always be the case. This is a problem that affects HEIC currently, & is a known potential weakness of video-based image codecs.

## Performance Checklist

Lossless? *Poorly*

Lossy? *Yes*

Supported Bit Depths:
*8 BPC, 10 BPC, 12 BPC*

HDR/Wide Gamut? *Yes*

Animation? *Yes*

Transparency? *Yes*

Progressive Decode? *No*

Royalty Free? *Yes*

**Compatible Browsers** (full support)

- [Google Chrome](https://www.google.com/chrome/) 85+

- [Safari](https://www.apple.com/safari/) 16.4+

- [Firefox](https://www.mozilla.org/en-US/firefox/new/) 113+

- [Opera](https://www.opera.com/) 71+

- [GNOME Web](https://apps.gnome.org/app/org.gnome.Epiphany/)

- [Thorium](https://thorium.rocks/)

- [Mercury](https://thorium.rocks/mercury)

## Format Breakdown

### Advantages

AVIF is known for its extremely strong lossy compression performance for non-photographic images as well as photographic images. AVIF is consistently better than JPEG visually.

Using [libaom](/tools/software-encoders/aomenc/)'s `--tune iq` or [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/)'s Tune 4, AVIF is generally the most efficient image codec on the Web for quality per bit. AVIF's quality per bit is generally better than [JPEG XL](/codecs/images/jxl/), though JPEG XL is considerably more feature-rich.

AVIF compatibility has grown rapidly since its adoption in Google Chrome in 2020. For a relatively new image format, its level of penetration has been stellar, especially in the browser market; AVIF is considered [Baseline](https://web-platform-dx.github.io/web-features/) as of 2024.

AVIF's wider featureset enables new experiences through images, including HDR. AVIF also presents astonishing animation prowess, as it is capable of using AV1's video coding techniques which make it easily the best animated image format for most use cases.

### Limitations

AVIF encoding implementations are difficult to use, and images require much longer encoding times for what can be considered competitive quality. Making encoding more difficult, AVIF's use of intra-frame coding techniques that share data between blocks reduces parallelization capability & worsens generation loss. Theoretically, this improves coding efficiency, though. Via the AVIF Encoding section of the aomenc page:

  AVIF Encoding with aomenc through avifenc

Using aomenc through avifenc is widely considered to be the best way to encode AVIF images, as SVT-AV1 only supports 4:2:0 chroma subsampling, rav1e isn't fast enough for still images, & the libaom team have put more effort into intra coding than the teams responsible for producing the other prominent open source AV1 encoders.

A sample command for encoding AVIF looks like this:

avifenc -c aom -s 4 -j 8 -d 10 -y 444 --min 1 --max 63 -a end-usage=q -a cq-level=16 -a tune=ssim \[input\] output.avif

Where:

`-c aom` is the encoder

`-s 4` is the speed. Speeds 4 & below offer the best compression quality at the expense of longer encode times.

`-j 8` is the number of threads the encoder is allowed to use. Increasing this past 12 will sometimes hurt encode times, as AVIF encoding via aomenc doesn't paralellize perfectly. Test using a speed benchmark to verify which value works best for you.

`-d 10` is the bit depth. Specifying a value below 10 isn't recommended, as it will hurt coding efficiency even with an 8 bit source image.

`-y 444` is the chroma subsampling mode. 4:4:4 chroma subsampling tends to provide better compression than 4:2:0 with AVIF, though on some images 4:2:0 chroma subsampling might be the better choice.

`cq-level=16` is how you specify quality. Lower values correspond to higher quality & filesize, while higher values mean a smaller, lower-quality output is desired. This is preceded by `-a` because it is an aomenc option, not an avifenc one.

`tune=ssim` is how the encoder handles RDO (rate-distortion optimization). This may be redundant with the default aomenc parameters, but specifying doesn't hurt to avoid an unintended change if a default is modified sometime in the future.

AVIF does not have progressive decode. This is a common weakness of video-based image codecs. While there is a hacky way to do progressive AVIF by encoding a low fidelity frame & then a high fidelity frame in an animated AVIF at a high framerate so the low fidelity frame is loaded & plays first, this is far from ideal for the average user & adds to an already burdensome encoding process. Additionally, this has issues with Firefox.

Finally, AVIF's lossless mode is underwhelming, often producing larger files than PNG. When compressing losslessly, it can generally be advised that you avoid AVIF.

### Encoders

#### AV1 Encoders

In practice, any usable AV1 encoder should be able to produce AVIF images. This makes our list of feature-complete, open-source AVIF encoders end up looking quite familiar:

- [libaom](/tools/software-encoders/aomenc/)
- [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/)
- [SVT-AV1](/tools/software-encoders/svt-av1/)
- [rav1e](/tools/software-encoders/rav1e/)

In the proprietary space, [Aurora1](/tools/software-encoders/aurora1/) was used at Cloudinary (a major multimedia CDN) for AVIF compression for a while, but has since been replaced by libaom.

#### tinyavif

Rachel Barker's tinyavif (whose original public repository is no longer available) is a barebones AVIF encoder that is designed to be easy to understand. It is not feature-complete, but it is a good choice for those who want to learn more about how AVIF works (and by extension, how AV1 works). It is written in Rust, and is described as "the world's most minimal AV1 encoder."

## Conclusion

AVIF is a strong image format that is worth considering for use on the web. It is a great choice for images that are not high fidelity, as it will compress better than JPEG in most cases. AVIF is also a great choice for animated images, as it is the best format for this use case. However, AVIF is not a great choice for lossless images, as it tends to produce larger files than PNG. Overall, AVIF is a great format to use for images that are not high fidelity, as it will compress better than JPEG in most cases.


title: GIF

description: Graphics Interchange Format is an image file format first released by CompuServe in 1987. It remains popular due to it's widespread support for animated images despite its obsolete efficency. Other animated image formats

text: Performance Checklist

markdown:
> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
Graphics Interchange Format (GIF) is an image file format first released by CompuServe in 1987. It remains popular due to it's widespread support for animated images despite its obsolete efficency. Other animated image formats like Animated [AVIF](/codecs/images/avif/) & Animated [WebP](/codecs/images/webp/) have since surpassed GIF in functionality, as has the animated [PNG](/codecs/images/png/) variant APNG.

## Performance Checklist

Lossless? *Yes*

Lossy? *No*

Supported Bit Depth:
*256 colors*

HDR/Wide Gamut? *No*

Animation? *Yes*

Transparency? *Yes*

Progressive Decode? *No*

Royalty Free? *Yes*


title: HEIC

description: The HEIC image format, also known as the High Efficiency Image Format, is a newer image codec that was developed to provide improved compression and better performance compared to traditional image formats like JPEG. HEI

text: Performance Checklist

markdown:
> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
The HEIC image format, also known as the High Efficiency Image Format, is a newer image codec that was developed to provide improved compression and better performance compared to traditional image formats like [JPEG](/codecs/images/jpeg/). HEIC files use [HEVC](/codecs/video/hevc/) internally, meaning the format is not royalty free. While this has limited its adoption across the Web, this format is supported by many modern devices including the entire Apple ecosystem. iPhones shoot HDR HEIC photos by default by utilizing the iPhone's HEVC hardware video encoder to capture these images. Some Android phones are capable of shooting HEIC as well, but these are often transcoded from JPEG. HEIC has largely been surpassed by [AVIF](/codecs/images/avif/), which uses the same container to store [AV1](/codecs/video/av1/)-compressed images.

## Performance Checklist

Lossless? *No*

Lossy? *Yes*

Supported Bit Depths:
*8 BPC, 10 BPC*
> *Higher bit depths not widely supported*

HDR/Wide Gamut? *Yes*

Animation? *Yes*

Transparency? *Yes*

Progressive Decode? *No*

Royalty Free? *No*


title: JPEG

description: JPEG compression is a widely used method for reducing the size of digital images while preserving visual quality. It's based on the principles of lossy compression, which means that some image data is discarded to achiev

status: current

text: Performance Checklist

text: Compression

anchor: compression

text: Color Space Conversion

anchor: color-space-conversion

text: Image Tiling

anchor: image-tiling

text: Discrete Cosine Transform (DCT)

anchor: discrete-cosine-transform-dct

text: Quantization

anchor: quantization

text: Zigzag Scanning

anchor: zigzag-scanning

text: Run-Length Encoding

anchor: run-length-encoding

text: Entropy Encoding

anchor: entropy-encoding

text: Saving the File

anchor: saving-the-file

text: Decoding

anchor: decoding

markdown:
JPEG (Joint Photographic Experts Group) compression is a widely used method for reducing the size of digital images while preserving visual quality. It's based on the principles of lossy compression, which means that some image data is discarded to achieve a smaller filesize.

## Performance Checklist

Lossless? *No*

Lossy? *Yes*

Supported Bit Depth:
*8 BPC*

HDR/Wide Gamut? *Kinda*

Animation? *No*

Transparency? *No*

Progressive Decode? *Yes*

Royalty Free? *Yes*

## Compression

Learning how JPEG compresses images is immensely helpful for understanding how other compression methods work in other codecs. It is definitely worth reading to get a useful background in understanding concepts like entropy coding, the DCT, and color spaces other than RGB. Here's a step-by-step explanation of how JPEG compression works:

#### Color Space Conversion
   Most digital images are originally in the RGB (Red, Green, Blue) color space. The first step in JPEG compression is to convert the image to the YCbCr color space. Y represents the luminance (brightness), while Cb and Cr represent the chrominance (color information). The Cb & Cr components are subsampled to a quarter of the resolution of the original image, meaning the resulting color space is chroma subsampled with *4:2:0* subsampling.

#### Image Tiling
   The image is divided into smaller blocks or tiles, typically 8x8 pixels each. Each of these blocks will be processed separately.

#### Discrete Cosine Transform (DCT)
   For each 8x8 block, a mathematical transformation called the [Discrete Cosine Transform](/start-here/terminology/#discrete-cosine-transform-dct) is applied. This transformation converts the pixel values into a set of frequency components, taking spatial data and transforming it to the frequency domain. The DCT is applied to each color channel in the YCbCr color space. This algorithm is a particularly good choice for image (and music/speech) compression because it has high energy compaction relative to our understanding of images & their perceptual quality. High energy compaction means the DCT is able to represent a signal with a small number of significant coefficients, in this case mainly in the lower frequencies.

#### Quantization
   After the DCT, the frequencies are quantized in a table representing frequency coefficients & their corresponding frequencies. Less perceptually important details can be omitted to reduce filesize by discarding coefficients in the table that correspond to less visually salient frequencies. This is "lossy" compression, and is the key step in achieving a high compression ratio while still maintaining an image that looks reasonable. The quantization table used in this step can vary in the number of frequencies it attempts to retain, affecting the trade-off between compression & image quality.

#### Zigzag Scanning
   The quantized coefficients are then reordered using a zigzag pattern. This is done to prepare the data for the next step.

#### Run-Length Encoding
   The zigzag-ordered coefficients are run-length encoded. This means that sequences of zeroes are compressed into a shorter representation. For example, if there are many consecutive zeroes in the data, they can be represented as (0, 10) instead of listing ten individual zeroes.

#### Entropy Encoding
   The run-length encoded data is further compressed using entropy encoding. JPEG uses Huffman coding, which assigns shorter codes to more frequently occurring values in the table of DCT coefficients, reducing the overall file size.

#### Saving the File
   The compressed luminance and chrominance data, along with information about color space conversion, quantization tables, and EXIF data, are saved in the JPEG file format.

#### Decoding
   When you open a JPEG image, the reverse process occurs. The file is decoded, and the DCT coefficients are dequantized, the inverse DCT is applied, and the image is converted back to the RGB color space to be displayed on a screen.

It's important to note that JPEG compression is ***lossy***, meaning that some image quality is discarded in the pursuit of smaller file sizes. This makes it different than codecs designed for lossless compression like [PNG](/codecs/images/png/), [WebP](/codecs/images/webp/)'s lossless mode, and [JPEG-XL](/codecs/images/jxl/)'s lossless mode. The degree of compression and the quality of the compressed image can be adjusted through settings when saving a JPEG, allowing for a trade-off between file size & image fidelity.

While JPEG is certainly not the most state of the art lossy image codec compared to its newer and (usually) better successors like [JPEG-XL](/codecs/images/jxl/) (an actual direct successor) & [AVIF](/codecs/images/avif/), it enjoys near universal compatibility with (likely) most utilities you would work with in your everyday life that have anything to do with images.


title: JPEG 2000

markdown:
> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
JPEG-2000 is an older image compression format that uses wavelet technology to achieve high compression ratios while maintaining image quality. It supports both lossy and lossless compression, and is commonly used in applications such as digital photography, medical imaging, and video surveillance. JPEG-2000 files can be transparently compressed and decompressed using a variety of software tools and libraries, making it a flexible and widely-supported format for image storage & transmission. JPEG-2000 never effectively took off on the Web, but digital cinema distribution is often done with JPEG-2000. A "DCP" is a "Digital Cinema Package," which is a format used to distribute and play back digital movies in theaters. These DCPs are often compressed losslessly with JPEG-2000.


title: JPEG XL

description: JPEG XL is a compression format for images that was developed by the Joint Photographic Experts Group in 2020. It is designed to provide improved compression efficiency compared to the traditional JPEG format, while stil

text: JPEG XL

anchor: jpeg-xl

text: Performance Checklist

anchor: performance-checklist

text: Format Breakdown

text: Lossless Compression

anchor: lossless-compression

text: Lossy Compression

anchor: lossy-compression

text: Supported Bit Depth(s)

anchor: supported-bit-depths

text: Progressive Decode

anchor: progressive-decode

text: Lossless JPEG Re-Compression

anchor: lossless-jpeg-re-compression

text: Industry Support

anchor: industry-support

text: Other Features

anchor: other-features

text: Encoders

text: libjxl

anchor: libjxl

text: libjxl-tiny

anchor: libjxl-tiny

text: Hydrium

anchor: hydrium

text: zune-jpegxl

anchor: zune-jpegxl

text: Decoders

anchor: decoders

text: jxl-oxide

anchor: jxl-oxide

markdown:
# JPEG XL

JPEG XL (JXL) is a compression format for images that was developed by the Joint Photographic Experts Group (JPEG) in 2020. It is designed to provide improved compression efficiency compared to the traditional [JPEG](/codecs/images/jpeg/) format, while still maintaining image quality.

JPEG XL uses a combination of techniques such as perceptual color encoding & advanced entropy coding to achieve its improved compression performance. It also has a lossless JPEG recompression mode, where an existing JPEG file can be turned into a JXL that can be decoded for a bit-for-bit exact replica of the original JPEG.

## Performance Checklist

Lossless? *Yes*

Lossy? *Yes*

Supported Bit Depths:
*Up to 32 BPC*

HDR/Wide Gamut? *Yes*

Animation? *Yes*

Transparency? *Yes*

Progressive Decode? *Yes*

Royalty Free? *Yes*

## Format Breakdown

JPEG XL has a number of standout features that make it an appealing image codec to work with for many use cases. From the [JPEG XL Info page](https://jpegxl.info), JXL has the following features:

- **Best lossless image compression**: It offers about 35% smaller file sizes than PNG (50% smaller for HDR).
- **High-fidelity lossy image compression**: JPEG XL provides about 60% smaller file sizes than JPEG for the same visual quality.
- **Progressive decoding**: This allows an image to be displayed in lower quality before the entire file has been downloaded, improving user experience on slow connections.
- **Lossless JPEG transcoding**: JPEG images can be converted to JPEG XL without any mathematical loss, and the resulting file is about 20% smaller.
- **Designed for both photographic and synthetic images**: JPEG XL works well with a wide range of image types, including photos, graphics, and illustrations.
- **Fast software encoding and decoding**: The codec is designed to be efficient and fast, enabling quick image loading and saving.
- **Full support for wide gamut and HDR**: JPEG XL supports a wide range of colors and high dynamic range, making it suitable for modern displays.
- **Perceptually optimizing reference encoder**: The encoder is designed to optimize image quality based on how humans perceive images.

### Lossless Compression

JPEG XL offers excellent lossless compression capabilities. While lossless WebP was an improvement over PNG for 8-bit lossless image encoding, JPEG XL manages not only to outdo lossless WebP in encoding efficiency but also be more versatile for bit depths greater than 8-bit (a category PNG previously dominated). 16-bit lossless imagery, especially HDR images that are becoming more popular & rarely utilize 8-bit color depth, are where JPEG XL shines, and it is the only codec to compete with PNG in that regard while providing better coding efficiency.

Example: JPEG XL compresses [this 16-bit AdobeRGB PNG](https://imgsaver.com/images/2023/10/03/16bit.png) better than PNG. Using: `cjxl 16bit.png 16bit.jxl -d 0.0 -e 9 -I 100 -g 3 -E 11`

16-bit PNG: `1533373` bytes.
16-bit JXL: `1211029` bytes.

### Lossy Compression

JPEG XL is also adept at lossy compression, especially at quality levels that we as humans care about. It promises to be around 60% better than JPEG. While video-based codecs like AVIF are often better in terms of quality per bit, JPEG XL is both fast and efficient for medium and high fidelity photographic image compression.

### Supported Bit Depth(s)

JPEG XL supports up to 32 bits per channel of bit depth, making it future proof for the increasingly popular HDR photos coming out of smartphones. There is essentially zero downside to encoding high bit depth with JXL relative to the resulting encode's size. Considering many smartphones take HDR photos now, JXL offers a compelling pipeline for these photos to make their way to the Web in the future especially as companies like Adobe & Apple have already embraced the new codec.

### Progressive Decode

JPEG XL provides actual progressive decode support that you can experiment with here on a supported browser like Safari, Waterfox, Thorium, Mercury, or any browser on iOS.

Progressive decode is a feature only JPEG is able to offer a real implementation of, rendering low frequency transform coefficients before the rest of the image arrives to allow an image to display before the entire thing has been sent over the network. Blurhashes do not replace this technology, but rather compliment it, allowing another layer of progressive decode that can be used even before the image begins to load progressively. This is an important feature to improve the user experience on websites featuring large images, or on any website if your Internet connection isn't strong.

### Lossless JPEG Re-Compression

An incredibly unique JPEG XL feature is lossless JPEG re-compression, or the ability to take a JPEG input and provide an output with a smaller filesize (on average, 20% smaller) that is pixel-for-pixel identical. This is why companies like Meta have endorsed JPEG XL, as it offers a path forward for the existing JPEGs on the Internet.

### Industry Support

From the JPEG XL Wikipedia page:

> Besides Cloudinary and Google originally, throughout JPEG XL's preliminary implementation in web browsers, various representatives of well-known industry brand names have publicly voiced support for JPEG XL as their preferred choice, including Facebook, Adobe, Intel and the Video Electronics Standards Association, The Guardian, Flickr and SmugMug, Shopify, the Krita Foundation, and Serif Ltd.

Apple also features ecosystem-wide JPEG XL support as of iOS 17 & macOS Sonoma.

### Other Features

JPEG XL has the potential to replace popular formats like TIFF for authoring workflows due to its broad feature set. From the JXL Wikipedia, some additional features include:

- Image dimensions of over a billion (2^30-1) pixels on each side.
- Up to 4099 channels, including support for alpha transparency
- There can be multiple frames with zero duration, allowing support for layers in graphics software
- Animation support, allowing JXL to rival GIF
- Images can be stored in tiles to reduce the time needed to decode them.
- Graceful quality degradation across a large range of bitrates means quality loss isn't as abrupt as with older formats.
- Perceptually optimized reference encoder which uses a perceptual color space, adaptive quantization, and conservative default settings.
- Support for wide color gamut and HDR
- Efficient encoding and decoding without requiring specialized hardware: JPEG XL is about as fast to encode and decode as old JPEG using libjpeg-turbo and an order of magnitude faster to encode and decode compared to HEIC with x265. It is also parallelizable.
- Royalty-free format with an open-source reference implementation available on GitHub.

## Encoders

JPEG XL has a couple of noteworthy encoders currently available to work with. Because JPEG XL is so new, most encoders aren't yet intelligent enough to take advantage of the whole format yet. Here's a quote from Jon Sneyers in the JPEG XL discord that sums it up nicely:

> Encode side: 80% or so of the coding tools are used in one way or another by the encoder (the 20% is splines and super large VarDCT blocks, and also the things that are not used by default without using special experimental options, such as delta palette and noise). But the coding tools that are used, are typically used in a specific, limited way that doesn't come anywhere close to exhausting the bitstream expressivity.

Sneyers is talking about libjxl's `cjxl` encoder, which will be discussed further below.

### libjxl

The reference [libjxl](https://github.com/libjxl/libjxl) implementation has the capability to both decode and encode JPEG XL image files. Both are discussed below.

    #### Encoding

libjxl's encoder `cjxl` has more options to play around with. It takes a few primary arguments, distance (`-d`), quality (`-q`), and effort (`-e`).

    **Distance and quality**

    Distance and quality are two ways of specifying *how much loss* you are willing to tolerate, and as such, they are mutually exclusive, as they pull the same levers under the hood.
    * Distance is designed to map to how 'close' one must be to the source to notice any loss. It is represented as a scale between 0.0 & 25.0. 0.0 is **mathematically lossless**, every pixel will have the exact same value as the source. 1.0 is designed to be **visually lossless**, look the same at a normal viewing distance, and higher values have more loss.
    * Quality is designed to roughly map to [JPEG](/codecs/images/jpeg/)'s quality argument. A range 0-100, where 100 is **mathematically lossless**, 90 is intended to be **visually lossless**, and 0 is almost unrecognizable as the original image.

    **Effort**

    Effort is similar to `cpu-used` in video encoding. It specifies the amount of effort the encoder will make in order to get the smallest file size it can. It takes the form of a range 1-9, where higher numbers will spend more resources to get diminishing returns in terms of smaller size, while lower values do the opposite, leaving file size on the table for faster encoding.

    ```bash title="Encoding with effort 9 and distance 1.0"
    cjxl -e 9 -d 1.0 example.png example.jxl
    ```

    ```bash title="This, by default uses lossless JPEG compression."
    cjxl example.jpg example.jxl
    ```
    #### Decoding

Decoding a `.jxl` image is straightforward with libjxl's decoder, `djxl`:

    ```bash
    djxl example.jxl example.png
    ```
    `djxl` can decode to pixels via pipes, png, apng for animated jxl, jpg, ppm, and pfm.

    By default, if the `.jxl` file was encoded with lossless jpeg recompression, `djxl` will rebuild the exact jpeg file that was originally compressed. To avoid this, and create a new jpeg file:
    ```bash
    djxl -j example.jxl example.jpg
    ```
    **Keep in mind this is now a lossy process as `djxl` will decode to pixels, then encode a new `.jpg` with those pixels.**
    #### Building

A full build guide is provided in the [libjxl build instructions](https://github.com/libjxl/libjxl/blob/main/BUILDING.md) in the GitHub repo. This guide is simplified, and is only focused on building a working efficient encoder & decoder.

    These instructions should work for macOS and Linux, although macOS support isn't guaranteed.

    ```bash title="1. Clone the repo"
    git clone https://github.com/libjxl/libjxl.git --recursive --shallow-submodules
    ```

    ```bash title="2. Install dependencies. May have to run these commands with root"
    apt install cmake pkg-config libbrotli-dev clang # Debian Linux
    pacman -Syu cmake pkgconf brotli clang # Arch Linux
    brew install cmake pkg-config brotli # macOS
    ```

    ```bash title="3. Set CC & CXX variables before building (Recommended)"
    export CC=clang CXX=clang++
    ```

    ```bash title="4. cjxl & djxl will be available in the build/tools directory."
    cd libjxl && mkdir build && cd build
    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-O3 -march=native" -DCMAKE_C_FLAGS="-O3 -march=native" -DBUILD_TESTING=OFF -DJPEGXL_WARNINGS_AS_ERRORS=OFF -DJPEGXL_ENABLE_SJPEG=OFF ..
    cmake --build . -- -j$(nproc)
    ```

    This will build `cjxl` and `djxl` with O3 optimization for your CPU architecture on Linux or macOS. Again, be aware that macOS support is not a priority. Via the libjxl OS X build guide:

    > OSX builds have "best effort" support, i.e. build might not work at all, some tests may fail and some sub-projects are excluded from build.

### libjxl-tiny

[libjxl-tiny](https://github.com/libjxl/libjxl-tiny) contains a simpler encoder implementation of JPEG XL, aimed at photographic images without an alpha channel. The goal is to guide hardware implementations of the encoder where support for the full set of encoding tools is not feasible. The color management is outside the scope of this library, the encoder input is given as a portable float map (PFM) in the linear sRGB colorspace, where individual sample values can be outside the \[0.0, 1.0\] range for out-of-gammut colors. For more details, see the [overview of the coding tools](https://github.com/libjxl/libjxl-tiny/blob/main/doc/coding_tools.md).

The last commit was ten months ago, so it is uncertain whether libjxl-tiny could be considered active.

### Hydrium

[Hydrium](https://github.com/Traneptora/hydrium) is a fast, ultra-low-memory, streaming JPEG XL encoder written in portable C. It is maintained by Traneptora.

### zune-jpegxl

[zune-jpegxl](https://github.com/etemesi254/zune-image/tree/dev/crates/zune-jpegxl) is a simple, fast and fully safe modular JXL encoder written in Rust. It is maintained by etemesi254.

zune-jpegxl has the following features:

- Lossless encoding
- 8 bit and 16 bit support
- Grayscale and RGBA encoding
- Threading capabilities

## Decoders

### jxl-oxide

[jxl-oxide](https://github.com/tirr-c/jxl-oxide) is a spec-conforming JPEG XL decoder written in pure Rust. It is maintained by Wonwoo Choi.

{}

Sources:

- [JXL Wikipedia](https://en.wikipedia.org/wiki/JPEG_XL)
- [JPEGXL.info: Why JXL](https://jpegxl.info/why-jxl.html)
- [Apple JXL Announcement](https://webkit.org/blog/14205/news-from-wwdc23-webkit-features-in-safari-17-beta/#images)
- [JPEG XL: How It Started, How It's Going](https://cloudinary.com/blog/jpeg-xl-how-it-started-how-its-going)
- [The Case for JPEG XL](https://cloudinary.com/blog/the-case-for-jpeg-xl)
- [Time for Next-Gen Codecs to Dethrone JPEG](https://cloudinary.com/blog/time_for_next_gen_codecs_to_dethrone_jpeg)
- [Image Codec Comparison](https://giannirosato.com/blog/post/image-comparison/)


title: PNG

description: Portable Network Graphics is a free lossless image file format released in 1996. It was ceated as an alternative to GIF, which was at the time a proprietary format. It gained animation support similar to GIF with the rel

text: Performance Checklist

markdown:
> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
Portable Network Graphics (PNG) is a free lossless image file format released in 1996. It was ceated as an alternative to [GIF](/codecs/images/gif/), which was at the time a proprietary format. It gained animation support similar to GIF with the release of APNG in 2008, which is now supported by all popular web browsers.

## Performance Checklist

Lossless? *Yes*

Lossy? *No*

Supported Bit Depths:
*8 BPC, 16 BPC*

HDR/Wide Gamut? *Yes*

Animation? *Yes*

Transparency? *Yes*

Progressive Decode? *Kinda*

Royalty Free? *Yes*


title: QOI

description: QOI is an image compression format that aims to provide a simple, fast, and efficient way to compress and decompress images losslessly. It was designed to be easy to implement while offering better compression ratios tha

text: Performance Checklist

text: Format Breakdown

anchor: format-breakdown

text: Benchmarks

anchor: benchmarks

text: Advantages

anchor: advantages

text: Limitations

anchor: limitations

markdown:
QOI (Quite OK Image Format) is an image compression format that aims to provide a simple, fast, and efficient way to compress and decompress images losslessly. It was designed to be easy to implement while offering better compression ratios than the widely used but more complex [PNG](/codecs/images/png/) format while achieving much faster encoding & decoding speeds.

## Performance Checklist

Lossless? *Yes*

Lossy? *No*

Supported Bit Depths:
*8 BPC*

HDR/Wide Gamut? *No*

Animation? *No*

Transparency? *Yes*

Progressive Decode? *No*

Royalty Free? *Yes*

## Format Breakdown

QOI compression is based on a simple and fast algorithm that exploits spatial redundancy in images. The algorithm uses a combination of run-length encoding (RLE), a small lookup table, delta encoding, and full-color pixel storage to achieve efficient compression. Depending on the algorithm's decision, a chunk (pixel) can take up one to five bytes.

The QOI format supports images with 3 or 4 channels (RGB or RGBA) and 8 bits per channel. The format supports two colorspaces: Linear RGB & sRGB with linear alpha. These do not affect the way pixels are encoded.

Here is a breakdown of the various chunk types in QOI:

1. **`QOI_OP_RGB`**: Full RGB pixel value using 8 bits (1 byte) for each of the red, green, and blue channels. The alpha channel is 255 in RGB images, and always remains unchanged.

2. **`QOI_OP_RGBA`**: Full RGBA pixel value using 8 bits for each of the red, green, blue, & alpha channels.

3. **`QOI_OP_DIFF`**: The difference between the current pixel and the previous pixel for the red, green, and blue channels are stored using 2 bits for each channel. The differences are stored with a bias of 2 and wrap (so 1 minus 2 would be 255). The alpha channel remains unchanged.

4. **`QOI_OP_LUMA`**: These pixels encode the green channel difference from the previous pixel using 6 bits, and then encode the red and blue channel differences relative to the green channel difference using 4 bits each. This allows for more efficient encoding of small color changes. The alpha channel remains unchanged.

5. **`QOI_OP_RUN`**: These are the simplest, encoding a run-length of pixels that are identical to the previous pixel. The run length is stored using 6 bits with a bias of -1, allowing for runs of 1 to 62 pixels.

6. **`QOI_OP_INDEX`**: These are stored by referencing a previously seen pixel value from a rolling array of 64 recent pixel values by using a simple hash on each pixel as it is identified. If another pixel matches a previously seen hash value in the array, the index of the referenced pixel is stored.

The QOI format also includes a simple 14-byte header that stores the image dimensions, color space, and channel depth information. The end of file is signaled by an 8-byte end marker.

### Benchmarks

The creator of QOI benchmarked the format against libpng & `stbi_image_write` using the C implementation in QOI via [`qoibench.c`](https://github.com/phoboslab/qoi/blob/master/qoibench.c) on a collection of 2,879 screenshots, icons, photos, & textures ([source](https://qoiformat.org/benchmark/qoi_benchmark_suite.tar)). The results are as follows:

| **Library** | **Decode (ms)** | **Encode (ms)** | **Decode MP/s**  | **Encode MP/s**  | **Size (kb)** | **Compression Rate** |
|---------|-----------|-----------|--------------|--------------|---------|-------:|
| `libpng`| 7.0       | 83.8      | 66.56        | 5.54         | 398     | 24.2%  |
| `stbi`  | 7.0       | 60.5      | 66.63        | 7.67         | 561     | 34.2%  |
| `qoi`   | 2.1       | 2.9       | 226.03       | 161.99       | 463     | 28.2%  |

The results show that QOI is significantly faster than libpng and `stb_image_write`, and it also achieves better compression ratios than libpng on average.

### Advantages

Some of the key advantages of QOI include:

- Super simple: [the spec](https://qoiformat.org/qoi-specification.pdf) is only one page
- Extremely fast encoding & decoding speeds
- Data chunks are byte-aligned, so data can be streamed to a decoder one byte at a time
- Better compression ratios compared to PNG for many types of images
- Supports transparency
- Royalty-free, open-source (CC0), & easy to integrate into any application

### Limitations

- Limited to 8 bits per channel (no support for higher bit depths)
- Not suitable for images with high-frequency noise or very little spatial redundancy
- Lacks advanced features like progressive loading, interlacing, or custom metadata

Despite its limitations, QOI provides a compelling alternative to PNG for many use cases where simplicity, speed, and good compression ratios are desired. QOI is not especially well supported at present, but adoption is rapidly growing as developers can easily integrate support into their applications due to the format's simplicity.


title: WebP

description: WebP is a free image file format first released by Google in 2010. It consists of 2 primary "modes" of operation. A lossy mode derived from the VP8 video codec, and a novel lossless mode added in 2011.

text: Performance Checklist

anchor: performance-checklist

text: History

anchor: history

text: Adoption

anchor: adoption

text: Criticism

anchor: criticism

text: Vulnerabilities

anchor: vulnerabilities

text: WebP 2

text: Technical Details

anchor: technical-details

text: Lossy

anchor: lossy

text: Lossless

anchor: lossless

text: Extended Format Features

anchor: extended-format-features

text: Conclusion

anchor: conclusion

markdown:
WebP is a free image file format first released by Google in 2010. It consists of 2 primary "modes" of operation. A lossy mode derived from the [VP8](/codecs/video/vp8/) video codec, and a novel lossless mode added in 2011.

## Performance Checklist

Lossless? *Yes*

Lossy? *Yes*

Supported Bit Depth:
*8 BPC*

HDR/Wide Gamut? *No*

Animation? *Yes*

Transparency? *Yes*

Progressive Decode? *No*

Royalty Free? *Yes*

## History

Google announced the WebP format on 30 September 2010. It was initially proposed as a new open format for lossy compressed true-color graphics on the web, aiming to create files that were smaller than comparable JPEG files while maintaining similar image quality. The foundation of WebP utilized technology acquired by Google during its purchase of On2 Technologies. WebP is closely related to VP8, serving as a derivative, and is a sister project to the WebM multimedia container format. The reference library (libwebp) is released under a BSD free software license.

Additional features were added after the initial release:

- Extended File Format (October 2011): An extension enabling WebP to support features like animation, embedding an ICC profile, and including XMP and Exif metadata. This format also initially supported tiling, but that was later removed.
- Lossless Compression and Transparency (November 2011): Google announced a new lossless compression mode and the ability to support transparency (alpha channel) in both lossy & lossless modes. This support was enabled by default in libwebp, starting with version 0.2.0 on 16 August 2012. Google's initial benchmarks showed that converting PNG files found on the web to lossless WebP resulted in a 45% reduction in file size.

### Adoption

libwebp is developed by Google, and reached version 1.0 in April 2018. In November 2024, WebP was formally specified and published by the IETF as [RFC 9649](https://www.rfc-editor.org/rfc/rfc9649).

WebP has seen widespread adoption across the internet to reduce image sizes. [caniuse.com](https://caniuse.com) reports that over [95% of web browsers fully support WebP](https://caniuse.com/?search=webp). Support has also grown across various software programs since 2010.

### Criticism

Despite its adoption, WebP has faced criticism:

Critics, including [Josh Aas from Mozilla Research](https://research.mozilla.org/2014/07/15/mozilla-advances-jpeg-encoding-with-mozjpeg-2-0/) in 2014, have questioned whether the format offers significant speed benefits, noting that studies were "not able to conclude that WebP outperformed JPEG by any significant margin". Early critiques in 2010 noted that the quality of WebP-encoded results could be poor, particularly due to blurriness, arguing that the encoder attempted to optimize too hard for [PSNR](/reference/metrics/psnr/) rather than [psychovisual quality](/start-here/psychovisual/). There is apparently an attempt to address this issue in the form of Halide Compression's [Iris-WebP](https://halide.cx/iris/) encoder released in 2025, though performance claims cannot be directly validated due to the encoder's proprietary nature.

The biggest challenge facing WebP's reputation has been lack of compatibility with older software, and the practice of content delivery networks replacing source JPEG/PNG files with WebP versions have been cited as making the format "user-unfriendly". Users who download images often find they must convert the file format later.

### Vulnerabilities

In September 2023, critical vulnerabilities were discovered relating to WebP images in libwebp's decoder. [CVE-2023-4863](https://www.cve.org/CVERecord?id=CVE-2023-4863), was actively exploited and carried a high-risk rating (CVSS 8.8). This flaw could be triggered by a maliciously crafted lossless WebP file, potentially causing an overflow condition that could result in denial of service or remote code execution. The extensive use of libwebp across major browsers posed a patching challenge as well.

### WebP 2

In June 2021, Google began developing WebP 2, intended to achieve better compression ratios while providing faster encoding and decoding speeds than other modern formats. However, in October 2022 Google changed [the development repository](https://chromium.googlesource.com/codecs/libwebp2/)'s README file to state that "WebP 2 would not be released as an image format."

## Technical Details

The WebP file format is based on the [Resource Interchange File Format](https://en.wikipedia.org/wiki/Resource_Interchange_File_Format) (RIFF). The file structure is composed of chunks, each identified by a 32-bit FourCC (four-character code) and followed by a 32-bit size field indicating the payload size. The WebP container allows for features beyond a single VP8 key frame. The side length of WebP images is limited to 16,383 pixels squared.

WebP uses two primary compression schemes: lossy (based on VP8) and lossless (novel).

### Lossy

See the [VP8](/codecs/video/vp8/) page for more information.

### Lossless

> **Note — Lossless Compression.**
>
> If you would like to dive more deeply into the topic of lossless compression, you can check out the [Lossless Compression](/start-here/lossless/) entry in the Introduction section of the wiki.
WebP's lossless compression uses a newer algorithm designed by Google software engineer Jyrki Alakuijala, which is unrelated to VP8. Simple lossless WebP files use a RIFF container followed by a "VP8L" chunk containing the VP8L bitstream data.

Lossless WebP supports **8-bit RGBA** (red, green, blue, alpha) color space exclusively. The format stores and restores pixel values exactly, even for fully transparent pixels. It relies on a universal algorithm for sequential data compression (LZ77), prefix coding (Huffman coding), and a color cache to compress bulk data.

The lossless mode employs dedicated entropy codes for different color channels, exploitation of 2D locality of backward reference distances, A color cache for recently used colors (which allows referencing them with shorter codes), and transforms such as the Predictor Transform, Color Transform, Subtract Green Transform, and Color Indexing Transform are applied before entropy coding to reduce symbolic entropy.

A distinctive aspect of the lossless format is its *recursive definition*: control images used for functions like local entropy code selection are encoded using the same methods as the main image itself.

### Extended Format Features

The WebP container format (the RIFF container for WebP) enables several advanced features collectively known as the *Extended File Format*.

WebP supports animation, which is promoted by Google as an alternative to GIF. Animated WebP supports 24-bit color depth with transparency, allows combining frames using both lossy and lossless compression within the same animation, and supports seeking to specific frames. The animation parameters are managed by:
- A 'VP8X' chunk to indicate the extended WebP spec
- An 'ANIM' Chunk containing global parameters, such as the background color & loop count (0 = infinite loops)
- Multiple 'ANMF' Chunks: Contain information for a single frame, including its position, duration, blending method (alpha-blending or overwrite), and disposal method (leave canvas as is, or dispose to background color).

Converting animated GIFs to lossy WebP can reportedly reduce file size by 64%.

Additionally, transparency information is supported through an alpha channel. In the extended format, alpha data for lossy images is stored in an optional 'ALPH' chunk. This data can be stored as raw 8-bit transparency values or compressed using the WebP lossless format.

The extended format also allows embedding metadata and color profiles. An image may contain an embedded ICC profile, described by the International Color Consortium. If not present, sRGB is assumed. Metadata can be stored in Exif or XMP formats, placed in optional 'EXIF' or 'XMP' chunks.

The RIFF container allows for the inclusion of *unknown chunks* (FourCCs not defined in the specification) for future extensions or application-specific data. Readers should ignore these chunks, and writers should preserve them.

## Conclusion

WebP is largely succeeded by [AVIF](/codecs/images/avif/), and the libwebp encoder has not received adequate psychovisual attention to compete with more technically advanced encoders like [libaom](/tools/software-encoders/aomenc/) or more well-optimized perceptual encoders like Google's [jpegli](https://github.com/google/jpegli) JPEG encoder.

While a lot of WebP's original criticisms aren't as relevant in 2025, the three biggest issues with the format that have historically drawn the most ire (lack of support, vulnerabilities, poor perceptual encoding) have marred WebP's reputation in a way that the community may not reconcile for a long time, if ever. Especially since the Chromium browser engine's removal of the [JPEG XL](/codecs/images/jxl/) image format (effectively killing widespread web adoption), new video-derived web image codecs sit in an uncomfortable place, and WebP is the poster child of this unfortunate discomfort.


title: SRT

description: SubRip Text is a text format for subtitles, described as 'the most basic of all subtitle formats'. SRT files are plain text with the extension .srt.

text: Format

anchor: format

text: Unoffical features

anchor: unoffical-features

text: Example

anchor: example

markdown:
SubRip Text (SRT) is a text format for subtitles, described as 'the most basic of all subtitle formats'. SRT files are plain text with the extension `.srt`.

## Format
Subtitles are placed into sequentially ordered groups, called cues, with a starting and ending timestamp, encoded `hours:minutes:seconds,milliseconds`. Note the seperator for the millisecond value is a comma. The starting and ending value are seperated by ` --> `.
### Unoffical features
Some basic HTML tags are supported by some viewers, such as:
* `<b>bold</b>` **bold**
* `<i>italics</i>` *italics*
* `<u></u>` underlined
* `<font color="#0000FF">Blue</font>` colored text.

Note these will be displayed verbatim on viewers that don't support these features.
## Example
```
1
00:00:00,000 --> 00:01:00,000
This subtitle will be visible for the first minute of the stream

2
00:01:00,000 --> 00:01:30,000
and this one for thirty seconds after that.
```


title: SubStation Alpha

description: The SubStation Alpha subtitle format.

text: Overview

anchor: overview

text: Format

text: ASS (SSA v4+) header

text: Further reading:

anchor: further-reading

markdown:
SubStation Alpha (SSA), also known as Advanced Substation Alpha (ASS) for v4+, is a subtitle format. It was originally used by the Windows program of the same name, aimed at the karaoke and anime communities. It's advanced styling compared to alternatives made it popular with release groups.

# Overview
SSA is a "plain" text format, with Unicode support in ASS. It can either be left as a plain text file with the extension of .ssa or .ass, or muxed into a Matroska (.mkv) or AVI (.avi) file. The original SubStation Alpha software is abandonware, however a wide variety of media authoring, muxing, and playing software supports SSA, including [VLC](/tools/video-players/), [MPV](/tools/video-players/), and [FFmpeg](/tools/utilities/ffmpeg/).

## Format
SSA uses the word "script" to refer to the subtitles that track a video. The character ';' at the beginning of a line is used to mark comments.
### ASS (SSA v4+) header
```
[Script Info]
; This is an Advanced Sub Station Alpha v4+ script.
; For Sub Station Alpha info and downloads,
; go to http://www.eswat.demon.co.uk/
; or email kotus@eswat.demon.co.uk
;
; Advanced Sub Station Alpha script format developed by #Anime-Fansubs@EfNET
; http://www.anime-fansubs.org
;
; For additional info and downloads go to http://vobsub.edensrising.com/
; or email gabest@freemail.hu
;
; Note: This file was saved by Subresync.
;
ScriptType: v4.00+
Collisions: Normal
PlayResX: 384
PlayResY: 288
Timer: 100.0000

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Default,Tahoma,16,&H00000000,&H00ffffff,&H00ffffff,&H00c0c0c0,-1,0,0,0,100,100,0,0.00,1,2,3,2,20,20,20,1

[Events]
Format: Layer, Start, End, Style, Actor, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:01:41.70,0:01:46.84,Default,,0000,0000,0000,,Le rugissement des larmes !\NTu es mon ami.
Dialogue: 0,0:02:00.99,0:02:02.87,Default,,0000,0000,0000,,Est-ce vraiment Naruto ?
```

## Further reading:
[Multimedia wiki](https://wiki.multimedia.cx/index.php/SubStation_Alpha)

[Specification](http://moodub.free.fr/video/ass-specs.doc)

[Archived original software release](https://web.archive.org/web/20030603235926/http://www.eswat.demon.co.uk/substation.html)


title: WebVTT

description: WebVTT, or Web Video Text Tracks, is the format for subtitles on the web. It is used with the HTML \ element, or embedded into a webm container.

text: Structure

anchor: structure

text: Cue

anchor: cue

text: Example

markdown:
WebVTT, or Web Video Text Tracks, is the format for subtitles on the web. It is used with the [HTML \<track\> element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/track), or embedded into a webm container.

## Structure
WebVTT is a simple, text-based format, based on [SRT](/codecs/subtitles/srt/). All files start with the string
`WEBVTT`, optionally, some text, then two new lines. That's where the
data we're interested in starts.

### Cue

A WebVTT file is basically a bunch of cues. They can have a line with
an ID, then they have to have a line specifying from where to where
the cue should be displayed like this: `STARTTIME -> ENDTIME [optional
settings go here]`, then all the text to be displayed goes after
it. That text can have some HTML-like formatting in it. To learn about
them, see [the documentation](https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API#webvtt_cues).

## Example
```
WEBVTT

00:01.000 --> 00:04.000
- Never drink liquid nitrogen.

00:05.000 --> 00:09.000
- It will perforate your stomach.
- You could die.
```


title: AV1

description: AV1 is a royalty-free video compression format designed to succeed VP9. It presently competes with VP9, VVC, and HEVC. AV1 is computationally more complex than VP9, but is fast to decode due to the mature and efficient d

text: A Technical Overview of AV1

text: Abstract

anchor: abstract

text: Introduction

anchor: introduction

text: Profile & Levels

anchor: profile-levels

text: Block Structure

anchor: block-structure

text: Basic Coding block

anchor: basic-coding-block

text: Basic Prediction Block

anchor: basic-prediction-block

text: Basic Transform Block

anchor: basic-transform-block

text: Intra Prediction

anchor: intra-prediction

text: Directional Intra Prediction Mode

anchor: directional-intra-prediction-mode

text: Smooth Mode

anchor: smooth-mode

text: Paeth Mode

anchor: paeth-mode

text: Palette Mode

anchor: palette-mode

text: Filter Intra mode

anchor: filter-intra-mode

text: Intra Block Copy Mode

anchor: intra-block-copy-mode

text: Predict Chroma from Luma

anchor: predict-chroma-from-luma

text: Inter Prediction

anchor: inter-prediction

text: Affine/Warped Motion Compensation

anchor: affinewarped-motion-compensation

text: Sub-pixel Interpolation Filter

anchor: sub-pixel-interpolation-filter

text: Dynamic MV reference

anchor: dynamic-mv-reference

text: Extended Compound Modes

anchor: extended-compound-modes

text: Extended Reference frame Number

anchor: extended-reference-frame-number

text: In-loop Filter

anchor: in-loop-filter

text: De-blocking filter

anchor: de-blocking-filter

text: LR (In-loop Restoration) filter

anchor: lr-in-loop-restoration-filter

text: Multi-Symbol Entropy Coder

anchor: multi-symbol-entropy-coder

text: Transform

anchor: transform

text: Transform type

anchor: transform-type

text: Transform Block Shape and Size

anchor: transform-block-shape-and-size

text: Tiles

anchor: tiles

text: Segment

anchor: segment

text: SVC (Scalable Video Coding)

anchor: svc-scalable-video-coding

text: Other Tools

anchor: other-tools

text: Quantization Matrices

anchor: quantization-matrices

text: Superblock Delta-quantization

anchor: superblock-delta-quantization

text: OBU (Open Bitstream Unit)

anchor: obu-open-bitstream-unit

text: References

anchor: references

markdown:
AV1 is a royalty-free video compression format designed to succeed [VP9](/codecs/video/vp9/). It presently competes with [VP9](/codecs/video/vp9/), [VVC](/codecs/video/vvc/), and [HEVC](/codecs/video/hevc/). AV1 is computationally more complex than VP9, but is fast to decode due to the mature and efficient dav1d AV1 decoder. AV1 hardware accelerated decoding is also available on a variety of different consumer hardware devices, all of which are enumerated [on Wikipedia](https://en.wikipedia.org/wiki/AV1#Hardware). Standout entries include modern Intel, AMD, & Nvidia integrated & discrete GPUs, Google's Tensor SoC powering the Pixel line, Apple's A17 Pro in the iPhone 15 Pro series, and modern Mediatek & Qualcomm chips. YouTube is currently in the process of transitioning their videos to use AV1.

There are a number of viable AV1 encoding solutions available today. The three best, most ubiquitous, and free implementations are [aomenc](/tools/software-encoders/aomenc/), [SVT-AV1](/tools/software-encoders/svt-av1/), & [rav1e](/tools/software-encoders/rav1e/).

# A Technical Overview of AV1

*This section has been graciously borrowed from Qu Pengfei's amazing [AV1 README.md](https://github.com/QuPengfei/Technical-Overview-Of-AV1-Spec/blob/master/README.md), with some minor grammar, formatting, and spelling corrections. Thank you, Qu Pengfei!*

## Abstract

AV1 (AOMedia Video Codec 1.0) evolved on the basis of VP9 (Google), Thor (Cisco)
and Daala (Mozila) under the AOM (Alliance for Open Media). It includes a number
of enhancement and the new tools that have been added to improve the coding
efficiency. The new tools that are added so far include 4 main aspects:
prediction, transform, in-loop filter and entropy encoder. This document
provides a snapshot of the coding tools in the current finalized version (on
March, 2018) of AV1 spec.

## Introduction

According to the AOM web page, AV1 is designed with the following feature.

-   Royally free

-   Scales to any modern device at any bandwidth

-   For use in both commercial and non-commercial content, including
    user-generated content

-   Developed for the internet and related applications and services-from
    browsers and streaming to videoconferencing services

-   Designed with a low computational footprint and optimized for hardware

-   Bringing features like 4k UHD, HDR, and WCG to real-time video

## Profile & Levels

Profiles and levels specify restrictions on the capabilities needed to decode
the bitstreams. The profile specifies the bit depth and subsampling formats
supported, while the level defines resolution and performance characteristics.
By now levels is still under discussion and there is no more details.

AV1 support the three named profiles as the table list.

| Profile | Bit depth | Monochrome support | Chroma subsampling  | Name         |
|---------|-----------|--------------------|---------------------|--------------|
| 0       | 8/10      | Yes                | 4:2:0               | Main         |
| 1       | 8/10      | No                 | 4:4:4               | High         |
| 2       | 8/10      | Yes                | 4:2:2               | Professional |
| 2       | 12        | Yes                | 4:2:0, 4:2:2, 4:4:4 | Professional |

Table 1. AV1 Profile

## Block Structure

### Basic Coding block

AV1 support the larger super block size, which is up to 128x128 super block is
allowed. It supports from 128x128 down to 4x4 coding block. Each 4x4 luma block
is allowed to independently select inter or intra mode, its reference mode, and
interpolation filter type. For Chroma, 2x2 block size is allowed but still 4x4
transform block size is used.

### Basic Prediction Block

AV1 support up to 10 partition type. The size of partition unit is allowed down
to 4x4 and totally there are 24 types of block size.

| Partition index | Type of partition |
|-----------------|-------------------|
| 0               | PARTITION_NONE    |
| 1               | PARTITION_HORZ    |
| 2               | PARTITION_VERT    |
| 3               | PARTITION_SPLIT   |
| 4               | PARTITION_HORZ_A  |
| 5               | PARTITION_HORZ_B  |
| 6               | PARTITION_VERT_A  |
| 7               | PARTITION_VERT_B  |
| 8               | PARTITION_HORZ_4  |
| 9               | PARTITION_VERT_4  |

Table 2. Type of Block partition

| Index | Partition Block size | Index | Partition Block size |
|-------|----------------------|-------|----------------------|
| 0     | BLOCK_4X4            | 12    | BLOCK_64X64          |
| 1     | BLOCK_4X8            | 13    | BLOCK_64X128         |
| 2     | BLOCK_8X4            | 14    | BLOCK_128X64         |
| 3     | BLOCK_8X8            | 15    | BLOCK_128X128        |
| 4     | BLOCK_8X16           | 16    | BLOCK_4X16           |
| 5     | BLOCK_16X8           | 17    | BLOCK_16X4           |
| 6     | BLOCK_16X16          | 18    | BLOCK_8X32           |
| 7     | BLOCK_16X32          | 19    | BLOCK_32X8           |
| 8     | BLOCK_32X16          | 20    | BLOCK_16X64          |
| 9     | BLOCK_32X32          | 21    | BLOCK_64X16          |
| 10    | BLOCK_32X64          | 22    | BLOCK_32X128         |
| 11    | BLOCK_64X32          | 23    | BLOCK_128X32         |

Table 3. Size of Block Partition

### Basic Transform Block

Both square and rectangle transform block size is supported in AV1. There are
total 19 transform block size.

| Index | TxSize   | Index | TxSize   |
|-------|----------|-------|----------|
| 0     | TX_4X4   | 10    | TX_32X16 |
| 1     | TX_8X8   | 11    | TX_32X64 |
| 2     | TX_16X16 | 12    | TX_64X32 |
| 3     | TX_32X32 | 13    | TX_4X16  |
| 4     | TX_64X64 | 14    | TX_16X4  |
| 5     | TX_4X8   | 15    | TX_8X32  |
| 6     | TX_8X4   | 16    | TX_32X8  |
| 7     | TX_8X16  | 17    | TX_16X64 |
| 8     | TX_16X8  | 18    | TX_64X16 |
| 9     | TX_16X32 |       |          |

>   Table 4. Size of Transform Block

## Intra Prediction

Intra Prediction in AV1 expends largely compared to VP9. Here is snapshot of
Intra Mode.

| Index | Intra mode          | AV1 | VP9 | Comments                                      |
|-------|---------------------|-----|-----|-----------------------------------------------|
| 0     | DC_PRED             | X   | X   |                                               |
| 1     | V_PRED              | X   | X   | AV1 support 7 kind of mode based on this mode |
| 2     | H_PRED              | X   | X   | AV1 support 7 kind of mode based on this mode |
| 3     | D45_PRED            | X   | X   | AV1 support 7 kind of mode based on this mode |
| 4     | D135_PRED           | X   | X   | AV1 support 7 kind of mode based on this mode |
| 5     | D113_PRED           | X   | X   | AV1 support 7 kind of mode based on this mode |
| 6     | D157_PRED           | X   | X   | AV1 support 7 kind of mode based on this mode |
| 7     | D203_PRED           | X   | X   | AV1 support 7 kind of mode based on this mode |
| 8     | D67_PRED            | X   | X   | AV1 support 7 kind of mode based on this mode |
| 9     | SMOOTH_PRED         | X   |     |                                               |
| 10    | SMOOTH_V_PRED       | X   |     |                                               |
| 11    | SMOOTH_H_PRED       | X   |     |                                               |
| 12    | TM_PRED(PAETH_PRED) | X   | X   | AV1 replace TM_PRED with PAETH_PRED           |
| 13    | Palette Mode        | X   |     |                                               |

Table 5. Summary of Intra Mode between AV1 and VP9

### Directional Intra Prediction Mode

VP9 only supports 8 directional intra prediction modes: D45_PRED, D63_PRED,
H_PRED, D117_PRED, D135_PRED, D153_PRED, V_PRED, D207_PRED. These modes
correspond to prediction angles of 45, 63, 90, 117, 135, 153, 180, and 207
degrees, respectively.

To improve intra coding efficiency, more prediction angle options are added to
AV1. The prediction angle is calculated as the following:

Prediction angle = nominal_angle + (angle_delta \* angle_step),

| nominal_angle                       | angle_step | angle_delta | Total number of angles |
|-------------------------------------|------------|-------------|------------------------|
| 45, 63, 90, 117, 135, 153, 180, 207 | 3          | [-3, +3]    | 8\*7=56                |

Table 6. Finer of Intra Mode

-   norminal_angle is determined by the prediction mode, and is the same as VP9;

-   angle_delta is in a predefined range and angle_step is a predefined value.
    In current configuration, angle_delta is in the range of [-3, +3] and
    angle_step is 3. These settings are selected experimentally.

-   The total number of supported prediction angles is therefore increased from
    8 to 8 \* 7 = 56.

### Smooth Mode

It is a Non- Directional Intra Prediction mode. VP9 has 2 non-directional intra
prediction modes: DC_PRED and TM_PRED. AV1 expands on this by adding 3 new
smooth prediction modes: SMOOTH_PRED, SMOOTH_V_PRED and SMOOTH_H_PRED. The new
modes work as follows:

|Mode|Comments|
|-|-|
| SMOOTH_PRED   | Useful for predicting blocks that have a smooth gradient. It works as follows: estimate the pixels on the rightmost column with the value of the last pixel in top row, and estimate the pixels in the last row of the current block using the last pixel in left column. Then calculate the rest of the pixels by an average of quadratic interpolation in vertical and horizontal directions, based on distance of the pixel from the predicted pixels. |
| SMOOTH_V_PRED | Similar to SMOOTH_PRED, but uses quadratic interpolation only in the vertical direction                                                                                |
| SMOOTH_H_PRED | Similar to SMOOTH_PRED, but uses quadratic interpolation only in the horizontal direction                                                                                |

Table 7. Smooth mode of Intra mode

### Paeth Mode

It is a Non- Directional Intra Prediction mode. The new prediction mode
PAETH_PRED replaces the existing mode TM_PRED.

TM_PRED: Predictor(TM) = left + top – top_left

PAETH_PRED: Predictor (PAETH) = argmin \|x- Predictor(TM)\|

The idea is to find out the One of left, top, top_left closest in value to
Predictor(TM).

### Palette Mode

Sometimes, given intra block can be approximated by a block with small number of
unique colors. This is especially true for artificial videos like
screen-capture, games etc. For such cases, AV1 introduces a new intra coding
mode called palette mode. This predictor for a block is signaled by storing (i)
a color palette, with 2 to 8 colors, and (ii) color indices into the palette for
all pixels in the block. The residual pixel values of the block are as usual
transformed and quantized before being entropy-coded.

Palette mode can be used by both intra-only as well as inter frames. The number
of base colors determines the trade-off between fidelity and compactness. The
color indices for pixels are obtained by the nearest neighbor method. The color
indices are encoded using the neighborhood-based context to be as compact as
possible.

Palette Mode is not new. We can see the Palette Mode and Intra block copy in the
HEVC SCC (Screen Content Coding) extension.

### Filter Intra mode

AV1 adopt the new mode to interpolate (intra filter) the reference samples
before prediction. This will reduce the impact of quantization noise. Here is
the table to specify the type of intra filtering.

| Index | Filter intra type |
|-------|-------------------|
| 0     | INTRA_DC_PRED     |
| 1     | INTRA_V_PRED      |
| 2     | INTRA_H_PRED      |
| 3     | INTRA_D153_PRED   |
| 4     | INTRA_TM_PRED     |

Table 8 Type of Intra filter Mode

### Intra Block Copy Mode

This tool is very efficient for coding of screen content video in that repeated
patterns in text and graphics rich content occur frequently within the same
picture. Having a previously reconstructed block with equal or similar pattern
as a predictor can effectively reduce the prediction error and therefore improve
coding efficiency.

In AV1, Intra block copy is only allowed in intra frames. It disables all loop
filtering and only integer offsets are allowed in block copy mode.

### Predict Chroma from Luma

Chroma from luma (CfL) prediction is a new and promising chroma-only intra
predictor that models chroma pixels as a linear function of the coincident
reconstructed luma pixels.

## Inter Prediction

### Affine/Warped Motion Compensation

Traditional modern codecs, including VP9, use block motion compensation where
motion vectors are translational only. This is not sufficient for real video
which often contains complex motion. For example, motion due to camera shake,
panning and zoom might require transformations that support shearing, scaling,
rotation and changes in aspect ratio. In AV1, we introduce warped motion
compensation implemented as similarity and affine transformations to better
capture the diversity of motion that exists in real video. There are two
affine/warped motion compensation.

| Affine Motion Compensation | Comments                                                                                |
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Global                     | It is common for videos to contain a global camera motion which is pertinent to an entire inter frame. It is therefore beneficial to transmit a set of motion parameters at the frame level that is applicable to a large number of blocks in the frame. When a frame is encoded, a set of global motion parameters is computed and transmitted between that frame and each reference frame. These parameters may be either translational, similarity or affine motion model. Subsequently, any block in the frame can signal use of the global motion mode with a given reference to create a suitable predictor. |
| Local                      | Affine motion compensation is also useful to describe complex local object motion. Here, we estimate affine parameters for a single block using the translational motion vectors that are typically conveyed for all inter blocks. Specifically, we estimate an affine or similarity model using the motion vectors from the current block and its causal neighbors which share the same reference frame.                                                                                |

Table 9 Affine Motion Compensation

### OBMC (Overlapped Block Motion Compensation)

Motions assigned to surrounding blocks will contribute to predicting a current
block, via a well-defined overlapping scheme appropriately designed for advanced
variable block-size partitioning frameworks.

The OBMC will blend multiple predictors from neighbor blocks. It is not new
concept and was proposed and implemented back in the era of h.263. The OBMC was
proved to largely reduce prediction errors but not adopted by recent codecs due
to extra complexity in the scenario of hybrid inter/intra variable block size
coding. In AV1, a practical overlapping mechanism based on two-stage 1-D
filtering is proposed for the advanced partitioning framework to implement
causal overlapped block prediction.

### Sub-pixel Interpolation Filter

The motion vector used in modern video codecs is allowed to have a fractional
position for a better prediction quality. So, an interpolation filter module is
needed to generate the prediction block at a fractional position in the
reference frame. VP9 codec uses a separable interpolation filter to perform
inter prediction with ⅛ motion vector precision. Three filter types, SHARP,
REGULAR and SMOOTH, in descending order of cutoff frequencies, are provided to
deal with various types of noise/distortions that can occur in reference
frames/blocks. Given a filter type and a motion vector, the interpolation filter
is performed by two one-dimensional filters, one for horizontal direction and
one for vertical direction.

In AV1 codec, dual interpolation filter is introduced on top of the
interpolation module inherited from VP9. Dual filter allows each block/frame to
use a different interpolation filter type in horizontal and vertical direction.
Up to 9 types of filter will be applied to the block.

This idea is based on the observation that a reference frame/block’s horizontal
and vertical signals may have distinct frequency characteristics; therefore,
using different filter types may produce a better prediction. As before, both
the filter types are transmitted in the bitstream on a per block or per frame
basis.

At the same time AV1 use the high intermediate precision between the horizontal
and vertical filter. The same high precision before average the predictors with
compound mode.

### Dynamic MV reference

VP9 has two candidates MV in the ref list and 4 type of mode (NEARESTMV, NEARMV,
NEWMV, and ZEROMV) are used. AV1 support 4 candidate MV and more modes.

For single ref mode, AV1 is same as VP9.

For compound mode, VP9 restricts motion vectors for a compound predictor to
share one motion vector referencing mode, even though they may use different
reference frames. To add more flexibility, on top of existing four combinations
(NEAREST_NEARESTMV, NEAR_NEARMV, NEW_NEWMV, ZERO_ZEROMV) in VP9, AV1 supports
four more empirically selected combinations: NEAREST_NEWMV, NEW_NEARESTMV,
NEAR_NEWMV, and NEW_NEARMV.

| Index | Type                         | Ref Mode        |
|-------|------------------------------|-----------------|
| 0     | NEARESTMV                    | single ref mode |
| 1     | NEARMV                       | single ref mode |
| 2     | GLOBALMV(ZEROMV)             | single ref mode |
| 3     | NEWMV                        | single ref mode |
| 4     | NEAREST_NEARESTMV            | compound mode   |
| 5     | NEAR_NEARMV                  | compound mode   |
| 6     | NEAREST_NEWMV                | compound mode   |
| 7     | NEW_NEARESTMV                | compound mode   |
| 8     | NEAR_NEWMV                   | compound mode   |
| 9     | NEW_NEARMV                   | compound mode   |
| 10    | GLOBAL_GLOBALMV(ZERO_ZEROMV) | compound mode   |
| 11    | NEW_NEWMV                    | compound mode   |

Table 10 MV mode

### Extended Compound Modes

AV1 Compound mode support both predictors from the same direction and VP9 only
support from the different direction (One forward and one backward reference
frame). VP9 only support 1/2 weight to blend the two predictor and AV1 support
more flexible weight blending.

| Index | Compound type     | Comments                                                                                |
|-------|-------------------|---------------------------------------------------------------------------------------------------------------------------------------|
| 0     | COMPOUND_WEDGE    | Inter-Inter Wedge mode Inter-Intra Wedge mode                                                                                |
| 1     | COMPOUND_SEG      | Inter-Inter Compound Segment mode                                                                                |
| 2     | COMPOUND_AVERAGE  | (1/2,1/2) weight will be applied to blend the predictors                                                                              |
| 3     | COMPOUND_INTRA    | Inter-Intra Gradual mode                                                                                |
| 4     | COMPOUND_DISTANCE | This process computes weights to be used for blending predictions together based on the expected output times of the reference frames |

Table 11. Compound type

Here are more details about the Compound Segment Mode:

-   Inter-Inter Compound Segment mode

In many cases, regions in one predictor will contain useful content that is not
present in the other. The two inter predictors have a larger pixel difference
generally.

-   Inter-Inter Wedge mode

Boundaries of moving objects in a video often separate two regions with distinct
motions. Coding these regions with separate motion vector reference combinations
should be beneficial; however, finding exact object boundaries is not only
difficult, but expensive to communicate in the bitstream. Our approach is to
design a codebook of masks with only a few possible partitioning combinations
and signaling the codebook index in the bitstream.

The AV1 wedge codebook contains partition orientations that are either
horizontal, vertical or oblique with slopes: 2, -2, 0.5 and -0.5. The wedge
prediction mode is used for all square and rectangular blocks, using the 16-ary
shape codebooks.

| Index | Wedge direction  | Comments |
|-------|------------------|----------|
| 0     | WEDGE_HORIZONTAL |          |
| 1     | WEDGE_VERTICAL   |          |
| 2     | WEDGE_OBLIQUE27  |          |
| 3     | WEDGE_OBLIQUE63  |          |
| 4     | WEDGE_OBLIQUE117 |          |
| 5     | WEDGE_OBLIQUE153 |          |

Table 12. Wedge direction

-   Inter-Intra Gradual mode

Decay the weight gradually for the intra from the prediction boundary and
increase the weight of inter correspondingly. It support four modes, which
include horizontal mode, vertical mode, DC_PRED, and SMOOTH_PRED.

-   Inter-Intra Wedge mode

Blocks cannot always perfectly partition moving objects. For example, occlusion
can occur in the middle of a block, it is better to apply different prediction
techniques to different contents. Contents that are not occluded in reference
frame will prefer inter prediction, while newly revealed content could benefit
more from intra prediction using local reference.

### Extended Reference frame Number

Up to 7 reference frames out of 8 in the frame stored buffer are extended to be
used in the inter mode. The reference frames is allowed to come from the same
side or different side in the AV1.

LAST3_FRAME, LAST2_FRAME and LAST_FRAME are forward references and LAST_FRAME is
the near past frame. BWDREF_FRAME is a backward reference, similar to
ALTREF_FRAME.

Here is the table to show the reference frame type.

| Index | Ref frame Name |
|-------|----------------|
| 0     | INTRA_FRAME    |
| 1     | LAST_FRAME     |
| 2     | LAST2_FRAME    |
| 3     | LAST3_FRAME    |
| 4     | GOLDEN_FRAME   |
| 5     | BWDREF_FRAME   |
| 6     | ALTREF2_FRAME  |
| 7     | ALTREF_FRAME   |

Table 13 Reference frame type

## In-loop Filter

Several in-loop tools in AV1 are employed. De-blocking, CDEF and loop
restoration are cascaded.

### De-blocking filter

AV1 support 4 filter levels per frame and VP9 only has one. Two levels are for
Luma component (horizontal and vertical levels). The other two levels are for U
and V component separately. In AV1, filter level is allowed to change superblock
by superblock.

### CDEF (Constrained Directional Enhancement Filter)

CDEF is the combination of CLPF (Constrained Low Pass Filter) and Deringing
filter. The main goal of the in-loop CEDF is to filter the coding artifacts and
ringing while preserving the detail of image. It takes into account the
direction of edge and patterns in the image. It is the similar to the SAO of
HEVC.

The CDEF is based on the following observation. The amount of ringing artifacts
in a coded image tends to be roughly proportional to the quantization step size.
The amount of detail is a property of the input image, but the smallest detail
actually retained in the quantized image tends to also be proportional to the
quantization step size. For a given quantization step size, the amplitude of the
ringing is generally less than the amplitude of the details.

CDEF works as the following steps:

-   The frame is divided into filter blocks of 64x64 pixels. Some CDEF
    parameters are signaled at the frame level, and some may be signaled at the
    filter block level.

-   To identify the direction of edge or pattern in each filter block.

-   To adaptively filter along the identified direction and to a lesser degree
    along directions rotated 45 degrees from the identified direction. The
    filter strengths are signaled explicitly, which allows a high degree of
    control over the blurring.

The main reason for identifying the direction is to align the filter taps along
that direction to reduce ringing while preserving the directional edges or
patterns. CDEF defines primary taps and secondary taps filter. The primary taps
follow the direction and the secondary taps form a cross, oriented 45 off the
direction. Both primary and secondary taps filter have 8 types.

### LR (In-loop Restoration) filter

AV1 employ a set of in-loop image restoration tool after de-blocking to
generally de-noise and enhance the quality of the edge. In-loop restoration
scheme have two types of filter to remove blur artifacts due to block
processing. One is Wiener Filter. The other is Dual Self-Guided filter. These
tools are integrated into AV1 with a switchable framework, which trigger the
different tool in the different image region.

## Multi-Symbol Entropy Coder

Multi-symbol adaptive arithmetic coding model is adopted in AV1. Both syntax
element and coefficient are coded with this model.

Most recent video codecs encode information using binary arithmetic coding, such
as CABA or CAVLC in AVC/HEVC, meaning that each symbol can only take two values.
The AV1 entropy encoder come from the Daala range coder and supports up to 16
values per symbol, making it possible to encode fewer symbols. This is
equivalent to coding up to four binary values in parallel and reduces serial
dependencies, allowing hardware implementations to use lower clock rates, and
thus less power.

## Transform

### Transform type

For AV1, there is a richer set of transforms for coding Inter and Intra
prediction residues. Inter prediction residues do not have a well-defined
structure as in the Intra case, but using a bank of transforms, each adapted to
a specific type of residue profile within the block, is generally helpful.

In AV1, four types of transform are used mainly in the horizontal and vertical
direction separately. The total 16 different transforms are available.

| Transform type | Comments                                                                                |
|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
| DCT            | Inter and Intra modes continue to make use of DCT.                                                                                |
| ADST           | Asymmetric Discrete Sine Transform                                                                                |
| Flip ADST      | It applies ADST in reverse order                                                                                |
| IDTX           | Identity transform seems to be particularly useful for coding residue with sharp lines and edges. Identity transform is useful for screen content coding |

Table 14 The Main Transform Type in each of direction

For each small coded block (4x4 or 8x8), it is possible to choose one of up to
16 different transforms as follows(Detail in Table):

\{DCT, ADST, FlipADST, IDTX\} horizontal x \{DCT, ADST, FlipADST, IDTX\} vertical

As block sizes get larger, some of these transforms begin to act similarly.
Thus, a reduced set of transforms is used for 16x16, 32x32 and 64x64 block
sizes. In the transform selection process for Inter and Intra modes, the encoder
does a search over the entire set of transforms and selects the one that
produces the best rate-distortion cost. Once a transform is selected, a
transform type symbol from the set of types available at that size is used to
indicate the actual transform used in the bitstream.

There are 6 types of transform sets in the AV1 spec, which specify the transform
type of Intra and Inter blocks. The transform sets determine what subset of
transform types can be used, according to the following table.

| Inter or not | Set Number | Transform set  |
|--------------|------------|----------------|
| Don't care   | 0          | TX_SET_DCTONLY |
| 0            | 1          | TX_SET_INTRA_1 |
| 0            | 2          | TX_SET_INTRA_2 |
| 1            | 1          | TX_SET_INTER_1 |
| 1            | 2          | TX_SET_INTER_2 |
| 1            | 3          | TX_SET_INTER_3 |

Table 15 Transform Set in the AV1 spec

| Transform type    | TX_SET_DCTONLY | TX_SET_INTRA_1 | TX_SET_INTRA_2 | TX_SET_INTER_1 | TX_SET_INTER_2 |
|-------------------|----------------|----------------|----------------|----------------|----------------|
| DCT_DCT           | X              | X              | X              | X              | X              |
| ADST_DCT          |                | X              | X              | X              | X              |
| DCT_ADST          |                | X              | X              | X              | X              |
| ADST_ADST         |                | X              | X              | X              | X              |
| FLIPADST_DCT      |                |                |                | X              | X              |
| DCT_FLIPADST      |                |                |                | X              | X              |
| FLIPADST_FLIPADST |                |                |                | X              | X              |
| ADST_FLIPADST     |                |                |                | X              | X              |
| FLIPADST_ADST     |                |                |                | X              | X              |
| IDTX              |                | X              | X              | X              | X              |
| V_DCT             |                | X              |                | X              | X              |
| H_DCT             |                | X              |                | X              | X              |
| V_ADST            |                |                |                | X              |                |
| H_ADST            |                |                |                | X              |                |
| V_FLIPADST        |                |                |                | X              |                |
| H_FLIPADST        |                |                |                | X              |                |

Table 16 Detailed Transform type supported in each transform set.

### Transform Block Shape and Size

Both square and rectangle shape block are used in AV1. The transform block size
is less than the partition block size. The block size is very flexible and up to
64x64 and down to 4x4. Details see the table in the Block section.

## Tiles

AV1 support flexible tiles, which include uniform and non-uniform tile spacing.
Tile area is limited to a maximum 4096x2304. Tiles can be grouped into tile
group and each group can be decoded independently to achieve error resilience.
Loop filter can be enabled or disabled across tiles.

## Segment

Same as VP9, AV1 provides a means of segmenting the image and then applying
various adjustments at the segment level. Up to 8 segments may be specified for
any given frame. For each of these segments it is possible to specify:

- A quantizer (absolute value or delta).

- A loop filter strength (absolute value or delta).

- A prediction reference frame.

- A block skip mode that implies both the use of a (0,0) motion vector and that
no residual will be coded.

## SVC (Scalable Video Coding)

AV1 support temporal and spatial layer coding. Temporal layer support up to 8
layers and spatial layer support up to 3 layers.

| Index | Scalability mode | Index  | Scalability mode  |
|-------|------------------|--------|-------------------|
| 0     | SCALABILITY_L1T2 | 8      | SCALABILITY_L2T2h |
| 1     | SCALABILITY_L1T3 | 9      | SCALABILITY_L2T3h |
| 2     | SCALABILITY_L2T1 | 10     | SCALABILITY_S2T1h |
| 3     | SCALABILITY_L2T2 | 11     | SCALABILITY_S2T2h |
| 4     | SCALABILITY_L2T3 | 12     | SCALABILITY_S2T3h |
| 5     | SCALABILITY_S2T1 | 13     | SCALABILITY_SS    |
| 6     | SCALABILITY_S2T2 | 14-255 | reserved          |
| 7     | SCALABILITY_S2T3 |        |                   |

Table 17. Temporal and Spatial Mode

| Scalability mode  | Spatial Layers | Resolution Ratio | Temporal Layers | Inter-layer-dependency |
|-------------------|----------------|------------------|-----------------|------------------------|
| SCALABILITY_L1T2  | 1              |                  | 2               |                        |
| SCALABILITY_L1T3  | 1              |                  | 3               |                        |
| SCALABILITY_L2T1  | 2              | 2:1              | 1               | Yes                    |
| SCALABILITY_L2T2  | 2              | 2:1              | 2               | Yes                    |
| SCALABILITY_L2T3  | 2              | 2:1              | 3               | Yes                    |
| SCALABILITY_S2T1  | 2              | 2:1              | 1               | No                     |
| SCALABILITY_S2T2  | 2              | 2:1              | 2               | No                     |
| SCALABILITY_S2T3  | 2              | 2:1              | 3               | No                     |
| SCALABILITY_L2T2h | 2              | 1.5:1            | 2               | Yes                    |
| SCALABILITY_L2T3h | 2              | 1.5:1            | 3               | Yes                    |
| SCALABILITY_S2T1h | 2              | 1.5:1            | 1               | No                     |

Table 17. Details in the Temporal and Spatial Mode

## Other Tools

### Quantization Matrices

AV1 support 15 sets of QMs, which are based on the contrast-sensitive functions.
QMs are applied to a frame based on selectable scaling of its quantization
level, higher level of quantization imply flatter matrices. The matrices become
flatter as the quantization index value increases (and the quality decreases).
Inter matrices are slightly flatter than intra matrices.

### Superblock Delta-quantization

AV1 allow the per-superblock changes in quantization parameter to support
sub-frame rate control. At the same time it support the ROI level rate control
on the top of segmentation level parameter.

### OBU (Open Bitstream Unit)

An AV1 bitstream consists of a number of OBUs that are normally held within a
container format alongside audio and timing information. Here the new tool OBU
is introduced in AV1 and it is similar to NAL (Network Abstract Layer) in
AVC/HEVC spec.

The OBU header is similar to the NAL header. In general the total 8 bits are
presented. The OBU extra 8 bits of extension header is used if temporal and
spatial layer exist in the bitstream. obu_type is the most important syntax to
describe the type of OBU .

| Index | obu_type                   |
|-------|----------------------------|
| 0     | Reserved                   |
| 1     | OBU_SEQUENCE_HEADER        |
| 2     | OBU_TD                     |
| 3     | OBU_FRAME_HEADER           |
| 4     | OBU_TILE_GROUP             |
| 5     | OBU_METADATA               |
| 6     | OBU_FRAME                  |
| 7     | OBU_REDUNDANT_FRAME_HEADER |
| 8-14  | Reserved                   |
| 15    | OBU_PADDING                |

Table 18. Type of OBU

## References

1.  https://aomediacodec.github.io/av1-spec/av1-spec.pdf


title: AVC / H.264

description: H.264, also known as AVC , is a video compression standard that has played a significant role in multimedia codec technology. H.264 revolutionized video encoding by offering vastly more efficient compression than predece

markdown:
# H.264

> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
H.264, also known as AVC (Advanced Video Coding), is a video compression standard that has played a significant role in multimedia codec technology. H.264 revolutionized video encoding by offering vastly more efficient compression than predecessors, and has been the nearly universal defacto video compression standard since. Its history is marked by continuous refinement and widespread adoption, especially as it pertains to the highly optimized [x264](/tools/software-encoders/x264/) video encoder. Despite being over a decade old, H.264 remains relevant today. Newer codecs like [H.265](/codecs/video/hevc/), [VP9](/codecs/video/vp9/), and [AV1](/codecs/video/av1/) aim to provide more efficient compression than H.264 but are currently not as universal. The choice between these codecs largely depends on the specific requirements of the application, compatibility and support, and the balance between compression efficiency and computational complexity.


title: AVS3

description: AVS3 is the most recent codec from the AVS family. It aims to bring a significant coding improvement over AVS2 and HEVC. The specification was divided into 2 phases, the first one finished in 2019 and the second in 2021.

markdown:
> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
AVS3 is the most recent codec from the AVS family. It aims to bring a significant coding improvement over AVS2 and [HEVC](/codecs/video/hevc/). The specification was divided into 2 phases, the first one (main profile) finished in 2019 and the second (high profile) in 2021.

AVS3 has more flexible coding tree and block partitioning shemes compared to previous generation AVS codecs. It also employs new DCT-VII and DST-VII transforms. The work on AVS3 was sponsored by organizations including Peking University, Pengcheng Laboratory and Huawei.

In 2022 it was [added](https://dvb.org/news/dvb-test-content-for-vvc-and-avs3-codecs-now-available) to worldwide DVB standard alongside [VVC](/codecs/video/vvc/). As of 2024 there several hardware AVS3 decoders including Mediatek and HiSilicon. We should be seeing more AVS3 in hardware since it was added to DVB toolbox.

[uavs3](https://github.com/uavs3) is an open source high performance software encoder and decoder of AVS3-P2. The project was initialized by the Peking University Shenzhen Graduate School and over the time was optimized for x86 and arm processors. FFmpeg since version 6 supports only avs3 decoding via [uavs3d](https://github.com/uavs3/uavs3d).

Intel partnered with Boya and created [SVT-AVS3](https://www.intel.com/content/www/us/en/developer/articles/technical/scalable-video-technology.html) which to my knowledge is not available to the public. The only open-source encoder available to general public is [uavs3e](/tools/software-encoders/uavs3e/).