title: Machine-readable access

description: Stable Markdown, JSON, OpenAPI, and downloadable corpus interfaces for agents and independent clients.

text: Start with the index

anchor: start-with-the-index

text: Contracts and release detection

anchor: contracts-and-release-detection

text: Bulk and agent-oriented files

anchor: bulk-and-agent-oriented-files

text: Attribution and content status

anchor: attribution-and-content-status

markdown:
Encode Wiki publishes the same public content in deterministic, machine-readable forms. These
interfaces are generated during the static build and require no account or API key.

## Start with the index

[`/api/v1/index.json`](/api/v1/index.json) enumerates every public document and article. Each record
contains its stable ID, canonical page URL, clean Markdown and JSON retrieval URLs, status, tags,
dates, authors, source and license metadata, headings, outbound references, byte count, and SHA-256
checksum.

Retrieve an individual record using the URL in its `apiUrl` field. The corresponding `markdownUrl`
returns plain Markdown with a short attribution header. Do not construct URLs from titles; follow
the URLs in the index so nested identifiers remain opaque to clients.

## Contracts and release detection

- [`/api/v1/openapi.json`](/api/v1/openapi.json) describes the read-only API using OpenAPI 3.1.
- [`/api/v1/schema.json`](/api/v1/schema.json) is the JSON Schema for an individual corpus record.
- [`/api/v1/release.json`](/api/v1/release.json) reports the Git content version, deterministic
  timestamp, corpus checksum, counts, and knowledge-bundle checksums.

The API path changes only for breaking contract revisions. Compare `schemaVersion` before parsing
and `contentVersion` or `contentChecksum` to detect a new release. Page checksums cover cleaned
Markdown only.

## Bulk and agent-oriented files

- [`/llms.txt`](/llms.txt) is a compact curated map of high-value pages and complete interfaces.
- [`/llms-docs.txt`](/llms-docs.txt) and [`/llms-articles.txt`](/llms-articles.txt) contain full,
  cleaned collection text.
- [`/llms-full.txt`](/llms-full.txt) combines both collections and may be too large for smaller
  context windows.
- [`/api/v1/knowledge.ndjson.gz`](/api/v1/knowledge.ndjson.gz) is the complete corpus as
  gzip-compressed newline-delimited JSON for ingestion by future search, Ask AI, or MCP services.

All API and text responses permit cross-origin reads and use short shared-cache lifetimes. The
downloadable `.gz` file is an actual gzip payload, not an HTTP content-encoding wrapper. Its
uncompressed and compressed checksums are in the release record.

## Attribution and content status

Machine clients must retain each record's `canonicalUrl`, `license`, sources, provenance, and
modification notice when redistributing migrated material. Treat `needs-review` content as
potentially stale and `archived` content as historical. The API does not turn those records into
current recommendations.


collection: docs

title: Butteraugli

status: needs-review

license: CC BY-SA 4.0

text: Butteraugli

anchor: butteraugli

text: Installing

anchor: installing

text: Running

anchor: running

text: Using the standalone FFVship library

anchor: using-the-standalone-ffvship-library

text: In Vapoursynth scripts

anchor: in-vapoursynth-scripts

text: Scoring

anchor: scoring

contentFormat: markdown

markdown:
# Butteraugli

> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!
Butteraugli is a perceptual distance metric that estimates the psychovisual similarity of two images. It is capable of outputting a heatmap containing differences between two input clips.

While a [reference implementation by Google](https://github.com/google/butteraugli) exists,
most people will want to use the [HIP/CUDA implementation `vship`](https://codeberg.org/Line-fr/Vship).
Vship is intended for use in Vapoursynth scripts or as a standalone CLI tool without Vapoursynth and runs on GPU (Nvidia and AMD Radeon).

## Installing

The [Release page](https://codeberg.org/Line-fr/Vship/releases) of the repository provides up-to-date binaries for Windows users.

    #### Compiling

To compile Vship, follow the [instructions](https://codeberg.org/Line-fr/Vship#installation) on the repository.
    #### Archlinux AUR

On archlinux, you may use the [AUR](https://wiki.archlinux.org/title/Arch_User_Repository) to install.
    Simply use your favorite AUR helper to install `vapoursynth-plugin-vship-cuda-git` (Nvidia) or `vapoursynth-plugin-vship-amd-git` (AMD Radeon)

    ```bash
    paru -S vapoursynth-plugin-vship-amd-git
    ```

## Running

### Using the standalone FFVship library

```bash
usage: ./FFVship [-h] [--source SOURCE] [--encoded ENCODED]
                    [-m {SSIMULACRA2, Butteraugli}]
                    [--start start] [--end end] [-e --every every]
                    [-t THREADS] [-g gpuThreads] [--gpu-id gpu_id]
                    [--json OUTPUT]
                    [--list-gpu]
                    Specific to Butteraugli:
                    [--intensity-target Intensity(nits)]
```

### In Vapoursynth scripts

Example script:
```py
import vapoursynth as vs
core = vs.core

# Load reference and distorted clips
ref = core.bs.VideoSource("reference.mp4")
dist = core.bs.VideoSource("distorted.mp4")

# Calculate Butteraugli scores
# intensity_multiplier controls sensitivity
result = ref.vship.BUTTERAUGLI(dist, distmap=0, numStream = 4)

# Extract scores from frame properties (three different norms available)
scores_2norm = [frame.props["_BUTTERAUGLI_2Norm"] for frame in result.frames()]
scores_3norm = [frame.props["_BUTTERAUGLI_3Norm"] for frame in result.frames()]
scores_infnorm = [frame.props["_BUTTERAUGLI_INFNorm"] for frame in result.frames()]

# Get all scores in one pass
all_scores = [[frame.props["_BUTTERAUGLI_2Norm"],
               frame.props["_BUTTERAUGLI_3Norm"],
               frame.props["_BUTTERAUGLI_INFNorm"]]
              for frame in result.frames()]

# Print average scores
print(f"Average Butteraugli 3Norm distance: {sum(scores_3norm) / len(scores_3norm)})
print(f"Average Butteraugli 2Norm distance: {sum(scores_2norm) / len(scores_2norm)})
print(f"Average Butteraugli MaxNorm distance: {sum(scores_infnorm) / len(scores_infnorm)})
```

> **Tip — Graphical visualization.**
>
> You can optionally generate visual distortion maps with:
> ```py
> # Set distmap=1 to visualize distortion
> distmap_result = ref.vship.BUTTERAUGLI(dist, distmap=1)
>
> # The resulting clip is a grayscale visualization of distortions
> distmap_result.set_output()
> ```
## Scoring
Butteraugli outputs score in the range 0..inf with lower being better (closer to the source). A score below 1.0 is usually considered very good.


title: CVVDP

description: Codec Wiki reference material about CVVDP.

status: current

text: CVVDP

anchor: cvvdp

text: Metric Process

anchor: metric-process

text: Initialization and Display Modeling

anchor: initialization-and-display-modeling

text: Input Loading and Display Mapping

anchor: input-loading-and-display-mapping

text: Color Space Conversion

anchor: color-space-conversion

text: Temporal Decomposition

anchor: temporal-decomposition

text: Spatial Decomposition

anchor: spatial-decomposition

text: CSF Weighting & Difference Calculation

anchor: csf-weighting-difference-calculation

text: Visual Masking

anchor: visual-masking

text: Pooling & Scoring

anchor: pooling-scoring

text: Implementations

anchor: implementations

text: CVVDP

text: Vship

anchor: vship

text: fcvvdp

anchor: fcvvdp

text: Visualization

anchor: visualization

markdown:
# CVVDP

Via the [CVVDP GitHub README](https://github.com/gfxdisp/ColorVideoVDP/blob/main/README.md):

> ColorVideoVDP is a full-reference visual quality metric that predicts the perceptual difference between pairs of images or videos. Similar to popular metrics like PSNR, SSIM, and DeltaE 2000 it is aimed at comparing a ground truth reference against a distorted (e.g. blurry, noisy, color-shifted) version.

> This metric is unique because it is the first color-aware metric that accounts for spatial and temporal aspects of vision.

The description of the metric process below is borrowed from the [fcvvdp documentation](https://github.com/halidecx/fcvvdp/blob/main/doc/cvvdp.md).

## Metric Process

### Initialization and Display Modeling

Before processing pixels, the metric models the viewing environment through a
specified display's angular resolution (pixels per degree).

- Using specified display parameters (resolution, diagonal size, viewing
  distance), CVVDP calculates Pixels Per Degree (PPD). This determines how large
  a pixel appears to the eye.
- The Contrast Sensitivity Function (CSF) determines how sensitive the eye is to
  specific spatial frequencies. It maps the display's frequency bands (derived
  from PPD) to sensitivity values.

Available display model presets include:

| Model        | Description                                              |
| ------------ | -------------------------------------------------------- |
| `fhd`        | 24" FullHD monitor, 200 cd/m², office lighting (default) |
| `4k`         | 30" 4K monitor, 200 cd/m², office lighting               |
| `hdr_pq`     | 30" 4K HDR, 1500 cd/m², low light                        |
| `hdr_hlg`    | 30" 4K HDR HLG, 1500 cd/m², low light                    |
| `hdr_linear` | 30" 4K HDR linear, 1500 cd/m², low light                 |
| `hdr_dark`   | 30" 4K HDR, 1500 cd/m², dark room                        |
| `hdr_zoom`   | 30" 4K HDR, 10000 cd/m², close viewing                   |

### Input Loading and Display Mapping

- Input images (uint8, uint16, or float) are converted to linear float RGB. If
  the input is integer-based, sRGB gamma decoding (approx. 2.4 power) is
  applied.
- The linear RGB values are converted into absolute physical light units (nits)
  based on the display model.
- SDR: clips values between 0 and 1, scales by max luminance, adds black level &
  reflected ambient light.
- HDR: Performs tone mapping (PQ/HLG), clips to the display's peak luminance,
  adds black level and reflections.

### Color Space Conversion

1. Linear RGB is converted to the CIE XYZ color space
2. XYZ is transformed to DKL (Derrington-Krauskopf-Lennie), an opponent color
   space that models what is used by the human brain.
   - L: Luminance, a.k.a. achromatic brightness (L+M cones)
   - RG: Chromatic difference (L-M cones)
   - YV: S-cone opponent channel (S - (L+M))

### Temporal Decomposition

If the input is video (FPS > 0), the metric analyzes how pixel values change
over time. It maintains `TemporalRingBuf` to store previous frames.

- The code applies Finite Impulse Response (FIR) filters to the history of DKL
  frames (so, temporal filtering)
- Low temporal frequency information (static or slow-moving) is stored in the
  _sustained channels_. Calculated for Luminance (Y), Red-Green (RG), and
  Yellow-Violet (YV).
- High temporal frequency information (flicker or fast motion) is in the
  _transient channel_. Only calculated for luminance.

We now have 4 channels to process spatially: Y_sus, RG_sus, YV_sus,
Y_trans.

### Spatial Decomposition

The visual system processes different sizes of features (frequencies)
independently. CVVDP implements a _Gaussian pyramid_ to simulate this.

1. The image is repeatedly downscaled (blurred and subsampled).
2. At each level in the pyramid, local contrast is computed.
3. The upscaled version of the next lower level is subtracted from from the
   current level (Gaussian difference)
4. This difference is normalized by the local background luminance (`L_BKG`) to
   get _Weber Contrast_.

### CSF Weighting & Difference Calculation

- For every pixel at every pyramid level, the contrast is multiplied by the CSF
  sensitivity. This scaling depends on:
  - Spatial Frequency, which is determined by the pyramid level
  - Background Luminance, where brighter areas generally have lower sensitivity
    to absolute differences
  - Channel, because the eye is less sensitive to chroma changes (RG/YV) than
    luma changes.
- The absolute difference between the Reference and Distorted contrast values is
  calculated.

### Visual Masking

This is the most complex step. It is designed to account for the fact that
artifacts are harder to see in textured areas.

- The code computes the minimum activity between the reference and distorted
  signals
- A Gaussian blur is applied to this activity map to simulate the spatial extent
  of masking
- Activity in one channel can mask errors in another. The code computes a
  masking denominator using weighted sums of activity from all 4 channels
- The final difference d is compressed using a non-linear sigmoid-like function
  (not going to explain it here, probably best to read the code for more
  details)

### Pooling & Scoring

- The masked differences are aggregated across the image using a Minkowski norm
  (Power of 4), then averaged
- Scores from all pyramid levels and all four channels are summed
- Scores are accumulated over frames using a power sum
- The final raw quality metric (Q) is mapped to _JOD (Just Objectionable
  Difference)_, which is a more meaningful perceptual score. 10.0 is a perfect
  match (no visible difference), and lower scores mean the quality is worse

| Score      | Interpretation                     |
| ---------- | ---------------------------------- |
| 10.0       | Images are identical               |
| 9.0 - 10.0 | Barely visible difference          |
| 8.0 - 9.0  | Slight visible difference          |
| 7.0 - 8.0  | Noticeable but acceptable          |
| 5.0 - 7.0  | Clearly visible, somewhat annoying |
| 3.0 - 5.0  | Very visible, annoying difference  |
| < 3.0      | Large, unacceptable difference     |

## Implementations

Several different implementations of CVVDP are available. Third-party implementations tend to outperform the reference tools.

### CVVDP

[CVVDP](https://github.com/gfxdisp/colorvideovdp) is the first-party reference implementation of CVVDP by the University of Cambridge. It is implemented in Python, and makes use of libraries that allow utilization of the CPU or GPU.

### Vship

[Vship](https://codeberg.org/Line-fr/Vship) is a GPU-accelerated metrics toolkit compatible with Vapoursynth. It also features its own standalone FFVship binary, available independent of Vapoursynth. Vship's CVVDP implementation is an order of magnitude faster than the reference implementation.

### fcvvdp

[fcvvdp](https://github.com/halidecx/fcvvdp/tree/main) is a fast CPU-based CVVDP implementation by Halide Compression. It claims to be over 221% faster than the reference implementation with multithreading, making it useful in environments without access to GPUs.

## Visualization

The graph below (from fcvvdp's docs) visualizes the CVVDP metric process:

![CVVDP](https://mermaid.ink/svg/pako:eNp9VNty2jAQ_ZUdPXSaGZIJDiSBh84EiCkJJATTS2LyoNoLaGpLrizT0JB_71q-5NILLyBz9uw5Z1d-ZIEKkXXZMlI_gzXXBuaDhQT6pNm3lebJGkZSGMEj8YsboWTx55k_EGkS8S14CQbpPezvf9j1eRRkETeY7qDnT6cDeAc9LkNwNf7IUAYC0_uivpcXQN_PuaHvuYScY5wozSNwRWRQV0iU4UK-UTRFve9qHiNMtQowTYVcFZABMSaZgRkuiZI0GhjFfIWFQjj3hzyOOQwwt10grIsJHaOy47k1MxYSuYaxWK3NDlx_NuyBUfD19i7_GlyOS7RboBswGzbg9jP011xKjCiCoV9bmpFA6GXLJeqy7I2jGvnW0NDSu6NZGcsOPvpelhpO8sK6GdxW_e__VTby55rLVKA0VRnc_idjL-H51P8Q9JFSG9kwLyjMjP7hEqZbmoYIba5kJBX5ppTkFxZ86dfbAWMVEPEX_IYa-koazVNTgi8teOyfJUm0tYvhoczpNsJsqSSfBkn5j-4JT7_naeezzfOmtcMCMrbcEyvkwG7IJi1W5Bl5_xJ6VUIrynOJerUtIVc24TqBXpRRyNd-FZuXaOQhqGVVXZZdFzdFqzTdr8YwEQ8E2MG0tF31czMZvAhyQp6mVtiNn0No_s_CqSh5mcpfdmyqVFSP8aZY2xZcKR3vYFYLL1El18zCvCwGtaFpjXFjV9vzi-vnBUpXmXkWOhf0eCQNUs9c-w7mz9fgNffcevlEXhK4yegFQxOmq3VxPXjFW9t5Me1-sSbF4ZM9fH7_3hWSmlRUlmNvjzXYSouQdY3OsMFi1DHPj-wxr14ws8YYF6xLP0Ouvy_YQj5RTcLlnVJxVaZVtlqz7pJHKZ2yJKQ9HghOJuP6Kc0hRN1XmTSs2-50HMvCuo_sgc6HzYOjpnN0eOy0O8edTrvVYFvWPekcHLVO2i2nedpuNZ1W86nBftm-hwenbcdpN0-PnY5z2Dk6bjYYhsIoPSne2PbF_fQbjzvS_g)


title: PSNR

description: PSNR is one of the most widely used objective full-reference image and video quality metrics. It measures the ratio between the maximum possible signal power and the power of corrupting noise, expressed in decibels . The

text: Video Compression

anchor: video-compression

text: Inside Video Encoders

anchor: inside-video-encoders

text: Limitations

anchor: limitations

text: Practical Use Cases

anchor: practical-use-cases

markdown:
PSNR (Peak Signal-to-Noise Ratio) is one of the most widely used objective full-reference image and video quality metrics. It measures the ratio between the maximum possible signal power and the power of corrupting noise, expressed in decibels (dB). The metric is calculated using the Mean Squared Error (MSE) between a reference image and a distorted image. The theory and math behind PSNR are well covered in [Wikipedia's PSNR entry](https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio).

## Video Compression

PSNR is widely used in video encoding applications because it is fast to compute, making it practical for real-time encoding decisions, and because it provides a consistent mathematical basis for comparing different encoding approaches. It is also used within video encoders to help make compression decisions.

### Inside Video Encoders

Within video encoders, PSNR plays a crucial role in rate-distortion optimization (RDO), which is the process of finding the optimal balance between bitrate and quality. Encoders use PSNR as an in-loop metric when evaluating different encoding decisions, such as mode selection, motion estimation, and quantization parameter (QP) selection.

For example, when deciding between different prediction modes or block sizes, the encoder will calculate the PSNR impact of each option along with its bit cost to determine the best choice.

### Limitations

While PSNR is widely used due to its simplicity and computational efficiency, it has several notable limitations. The metric has notoriously poor correlation with the human eye's perception of quality, as PSNR is highly sensitive to all pixel-level errors when many do not have any perceptually relevant impact. When PSNR is used to inform RDO in video encoding, it can lead to suboptimal quality decisions; to combat this, encoders have to creatively take the human visual system into account. We cover some of this in the [Psychovisual entry](/start-here/psychovisual/).

PSNR's weaknesses as a full reference distortion metric have led to the development of more advanced metrics like [SSIM](/reference/metrics/ssim/), [VMAF](/reference/metrics/vmaf/), and [XPSNR](/reference/metrics/xpsnr/), which attempt to better model human visual perception.

## Practical Use Cases

PSNR is commonly used for evaluating image and video compression algorithms, assessing streaming quality, comparing codec performance, and more. Many video encoding tools and analysis suites report PSNR values for different luma/chroma components (Y, U, V) separately, as well as a weighted average. Despite its limitations, PSNR continues to be an important tool in the video compression field, particularly when used in conjunction with other quality metrics and subjective evaluation methods.


title: SSIM

description: The Structural Similarity Index Measure is a full-reference image and video quality metric that quantifies image fidelity degradation caused by processing such as lossy compression. Published in 2004 as part of an issue

text: Overview

anchor: overview

text: Limitations

markdown:
The Structural Similarity Index Measure (SSIM) is a full-reference image and video quality metric that quantifies image fidelity degradation caused by processing such as lossy compression. Published in 2004 as part of an issue of [*IEEE Transactions on Image Processing*](https://en.wikipedia.org/wiki/IEEE_Transactions_on_Image_Processing), SSIM attempts to address the limitations of traditional metrics like [Peak Signal-to-Noise Ratio (PSNR)](/reference/metrics/psnr/) by evaluating visual quality based on the structural information that humans naturally use to assess visual quality.

## Overview

SSIM works by comparing three key elements between the original and processed images: luminance, contrast, and structure. The luminance comparison measures the similarity of the average pixel intensities between the two images. These three comparisons are combined to produce a single similarity score ranging from -1 to 1, where 1 indicates perfect structural similarity.

One of SSIM's main advantages is its ability to better align with human visual perception compared to traditional metrics. SSIM recognizes that pixels have strong inter-dependencies, especially when they are spatially close, which makes SSIM particularly effective at detecting changes in structural information that human observers would notice, such as blurring, blocking artifacts, or noise.

As an in-loop metric in video encoders to improve decisionmaking, SSIM is more computationally expensive than PSNR, and doesn't always yield drastic improvements in fidelity per bit. [Psychovisual](/start-here/psychovisual/) encoder options are still necessary in many cases to achieve the best perceptual efficiency.

## Limitations

In multimedia compression, SSIM can be more valuable in optimization scenarios where the goal is to maintain optimal perceptual quality for a given size. However, SSIM doesn't perfectly correlate with the human visual system; newer metrics like [XPSNR](/reference/metrics/xpsnr/) and [SSIMULACRA2](/reference/metrics/ssimulacra2/) have been developed to correlate more closely with human perception. Modern variations and extensions of SSIM have been developed to address specific needs. Multi-scale SSIM (MS-SSIM) evaluates images at different scales to better match human visual perception. Color SSIM variants have been proposed to better handle color information, and SSIMULACRA (succeeded by SSIMULACRA2) was developed to improve correlation with human perception. These adaptations have improved upon SSIM's perceptual goals in an ever-changing multimedia compression landscape.


title: SSIMULACRA2

description: SSIMULACRA2 is a visual fidelity metric based on the concept of the multi-scale structural similarity index measure , computed in a perceptually relevant color space, adding two other error maps, and aggregating using tw

text: Scoring

anchor: scoring

text: Metric Breakdown

anchor: metric-breakdown

text: Convert sRGB to Linear RGB

anchor: convert-srgb-to-linear-rgb

text: Transform to XYB

anchor: transform-to-xyb

text: Normalize Values

anchor: normalize-values

text: Build a Multi-Scale Image Pyramid

anchor: build-a-multi-scale-image-pyramid

text: Blurred Statistics

anchor: blurred-statistics

text: Similarity & Artifacts

anchor: similarity-artifacts

text: Weighted combination of statistics

anchor: weighted-combination-of-statistics

text: Nonlinear mapping to final score

anchor: nonlinear-mapping-to-final-score

text: Notes

anchor: notes

text: Implementations

text: Cloudinary's SSIMULACRA2

text: vapoursynth-zip Filter

anchor: vapoursynth-zip-filter

text: Vship

markdown:
# SSIMULACRA2

SSIMULACRA2 is a visual fidelity metric based on the concept of the multi-scale structural similarity index measure (MS-SSIM), computed in a perceptually relevant color space, adding two other (asymmetric) error maps, and aggregating using two different norms. It is currently the most reputable visual quality metric according to its correlation with subjective results, and is considered a very robust means of comparing encoders. It is debatable whether [Butteraugli](/reference/metrics/butteraugli/) is better for very high fidelity, but SSIMULACRA2 is considered the best for medium/low fidelity comparisons.

## Scoring

The score that SSIMULACRA 2 outputs is simple: a number in range -inf..100. According to the developers of the metric, for image quality assessment, SSIMULACRA 2 scores correlate to subjective visual quality as follows:

- Very high quality: `90` and above
- High quality: `70` to `90`
- Medium quality: `50` to `70`
- Low quality: Below `50`

## Metric Breakdown

A step-by-step description of the SSIMULACRA2 metric follows. The steps assume we have two input images, a reference image as well as a distorted image to be compared and scored.

### Convert sRGB to Linear RGB

Undo the sRGB gamma curve to obtain linear light values for each RGB channel.
This converts perceptual-encoded pixel values into physically meaningful intensities.

### Transform to XYB

Map linear RGB into an opsin-inspired space that separates perceptual channels.
Steps: apply an absorbance-like linear transform, clamp negatives, take a cube root to compress dynamic range, add small biases, then mix into three channels (X, Y, B).
Resulting channels are tuned to align better with human vision than raw RGB.

### Normalize Values

Apply slight shifts and scalings so all channel values are positive and stable for later statistical operations.
This prevents division instability and extreme ratios in subsequent steps.

### Build a Multi-Scale Image Pyramid

Create multiple downscaled versions of both images (typically several scales, each half the previous dimension).
Each scale captures structure at a different spatial frequency.
The metric computes statistics independently at every scale.

### Blurred Statistics

For each scale and each perceptual channel, compute local blurred statistics. Local blurred quantities are computed on each pixel using a separable spatial blur:
- Local mean (blurred image).
- Local second moments: blurred squared values and blurred cross-products between reference and distorted channels.

These give local variance and covariance estimates analogous to SSIM’s variance/covariance.

### Similarity & Artifacts

**Similarity map (SSIM-like):**
Combine local means, variances, and covariance into a per-pixel similarity value.
This value measures structural agreement while using small stabilizing constants to avoid division by zero.
Similarity is clamped to a sensible range and emphasizes structural fidelity.

**Edge / artifact map:**
Compare local deviations from local means between distorted and reference images to detect:

- New artifacts (excess local detail or harsh edges introduced by distortion).
- Lost detail (original detail suppressed or blurred).
  Compute per-pixel artifact and lost-detail measures, and preserve higher-order statistics to capture outliers.

### Aggregate statistics per channel and scale

Across each channel and scale compute compact summaries:

- Mean of the per-pixel similarity and artifact/lost-detail measures.
- Higher-order moment summaries (fourth-moment based measures reduced by a fourth-root) to detect heavy tails and strong local errors.

These condensed statistics encode both average behavior and extreme localized errors.

### Weighted combination of statistics

Multiply each aggregated statistic by a pre-tuned weight.
Sum all weighted terms across channels and scales to produce a single scalar accumulator.
Weights are learned/tuned to map the diverse statistics into a perceptually meaningful predictor.

### Nonlinear mapping to final score

Pass the accumulator through a nonlinear curve (polynomial and a power-law transform).
This mapping compresses the predictor into a bounded perceptual score.
The final value is expressed on a convenient scale where higher means better and values near the top indicate imperceptible differences.

## Notes

- The metric examines structure and edge behavior separately. It penalizes both new artifacts and lost detail.
- Multi-scale analysis makes it sensitive to distortions at different spatial frequencies.
- Aggregating higher-order moments preserves sensitivity to rare but visually important outliers.

## Implementations

There are a couple of different SSIMULACRA2 implementations available, some useful in different contexts.

### Cloudinary's SSIMULACRA2

[Cloudinary's SSIMULACRA2 implementation](https://github.com/cloudinary/ssimulacra2) is the reference implementation written in C++. It comes from the libjxl project, the reference implementation of the [JPEG XL](/codecs/images/jxl/) image codec.

### `vapoursynth-zip` Filter

[`vapoursynth-zip`](https://github.com/dnjulek/vapoursynth-zip) is a collection of filters for use with [Vapoursynth](/tools/filtering/vapoursynth/). It is written in Zig, and features a SSIMULACRA2 implementation.

### fssimu2

[fssimu2](https://github.com/gianni-rosato/fssimu2) is a fast SSIMULACRA2 implementation written in Zig. It is designed for speed, claiming to be up to 14% more performant while using just 50% of the memory. It displays a recorded error of ~1.5% relative to Cloudinary's reference implementation, and it achieves 99.7% correlation according to the Pearson correlation coefficient documented in the README. It is based on Julek's Zig implementation.

### Vship

[Vship](https://codeberg.org/Line-fr/Vship) is a GPU-accelerated metrics toolkit compatible with Vapoursynth. It also features its own standalone FFVship binary, available independent of Vapoursynth. Vship's SSIMULACRA2 implementation is an order of magnitude faster than CPU-based implementations, and has reportedly high correlation with the reference implementation.

### `ssimulacra2_rs`

[`ssimulacra2_rs`](https://github.com/rust-av/ssimulacra2_bin) is a binary interface to the [Rust implementation of the SSIMULACRA2 metric](https://github.com/rust-av/ssimulacra2). It is notable for being one of the first independent implementations, as well as one of the first to consider video inputs.


title: VMAF

text: VMAF

anchor: vmaf

text: Installation

anchor: installation

text: Scoring

text: Some weaknesses

anchor: some-weaknesses

text: Comparing to SSIMULACRA2

text: Additional resources

anchor: additional-resources

markdown:
# VMAF

> **Note — Under Maintenance.**
>
> The content in this entry is incomplete & is in the process of being completed.
Short for Video Multimethod Assessment Fusion, VMAF is a full reference video quality assessment algorithm developed primarily by Netflix.

## Installation

Vmaf comes as a part of [libvmaf](https://github.com/Netflix/vmaf). There are two ways it is commonly used:
* As an [FFmpeg](/tools/utilities/ffmpeg/) filter
* As a standalone binary

The instructions below are written for Linux & macOS. On Windows, you can use the [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install) to follow along.

    #### Standalone Binary

In order to build from source, follow the instructions below.

    0. Install the required dependencies via your package manager of choice. The necessary dependencies are `nasm`, `ninja-build`, `doxygen`, & `xxd`.

    1. Clone the repository & enter the corresponding directory
    ```bash title="Clone & Enter"
    git clone https://github.com/Netflix/vmaf/
    cd vmaf/
    ```

    2. Compile with `meson` & `ninja`
    ```bash
    meson setup libvmaf libvmaf/build --buildtype release -Denable_float=true
    sudo ninja -vC libvmaf/build install
    ```

    Now, you can run the VMAF binary with the following command:
    ```bash
    vmaf --help
    ```

    If you would not like to build from source, you may grab the latest build from the VMAF [GitHub releases](https://github.com/Netflix/vmaf/releases) for your operating system.

    Now, you can:

    ```bash
    /path/to/vmaf --reference refrence.y4m --distorted distorted.y4m
    ```

    **Tip:** If the VMAF binary exists but is not market as executable, you might need to `chmod +x /path/to/vmaf`

    Explainer on command line flags can be found [here](https://github.com/Netflix/vmaf/blob/master/libvmaf/tools/README.md)

    The disadvantage of using the bin is that you need .yuv|y4m files, you can that overcome by using named pipes
    Simple example using ffmpeg as a decoder:

    ```bash
    # create the pipes
    mkfifo ref.pipe
    mkfido dist.pipe

    # run these each in a new terminal, order docent matter
    ffmpeg -v error -i ref.mkv -strict -1 -f yuv4mpegpipe - > ref.pipe
    ffmpeg -v error -i dist.mkv -strict -1 -f yuv4mpegpipe - > dist.pipe

    # after starting the two ffmpeg processes,
    # start the vmaf in a new terminal
    /path/to/vmaf --reference ref.pipe --distorted dist.pipe

    # delete the pipes after usage
    rm ref.pipe dist.pipe
    ```

    The Advantages of this are:
    * No need for a ffmpeg build with `--enable-libvmaf`
    * Clear & simple usage of VMAF's various options, like `--aom_ctc`

    Disadvantages are:
    * Difficult/awkward to use without a wrapper script
    #### FFmpeg Filter

If you are not sure if you have VMAF installed, you can check by running `ffmpeg -help` and looking for whether or not the `--enable-libvmaf` flag appears in the banner that is printed to the terminal. If you do not see this, you will need to build ffmpeg from source with the `--enable-libvmaf` flag or grab a pre-compiled build of FFmpeg with the flag enabled.

    *Via the [VMAF github repo](https://github.com/Netflix/vmaf/blob/master/resource/doc/ffmpeg.md):*
    ## Using VMAF with FFmpeg

    After installing `libvmaf`, you can use it with [FFmpeg](http://ffmpeg.org/). Under the FFmpeg directory, configure, build and install FFmpeg with:

    ```shell script
    ./configure --enable-libvmaf
    make -j4
    make install
    ```

    Using FFmpeg+libvmaf is very powerful, as you can create complex filters to calculate VMAF directly on videos of different encoding formats and resolutions. For the best practices of computing VMAF at the right resolution, refer to our [tech blog](https://medium.com/netflix-techblog/vmaf-the-journey-continues-44b51ee9ed12).

    We provide a few examples how you can construct the FFmpeg command line and use VMAF as a filter. Note that you may need to download the test videos from [vmaf_resource](https://github.com/Netflix/vmaf_resource/tree/master/python/test/resource).

    Below is an example on how you can run FFmpeg+libvmaf on a pair of YUV files. First, download the reference video [`src01_hrc00_576x324.yuv`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/yuv/src01_hrc00_576x324.yuv) and the distorted video [`src01_hrc01_576x324.yuv`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/yuv/src01_hrc01_576x324.yuv). `-r 24` sets the frame rate (note that it needs to be before `-i`), and `PTS-STARTPTS` synchronizes the PTS (presentation timestamp) of the two videos (this is crucial if one of your videos does not start at PTS 0, for example, if you cut your video out of a long video stream). It is important to set the frame rate and the PTS right, since FFmpeg filters synchronize based on timestamps instead of frames.

    The `log_path` is set to standard output `/dev/stdout`. It uses the `model_path` at location `/usr/local/share/model/vmaf_float_v0.6.1.json` (which is the default and can be omitted).

    ```shell script
    ffmpeg -video_size 576x324 -r 24 -pixel_format yuv420p -i src01_hrc00_576x324.yuv \
        -video_size 576x324 -r 24 -pixel_format yuv420p -i src01_hrc01_576x324.yuv \
        -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \
                [1:v]setpts=PTS-STARTPTS[distorted]; \
                [distorted][reference]libvmaf=log_fmt=xml:log_path=/dev/stdout:model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json:n_threads=4" \
        -f null -
    ```

    The expected output is:

    ```shell script
    [libvmaf @ 0x7fcfa3403980] VMAF score: 76.668905
    ```

    Below is a more complicated example where the inputs are packaged `.mp4` files. It takes in 1) a reference video [`Seeking_30_480_1050.mp4`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/mp4/Seeking_30_480_1050.mp4) of 480p and 2) a distorted video [`Seeking_10_288_375.mp4`](https://github.com/Netflix/vmaf_resource/blob/master/python/test/resource/mp4/Seeking_10_288_375.mp4) of 288p upsampled to `720x480` using bicubic, and compute VMAF on the two 480p videos. Bicubic is used as the recommended upsampling method (also see the [techblog](https://medium.com/netflix-techblog/vmaf-the-journey-continues-44b51ee9ed12) for more details).

    ```shell script
    ffmpeg \
        -r 24 -i Seeking_30_480_1050.mp4 \
        -r 24 -i Seeking_10_288_375.mp4 \
        -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \
                [1:v]scale=720:480:flags=bicubic,setpts=PTS-STARTPTS[distorted]; \
                [distorted][reference]libvmaf=log_fmt=xml:log_path=/dev/stdout:model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json:n_threads=4" \
        -f null -
    ```

    The expected output is:

    ```shell script
    [libvmaf @ 0x7fb5b672bc00] VMAF score: 51.017497
    ```

    See the [FFmpeg's guide to libvmaf](https://ffmpeg.org/ffmpeg-filters.html#libvmaf), the [FFmpeg Filtering Guide](https://trac.ffmpeg.org/wiki/FilteringGuide) for more examples of complex filters, and the [Scaling Guide](https://trac.ffmpeg.org/wiki/Scaling) for information about scaling and using different scaling algorithms.

    ### Note about the model path on Windows

    Due to Windows not having a good default for where to pull the VMAF model from, you will always need to specify `model_path` when calling libvmaf through `ffmpeg`. However, you will need to be careful about the path you pass to `model_path`.

    If you are using a relative path for your `model_path`, you can completely ignore this whole section, else if you are trying to use an absolute Windows path (`D:\mypath\vmaf_v0.6.1.json`) for your `model_path` argument, you will need to be careful so `ffmpeg` passes the right path to `libvmaf`.

    The final command line will depend on what shell you are running `ffmpeg` through, so you will need to go through the following steps to make sure your path is okay.

    1. Convert all of the backslashes `\` to forward slashes `/` (`D:/mypath/vmaf_v0.6.1.json`)
    2. Escape the colon `:` character by using a backslash `\` (`D\:/mypath/vmaf_v0.6.1.json`)
    3. Then escape that backslash with another backslash (`D\\:/mypath/vmaf_v0.6.1.json`)
    4. The next step will depend on the shell that will run `ffmpeg`:

        For PowerShell and Command Prompt, this will be enough and your final `ffmpeg` command line will look something like
        ```powershell
        ./ffmpeg.exe -i dist.y4m -i ref.y4m \
            -lavfi libvmaf=model_path="D\\:/mypath/vmaf_v0.6.1.json" \
            -f null -
        ```

> **Info — Quoting the path.**
>
> Note: I only quoted the path part for trivial reasons and in this specific case, it can be unquoted or you can quote the whole part after lavfi starting from `libvmaf` to `json` and it should give the same result due to neither shell treating the `\` as a special character
        For bash or specifically msys2 bash, it has some additional considerations. The first thing to know is that bash treats the backslash character `\` a bit special in that it's an escape character normally when not put inside single quotes. The second thing to know is that msys2's bash attempts convert a posix-like path (`/mingw64/share/model/vmaf_v0.6.1.json`) to a Windows mixed path (`D:/msys2/mingw64/share/model/vmaf_v0.6.1.json`) when passing arguments to a program. Normally, this would be fine, however, in our case, this works against us since we cannot allow it to convert the path to a normal path with an un-escaped colon. For this, we will need to not only escape the escaped backslash, but we will also need to pass the `MSYS2_ARG_CONV_EXCL` environment variable with the value of `*` to make sure it doesn't apply that special conversion on any of the arguments

        ```bash
        MSYS2_ARG_CONV_EXCL="*" \
            ./ffmpeg.exe -i dist.y4m -i ref.y4m -lavfi \
            libvmaf=model_path="D\\\:/mypath/vmaf_v0.6.1.json" -f null -
        ```
> **Info — Quotes.**
>
> Note: in this case, the quotes are not as trivial as the PowerShell/cmd version, as removing the quotes entirely will require you to re-escape the backslash resulting in 4 total backslashes, but quoting the whole argument will be fine.
> **Note — Single Quotes.**
>
> Second Note: if you use single quotes around the path, it will be fine as well and the final command line would look like
>
>         ```bash
>         MSYS2_ARG_CONV_EXCL="*" \
>             ./ffmpeg.exe -i dist.y4m -i ref.y4m -lavfi \
>             libvmaf=model_path='D\\:/mypath/vmaf_v0.6.1.json' -f null -
>         ```
>
>         with only a double backslash instead of a triple.
## Scoring
scores range from 0 to 100, and are best interpreted in a linear way,
100 meaning perfect quality, 0 meaning not recognisable,
more info [in the Best Practices section here](https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12)
It aligns with mean opinion scores (MOS) really well at low/medium bitrates,
as stated [in this benchmark](https://videoprocessing.ai/benchmarks/video-quality-metrics_both.html)

## Some weaknesses

* Newer codecs like AV1 and VVC introduce new kinds of artifacting that v0.6.2
(current model as of Jan 2024) doesn't recognise,
that's why its performance might degrade, for example, high motion scenes being affected badly
* It's bad at "transparent" levels of quality, kinds of quality that the average viewer might not notice
* Synthetic grain throws off scores, this issue is not isolated to vmaf, but it should be noted regardless
> **Tip — With ffmpeg you can disable application of synthetic grain.**
>
> place `-filmgrain 0` before `-i` in the above ffmpeg commands, limited to decoding with dav1d
> TODO: replace this tip with an export_side_data solution
* As of January 2024, it doesn't work on HDR content, nothing prevents you from feeding it un-tonemapped PQ but scores will be off
* In contrast with SSIMULACRA2, it focuses on appeal, not necessarily on fidelity. They often align, but not always.
* Due to the ML nature, comparing the same video to itself will not always result in a score of 100

### Comparing to SSIMULACRA2

One big advantage over SSIMULACRA2 is the inclusion of some temporal information.
This means that VMAF weights frames that have a lot of motion higher.
Meanwhile, SSIMULACRA2 based solutions compare each frame to the reference frame individually, since it is an image
metric at heart.
VMAF also wins in speed and general ease of use.

# Additional resources

- [VMAF Documentation on GitHub](https://github.com/Netflix/vmaf/tree/master/resource/doc)
- [Medium Article by Netflix](https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12)


title: XPSNR

description: XPSNR is a full-reference distortion measurement algorithm for video quality assessment, based on the concept of the peak signal to noise ratio , tuned to better reflect the human visual system.

text: Installation

text: Usage

anchor: usage

text: Notes

text: Comparing to SSIMULACRA 2

markdown:
> **Note — Under Maintenance.**
>
> The content in this entry is incomplete & is in the process of being completed.
XPSNR is a full-reference distortion measurement algorithm for video quality assessment, based on the concept of the peak signal to noise ratio (PSNR), tuned to better reflect the human visual system.

XPSNR is considered to be a highly reputable visual quality metric according to its correlation with subjective results relative to its impressive speed.
Unlike image-first metrics like [SSIMULACRA2](/reference/metrics/ssimulacra2/), XPSNR is a video-focused metric, and uses temporal information to compute its score.

The reference implementation by Fraunhofer HHI is a [FFmpeg 7.0 plug-in](https://github.com/fraunhoferhhi/xpsnr), and there is a [community-built standalone implementation](https://github.com/gianni-rosato/sxpsnr) available as well.
For more details about the algorithm and its design, [a scientific paper is available](https://www.itu.int/pub/S-JOURNAL-ICTS.V3I1-2020-8).

## Installation

XPSNR comes as an FFmpeg 7.0 plug-in. The process of installing it is somewhat straightforward:

1. Download the source code for FFmpeg 7.0:
```bash
git clone -b release/7.0 https://git.ffmpeg.org/ffmpeg.git ffmpeg
```

2. Download the code for the XPSNR plug-in:
```bash
git clone https://github.com/fraunhoferhhi/xpsnr
```

3. Copy the plug-in files from XPSNR's code to FFmpeg's:
```bash
cp xpsnr/libavfilter/* ffmpeg/libavfilter/
```

4. Configure and compile FFmpeg:
```bash
cd ffmpeg
./configure ...
make -j $(nproc)
```

## Usage

XPSNR is used as an FFmpeg filter, similarly to the built-in PSNR plugin:
```bash
./ffmpeg -i ref.mkv -i test.mkv -lavfi xpsnr="stats_file=xpsnr.log" -f null -
```

## Notes

* Being the reference implementation an FFmpeg plug-in and not a standalone binary is inconvenient for some workflows.
Currently, there are some attempts at making a [standalone XPSNR binary](https://github.com/gianni-rosato/sxpsnr), although they are not very widespread yet.
* XPSNR uses a PSNR-like scoring system: a logarithmic scale in range 0..inf.
This is considerably harder to interpret than other metrics (e.g. [SSIMULACRA 2](/reference/metrics/ssimulacra2/) or [VMAF](/reference/metrics/vmaf/));
and the threshold to what constitutes good quality is content-dependent.
(Preliminary testing seems to indicate that anything above a XPSNR score of 42.00 is visually lossless.)
* XPSNR scores are computed independently for the luma (Y) and two chroma (U, V) components of videos,
thus actually providing 3 scores.
Although the developers argue that using the minimum of these 3 scores as "definitive score"
(i.e. better correlates to human impressions),
some people in the community prefer using a weighted sum: `(4 * XPSNR_Y + XPSNR_U + XPSNR_V) / 6`.

## Comparing to SSIMULACRA 2

The main advantage of XPSNR over SSIMULACRA 2 is that it is considerably faster to compute.
XPSNR can achieve real-time speeds for 1080p 24 fps video, making it more convenient to quickly compare test encodes.

Regarding their visual assessment capabilities, XPSNR and SSIMULACRA 2 seem to complement each other quite well,
having both their own strong points and weak points with specific video content.
If possible, using both at the same time provides great benefits for video quality assessment and comparison.


title: Privacy

description: Analytics and data-handling information for Encode Wiki.

markdown:
Encode Wiki is a static site. Reading public pages does not require an account, and the site does not
currently provide comments, forms, or an AI chat service.

The production domain uses a self-hosted [Umami](https://umami.is/) analytics service at
`u.servers.moe` to understand aggregate site usage. The local analytics loader injects the tracker
only when the browser hostname is exactly `encode.wiki`, and the tracker itself is also restricted to
that domain. Local development and `dev.encode.wiki` therefore do not contact or record production
analytics. No analytics API or administrative credential is included in the site.

The hosting and analytics servers necessarily receive ordinary network request data such as IP
addresses, user-agent headers, requested URLs, timestamps, and error information. Encode Wiki does not
use advertising cookies or sell reader data. This notice will be revised before any feature that
accepts user-submitted data is launched.

Questions or privacy concerns can be reported through a confidential issue in the project's GitLab
repository.


title: Resources

markdown:
Encoding resources that you might find useful.

- https://guide.encode.moe - Filtering and fansubbing.
- https://lvsfunc.encode.moe/en/latest - lvsfunc documentation.
- https://silentaperture.gitlab.io/mdbook-guide/introduction.html - SilentAperture's Advanced Encoding guide, mostly about filtering.
- https://x265.readthedocs.io/en/master - x265 technical documentation, made by MulticoreWare themselves.
- http://www.chaneru.com/Roku/HLS/X264_Settings.htm - x264 settings.
- https://kokomins.wordpress.com/2019/10/10/anime-encoding-guide-for-x265-and-why-to-never-use-flac - Anime encoding guide by Kokomins. Has some pretty good advices regarding psychovisual stuff and x265.
- https://wiki.xiph.org/Main_Page - Xiph Wiki.


title: Start here

description: Learn the core ideas and vocabulary behind encoding.

markdown:
Encoding transforms information into another representation. For media, that usually means using a
codec to make audio, video, images, or subtitles practical to store and distribute.

This section will help you understand:

- what codecs, formats, encoders, and containers each do;
- the tradeoff between quality, size, speed, compatibility, and complexity;
- how lossy and lossless compression differ; and
- how to recognize and investigate common encoding artifacts.

Most of this material will be migrated from the existing Codec Wiki with explicit attribution and a
visible review status.


title: Prologue

text: What is a Codec

anchor: what-is-a-codec

markdown:
Multimedia compression has allowed us to communicate information and expression in novel ways as the unsung hero of the modern Web. Despite this, it is often difficult to uncover information about codec technology that is accurate, informed, and accessible. This wiki aims to demystify multimedia compression while connecting codec enthusiasts to create a sink of knowledge for everyone.

The Codec Wiki is not geared toward the mathematics, adoption/patent politics, or specifications of specific coder/decoder implementations – sources like Wikipedia cover these details. We are focused on making accurate, useful information highly accessible.

### What is a Codec

A codec, shortened from coder/decoder, is a system that handles digital data according to a specification. Typically this means compressing and decompressing digital media. Codecs are used to encode media for storage and transmission, and then decode for playback, editing, etc. Multimedia codecs generally either [discard data](/start-here/psychovisual/) using [lossy](/start-here/video-artifacts/) compression to reduce filesize, or they use clever lossless compression tricks to maintain a mathematically identical stream to the input media while still reducing filesize. Lossless compression can be reversed to be the exact same as the input data, while lossy compression does not share this quality as it discards data for smaller filesizes. Some common uses of codecs include:

- Video compression: Video codecs like [H.264](/codecs/video/avc/), [VP9](/codecs/video/vp9/) & [AV1](/codecs/video/av1/) allow digital video files to be compressed to much smaller sizes. A video codec can encode a video stream while it is being recorded or before it is distributed, and decode it when it is played back. This allows video to be shared more quickly while using less storage & bandwidth.
- Audio compression: Audio codecs like [MP3](/codecs/audio/mp3/), [AAC](/codecs/audio/aac/), and [Opus](/codecs/audio/opus/) compress audio files like songs & podcasts. This allows them to be easily distributed & stored.
- Image Compression: Image codecs compress images while maintaining a versatile featureset for the myriad of ways one may decide they'd like to compress an image. Color depth, HDR, transparency, color space information, EXIF data, and many other factors are relevant when working with images.
- Data Compression: General compression algorithms like [ZIP](/codecs/data/zip/) & [zstd](/codecs/data/zstd/) are designed to compress *any* kind of data, not just multimedia specific data. This includes web assets, executables, text archives, and even entire filesystems.


title: high-dynamic-range

description: HDR is a technology used in modern TVs and displays to produce more vibrant and lifelike images. In simple terms, it allows your TV to display a wider range of colors and brightness levels than standard displays. This me

text: High Dynamic Range

anchor: high-dynamic-range

text: HLG

anchor: hlg

text: Dolby Vision

anchor: dolby-vision

markdown:
# High Dynamic Range

HDR (High Dynamic Range) is a technology used in modern TVs and displays to produce more vibrant and lifelike images. In simple terms, it allows your TV to display a wider range of colors and brightness levels than standard displays. This means that you can see more details in both bright and dark areas of an image, which can make movies, TV shows, and video games look much more realistic.

HDR10 works by using metadata that tells your TV how to display the content in the best way possible. This metadata includes information about the maximum brightness level and color gamut of the content, which allows your TV to adjust its settings to match the content being displayed. In other words, HDR10 helps your TV display images that are closer to what the content creators intended you to see, resulting in a more immersive viewing experience.

## HLG

HLG (Hybrid log-gamma) is a type of HDR video format that was developed to optimize video for both standard dynamic range (SDR) and HDR displays, jointly developed by the BBC and NHK.

To understand how HLG works, it's helpful to know that the way we perceive brightness and color in a video is different from how it's captured and displayed on a screen. Brightness and color information is usually captured in a logarithmic curve, while SDR displays typically reproduce the image with a gamma curve. HDR displays, on the other hand, reproduce the image with a different type of curve, known as the Perceptual Quantizer (PQ) curve.

The HLG curve is a hybrid of these two curves, which means that it's optimized for both SDR and HDR displays. It's designed to work with a wider range of brightness levels than SDR displays, but also be backward compatible with SDR displays.

In simpler terms, the HLG curve is a way of capturing and displaying video that works well on both SDR and HDR displays. It's like a bridge between the way video is captured and the way it's displayed, and it's designed to optimize the video for a wider range of brightness levels than traditional SDR video. The result is video content that looks more realistic and vivid on both SDR and HDR displays.

## HDR10

HDR10 is an open high-dynamic-range video (HDR) standard announced on 27 August 2015 by the Consumer Technology Association. It is the most widespread of the HDR formats. It only allows static metadata.

## HDR10+

HDR10+ is basically an upgrade to the previous HDR10 by adding dynamic metadata support (in ``.json``) to optimize each scene's content light level as the director intended.

## Dolby Vision

Dolby Vision is proprietary HDR format developed by Dolby Laboratories and a direct competitor to HDR10+.


title: Lossless Compression

text: Redundancy & Entropy

anchor: redundancy-entropy

text: Techniques in Lossless Compression

anchor: techniques-in-lossless-compression

markdown:
> **Note — Under Maintenance.**
>
> The content in this entry is incomplete & is in the process of being completed.
Lossless compression is a method of data compression that allows the original data to be perfectly reconstructed from the compressed data. This is particularly important in applications where perfectly preserving the original fidelity of a medium is critical, such as in archiving, generic data compression, & professional media editing. To understand how lossless compression works, we will first delve into the concepts of redundancy, entropy, and specific compression techniques oft used in lossless compression.

### Redundancy & Entropy

The concepts of redundancy & entropy are important to understand as you continue reading.

**Redundancy** refers to the repetitive or predictable elements in data. These elements do not add new information and can be efficiently encoded to reduce the overall data size without losing any information.

**Entropy**, in the context of information theory, is a measure of the unpredictability or randomness of data. Lower entropy implies higher redundancy, implying that the data is theoretically more compressible.

In lossless compression, the goal is to reduce redundancy and encode data as efficiently as possible based on its entropy.

### Techniques in Lossless Compression

1. **Run-Length Encoding (RLE)**:
   RLE is a simple form of lossless compression where sequences of the same data value (runs) are stored as a single data value and a count. This technique is effective for compressing data with long runs of identical samples, such as silence or constant tones. For example, the sequence `AAAAABBBCC` could be encoded as `5A3B2C`.

2. **Huffman Coding**:
    Huffman coding is an entropy encoding algorithm used for lossless data compression that works by separating an input into component symbols and replacing each symbol with a code. The algorithm builds a binary tree, with each leaf node representing a symbol separated from the input data, and the path from the root to the leaf representing the binary code for that symbol.

    Huffman coding is effective when the probability distribution of the input characters is known and can be exploited. Imagine you are storing the state of a traffic light; it is either green, yellow, red, or off for maintenance. As the operator, you have determined that it is green 50% of the time, red 40% of the time, yellow 9% of the time, and disabled 1% of the time. Because there are four options, you can accurately represent all of the possible symbols in our example using two bits. Green could be `00`, red `01`, yellow `10`, and off `11`. While assigning two-bit codes accurately conveys the information, we're storing an average of two bits per symbol; we can reduce the average number of bits per symbol by taking the probabilities into account here. We'll assign green to `0` since it appears the most frequently; this is the first leaf on our binary tree. Next, we have the leaves that stem from the `1` code; red can simply be `11`, while yellow can be `100` and the disabled symbol can be represented by `101`. This gives us the following Huffman codes:
        - Green     (50%): `0`
        - Red       (40%): `11`
        - Yellow    ( 9%): `100`
        - Disabled  ( 1%): `101`

    Represented by the following tree:

        ![Traffic light Huffman tree](https://raw.githubusercontent.com/av1-community-contributors/images/main/color-huffman-tree-svg.svg)

    Now, if we do the math by multiplying the probability by the length of each code and taking the weighted sum:
        (50% • 1) + (40% • 2) + (9% • 3) + (1% • 3) = **1.6**

    We end up with an average of **1.6 bits per symbol**. Even in our relatively simple example, this shows how Huffman coding can save space quite effectively while still losslessly representing the same information.

3. **Arithmetic Coding**
   Arithmetic coding is another entropy encoding technique that represents an entire message as a single number in the interval `[0, 1)`. Unlike Huffman coding, which assigns fixed binary codes to component symbols separated from an input, arithmetic coding represents multiple symbols with a single floating-point number *q* which must be within the range `0.0 ≤ q < 1.0`. Arithmetic coding is particularly effective when the probability distribution of the symbols is skewed, as it can produce a more compact representation than Huffman coding. It is usually slower than Huffman coding.

   Let's return to our traffic light example. Let's say you aren't satisfied with our previous Huffman coding result, which has produced an average of 1.6 bits per symbol, and you'd like to use arithmetic coding instead. In arithmetic coding, each of our symbols should first be placed on a range within the interval `[0, 1)` based on its probability. Then, we can narrow down this range as we encode more symbols, eventually arriving at a single number that represents the entire sequence. The steps follow below.

    Let's use the same probabilities from before:
    - Green     (50%)
    - Red       (40%)
    - Yellow    ( 9%)
    - Disabled  ( 1%)

    We'll assign ranges to each symbol as follows:

    - Green:    `[0.00, 0.50)`
    - Red:      `[0.50, 0.90)`
    - Yellow:   `[0.90, 0.99)`
    - Disabled: `[0.99, 1.00)`

    Now, let's encode a sequence of traffic light states: "Green, Red, Yellow, Green"

    1. Start with the interval `[0, 1)`

    2. First symbol (Green):
        - `new_low = low + (high - low) • cumulative(s) / total`
        - `new_high = low + (high - low) • (cumulative(s) + prob(s)) / total`
        - `new_low = 0 + (0.5 - 0) • 0 / 1.0`
        - `new_high = 0 + (0.5 - 0) • (0 + 0.5) / 1.0`
        - Narrow range to `[0.00, 0.50)`

    3. Second symbol (Red):
        - From previous range: `[0.00, 0.50)`
        - `new_low = 0 + (0.50 - 0) * 0.50 / 1`
        - `new_high = 0 + (0.50 - 0) * (0.50 + 0.40) / 1`
        - Red is in the range `[0.25, 0.45)`

    4. Third symbol (Yellow):
        - From previous range: `[0.25, 0.45)`
        - `new_low = 0.25 + (0.45 - 0.25) * (0.50 + 0.40) / 1`
        - `new_high = 0.25 + (0.45 - 0.25) * (0.50 + 0.40 + 0.09) / 1`
        - Yellow is in the range `[0.43, 0.448)`

    5. Fourth symbol (Disabled):
    - From previous range: `[0.43, 0.448)`
        - `new_low = 0.43 + (0.448 - 0.43) * (0.50 + 0.40 + 0.09) / 1`
        - `new_low = 0.43 + (0.448 - 0.43) * (0.50 + 0.40 + 0.09 + 0.01) / 1`
        - Green is in the range `[0.44782, 0.448)`

    The final interval is `[0.44782, 0.448)`. Any number in this range (we'll pick the lower bound, which is inclusive of 0.44782) can represent our entire sequence "Green, Red, Yellow, Disabled".

    To decode, we would start with 0.44782 and use our original probability ranges to determine which symbol it corresponds to, then update the value and repeat the process.

    In this example, we aren't saving any space by using arithmetic coding because our sample is too short to have any pattern to effectively exploit. With longer sequences, arithmetic coding approaches the theoretical entropy limit of **1.408 bits per symbol**: `-(0.50 * log2(0.50) + 0.40 * log2(0.40) + 0.09 * log2(0.09) + 0.01 * log2(0.01)) ≈ 1.408`

    It is important to note that in practice, there often are additional considerations not present in this simplified example with regard to managing precision.

4. **Prediction and Residual Encoding**:
   Prediction involves using previous data to predict future data. The difference between the predicted and actual data (residual) is encoded instead of the actual data. Linear Predictive Coding (LPC) is a common method where a linear function of previous samples is used to predict the current sample. The residuals typically have lower entropy and can be encoded more efficiently.


title: Lossy Compression

text: How Lossy Compression Works

anchor: how-lossy-compression-works

markdown:
> **Note — Under Maintenance.**
>
> The content in this entry is incomplete & is in the process of being completed.
Lossy multimedia compression reduces the file size of multimedia data by permanently removing some of the information. This process leverages the limitations of the human senses, fidelity metrics, or appeal metrics to discard information that is considered less salient. The goal of lossy compression is to reduce the file size of multimedia data while maintaining a desired level of quality.

## How Lossy Compression Works

Lossy compression works by analyzing the input signal and removing parts of it that are less salient. Some processes by which this is done include:

- **Perceptual Coding** (audio): This technique removes audio frequencies that are outside the range of human hearing or masked by other sounds.
- **Quantization** (audio): This process reduces the precision of certain audio components, which can significantly reduce file size without a noticeable impact on perceived quality.

*More coming soon*


title: psychovisual

text: Explanation

anchor: explanation

text: Fidelity Versus Appeal

anchor: fidelity-versus-appeal

text: Psychovisual Options

anchor: psychovisual-options

text: Metrics

anchor: metrics

text: Conclusion

anchor: conclusion

markdown:
"Psychovisual fidelity" is a common term used in video encoding to describe the quality of an encoded video as perceived by the human visual system. It has a number of alternate terms that mean the same thing:
- "Perceptual quality"
- "Subjective quality"
- "Visual quality"

Psychovisual options are introduced into encoders to combat the limitations of traditional decision-making inside video encoders, which tends to prioritize efficiency as measured by simple metrics like [PSNR](/reference/metrics/psnr/).

The amount of care and attention an encoder directs towards preserving psychovisual fidelity is often indicative of an encoder's maturity, and is a key reason why mature encoders like [x264](/tools/software-encoders/x264/) and [x265](/tools/software-encoders/x265/) are so widely regarded as being well-designed and effective.

## Explanation

Via the [x265 documentation](https://x265.readthedocs.io/en/master/cli.html#psycho-visual-options):

> Left to its own devices, video encoders will make mode decisions based on a simple rate distortion formula, trading distortion for bitrate. This is generally effective except for the manner in which this distortion is measured. It tends to favor blurred reconstructed blocks over blocks which have wrong motion. The human eye generally prefers the wrong motion over the blur and thus x265 offers psycho-visual adjustments to the rate distortion algorithm.

An important concept that is essential to understanding psychovisual fidelity is the idea behind *visual energy*. While the x265 documentation snippet above correctly identifies that the human eye prefers wrong motion over blur, a similar concept applies in the case of detail preservation. The human eye is more forgiving in the presence of detail – even if it is wrong detail – than in the presence of blur due to our preference for visual energy. Psychovisual options are often tailored to preserving visual energy in the encoded video.

### Fidelity Versus Appeal

Via [Cloudinary's blog](https://cloudinary.com/blog/what_to_focus_on_in_image_compression_fidelity_or_appeal):

> Fidelity in images is about visually preserving the original; appeal is about hiding the compression artifacts. Depending on your priority, you would compress images with either of these approaches to reduce the file size while still maintaining a reasonable level of visual “quality” ...

Many people report that they prefer blurring in videos over blocking artifacts, which is a conflation of the value of fidelity versus appeal. While blurring can be beneficial at lower bitrates to help reduce the visibility of blocking artifacts, it is generally preferable to view a slightly blurry image rather than a blocky one with *distracting* artifacts. This underscores how essential it is to strike a balance with psychovisual options in encoders; heavily utilizing these settings can negatively effect visual quality in certain scenarios, like when encoding at low fidelity where artifacts would be erroneous enough that they would be distracting.

### Psychovisual Options

The following are some common psychovisual options that are available in x265, from the x265 documentation linked prior:

> `--psy-rd` will add an extra cost to reconstructed blocks which do not match the visual energy of the source block. The higher the strength of `--psy-rd` the more strongly it will favor similar energy over blur and the more aggressively it will ignore rate distortion. If it is too high, it will introduce visual artifacts and increase bitrate enough for rate control to increase quantization globally, reducing overall quality. psy-rd will tend to reduce the use of blurred prediction modes, like DC and planar intra and bi-directional inter prediction.

> `--psy-rdoq` will adjust the distortion cost used in rate-distortion optimized quantization (RDO quant), enabled by `--rdoq-level` 1 or 2, favoring the preservation of energy in the reconstructed image. `--psy-rdoq` prevents RDOQ from blurring all of the encoding options which psy-rd has to choose from. At low strength levels, psy-rdoq will influence the quantization level decisions, favoring higher AC energy in the reconstructed image. As psy-rdoq strength is increased, more non-zero coefficient levels are added, and fewer coefficients are zeroed by RDOQ's rate distortion analysis. High levels of psy-rdoq can double the bitrate which can have a drastic effect on rate control, forcing higher overall QP, and can cause ringing artifacts. psy-rdoq is less accurate than psy-rd, it is biasing towards energy in general while psy-rd biases towards the energy of the source image. But very large psy-rdoq values can sometimes be beneficial.

In summary, `--psy-rd` attempts to preserve the visual energy of the encode relative to the source video, while `--psy-rdoq` attempts to make decisions generally in favor of preserving visual energy.

Currently, [x264](/tools/software-encoders/x264/) includes both of the above options alongside a number of other psychovisual options, and is considered the most mature psychovisually optimized open source video encoder. While it doesn't have `--psy-rdoq`, [SVT-AV1-PSY](/tools/software-encoders/svt-av1-psy/) is an AV1 encoder that includes a number of psychovisual options (including `--psy-rd`, as of version 3.0.0).

### Metrics

Currently, it is difficult to measure the effectiveness of psychovisual options in encoders without subjective visual testing. While metrics like [VMAF](/reference/metrics/vmaf/) and [SSIM](/reference/metrics/ssim/) are designed to be more perceptually accurate than PSNR, they are far from perfect and are usually misleading.

More modern metrics like [SSIMULACRA2](/reference/metrics/ssimulacra2/) and [XPSNR](/reference/metrics/xpsnr/) are designed to be more perceptually accurate than others, and while SSIMULACRA2 largely achieves this goal, newer metrics are still not perfect and usually reward disabling all psychovisual options even when the options clearly help preserve psychovisual fidelity.

## Conclusion

Psychovisual options are essential to preserving visual quality. While perceptually driven features are hard to develop, difficult to test and measure, and somewhat rare, they are generally considered to be beneficial to the encoding process. The inclusion of well-built psychovisual options in encoders often signifies maturity and effectiveness, contributing to the overwhelmingly positive legacies of encoders like x264 and x265 which have consistently prioritized visual quality to great effect.


title: Terminology

text: Bitstream

anchor: bitstream

text: Lossy / Lossless

anchor: lossy-lossless

text: Elementary stream

anchor: elementary-stream

text: Muxing

anchor: muxing

text: Codec

anchor: codec

text: Filter

anchor: filter

text: Muxer/Demuxer

anchor: muxerdemuxer

text: Bitstream filter

anchor: bitstream-filter

text: Container

anchor: container

text: MP4 / M4V

text: MOV

anchor: mov

text: MKV / MKA / MKS / MK3D

text: WebM

anchor: webm

text: Transcoding

anchor: transcoding

text: RDO

anchor: rdo

anchor: perceputal-psychovisual-psychoacoustic

text: Discrete Cosine Transform (DCT)

anchor: discrete-cosine-transform-dct

markdown:
When learning about encoding technology, it is important to understand the vast terminology that is often used to describe concepts that are often not very complex to understand.

## Bitstream
A *bitstream* or *bit stream* is a media file, the kind that is played in a media player. It consists of a [container](#container) wrapping multiple [elementary streams](#elementary-stream)

## Lossy / Lossless
*Lossy* encoding throws out some of the detail to achieve a smaller size. Often, this is an acceptable trade-off, but if you need a perfect recreation of the data, you need *lossless* encoding.

## Elementary stream
An elementary stream is an audio, video, or subtitle track. Basically, it's the compressed data you want to [mux](#muxing) into the container.

## Muxing
Putting elementary streams into a container, which preserves them without making any changes to the data.

## Codec
A codec (**co**der/**dec**oder) is the piece of code that actually encodes the data you put in. It takes as input and produces as output an elementary stream. More information is provided [in the prologue](/start-here/encoding-basics/#what-is-a-codec) under "What is a Codec".

## Filter
A filter is a piece of code you can apply to the data to make something about it different, for instance sharpening, removing artifacts, shakiness, denoising, scaling, overlay, etc.

## Muxer/Demuxer
The pieces of code that [mux](#muxing) or do the reverse, getting elementary streams from the container.

## Bitstream filter
A bitstream filter is a filter that is directly applied to the [bitstream](#bitstream) in order to change something about the container, for instance, convert frame types, or corrupt some packets.

## Container
A container is a format for putting one or more elementary streams into one file, which is then called a [bitstream](#bitstream).

A video container is a digital file format that holds video and audio data, as well as additional information such as subtitles, metadata, and chapter markers. It acts as a "wrapper" that packages all these elements into a single file that can be played on various devices and software platforms. Think of it like a container you might use to transport goods - the video and audio data are like the items being transported, while the container itself provides a structure and organization for the contents.

Some kinds of containers:

#### MP4 / M4V

This is likely the most common container you've encountered, & has near universal compatibility. Has a limited maximum amount of streams. The supported video codecs are [H.264](/codecs/video/avc/), [H.265](/codecs/video/hevc/), [H.266](/codecs/video/vvc/), DivX, Xvid, [VP9](/codecs/video/vp9/) (Unofficial, hacky), and [AV1](/codecs/video/av1/) (Unofficial, hacky). For audio codecs it's many of the various flavors of [AAC](/codecs/audio/aac/), [MP3](/codecs/audio/mp3/), [FLAC](/codecs/audio/flac/) (Unofficial), [Opus](/codecs/audio/opus/) (Unofficial, hacky). For subtitles only MPEG-4 Timed Text (TTXT) is supported.

The best tool to work with this container is MP4Box, but FFmpeg also works.

#### MOV

Similar to MP4, but less supported. Made with Apple Quicktime in mind, supports ProRes.

#### MKV / MKA / MKS / MK3D

Also known as Matroska, allows an unlimited amount of video/audio/subtitle streams and any codec that probably still exists in Area 51, you can put literally anything in there and it won't even care, MPEG-2/DivX/H.266/Theora/Thor/RealVideo/MJPEG/AVS3/AMR-WB, you name it. All around best container for working with if you have the choice.

#### WebM

A container made with web streaming in mind. WebM is a stripped-down subset of MKV that only allows free & open source codecs such as VP8, VP9 or AV1 for video alongside Vorbis or Opus for audio. It is a common misconception that [WebVTT](/codecs/subtitles/webvtt/) tracks always work natively in browsers when within a WebM container; in practice, WebMs containing WebVTT subtitles *will usually not* play back the subtitles in browsers. WebVTT subtitles can be utilized with the `<track>` element instead, meaning they exist outside the WebM container itself. More info in the [Mozilla Web Docs](https://developer.mozilla.org/en-US/docs/Web/Media/Audio_and_video_delivery/Adding_captions_and_subtitles_to_HTML5_video).

## Transcoding
Taking an elementary stream & converting it to another format, lossless or lossy, using an encoder of some kind. For example, if I convert a lossless [FFV1](/codecs/video/ffv1/) video to a lossy AV1 video using an encoder like [rav1e](/tools/software-encoders/rav1e/), I have *transcoded* this lossless video to AV1. Transcoding doesn't have anything to do with the container.

## RDO
RDO, or Rate-Distortion Optimization, is a technique used to find the best trade-off between the bit rate & the quality of lossily encoded content. RDO can be metric-based, optimizing to score well on metrics like [PSNR](/reference/metrics/psnr/) or [SSIM](/reference/metrics/ssim/).

## Perceputal / Psychovisual / Psychoacoustic
"Psychovisual quality" (for videos), "Psychoacoustic quality" (for audio), or "perceptual quality" is a term used to describe the perception of quality of distorted media by the human senses. The goal of any multimedia codec is to minimize data while maintaining perceived quality, and optimizing around human perception theoretically yields the best performance within any limited set of coding techniques (like when using an older codec). Our model of human perception continues to evolve, which makes modelling perceptual quality very difficult. Presently, the metrics [SSIMULACRA2](/reference/metrics/ssimulacra2/) (Images/Video) & [Butteraugli](/reference/metrics/butteraugli/) (Video) are considered the most accurate to our human visual system.

## Discrete Cosine Transform (DCT)
The Discrete Cosine Transform is a mathematical transformation that can transform discrete data into the frequency domain. This discrete data could be pixels in an image/video compression block or data points recorded temporally representing an audio recording. This algorithm is a particularly good choice for image, video, music, & speech compression because it has high energy compaction relative to our understanding of fidelity in media. High energy compaction means the DCT is able to represent a signal with a small number of significant coefficients.


title: video-artifacts

description: Codec Wiki reference material about video-artifacts.

text: Spotting Video Artifacts

anchor: spotting-video-artifacts

text: Moiré Pattern

anchor: moiré-pattern

text: Staircase Effect

anchor: staircase-effect

text: Color Bleed

anchor: color-bleed

text: Ringing

anchor: ringing

text: Blocking

anchor: blocking

text: Banding/Contouring

anchor: bandingcontouring

text: Mosquito Noise

anchor: mosquito-noise

markdown:
# Spotting Video Artifacts

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/solar_system_compression_artifacts_2x.avif" type="image/avif" />

</picture>

*https://xkcd.com/2414*

Video artifacts are visual distortions or anomalies that appear in a video, which can affect the quality of the image. Artifacts are first categorized by whether they’re time/sequence-based (temporal) or location-based (spatial). If you can see the artifact when the video is paused, then it’s probably a spatial artifact. If it’s much more visible while the video plays, then it’s likely temporal.

## Moiré Pattern

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/moire-pattern.jxl" type="image/jxl" />

</picture>

Moiré patterns are large-scale spatial interference pattern produced when a pattern in the source image and the manner in which the encoder operates are slightly out of alignment spatially. The artifacts generated by the encoder then introduce strange, swirling effects in the source image's pattern upon decoding.

## Staircase Effect

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/staircase-effect.jxl" type="image/jxl" />

</picture>

The staircase effect is a spatial artifact that occurs when diagonal straight or curved edges that should be smooth take on a jagged appearance, looking somewhat like a set of stair steps. This is the effect that is being reduced by "anti-aliasing" filters.

## Color Bleed

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/color-bleed.jxl" type="image/jxl" />

</picture>

Color bleeding, as its name suggests, occurs when the edges of one color in the image unintentionally bleeds or overlaps into another color. The cause of color bleeding is usually related to the compression algorithm's handling of chrominance information. Chrominance refers to the color information in an image or video, and it's usually compressed separately from the luminance (brightness) information. In some compression algorithms, the chrominance information may be compressed more aggressively than the luminance information, resulting in a loss of color accuracy and detail.

## Ringing

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/ringing-artifact.jxl" type="image/jxl" />

</picture>

Note the blue and pink fringes around the edges of the star above (as well as the stepping and other significant compression artifacts). Those fringes are the ringing effect. It is very similar to mosquito noise which will be talked about down below.

## Blocking

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/blocking_on_cat.jxl" type="image/jxl" />

</picture>

To understand this artifact, we first need to know what DCT (Discrete Cosine Transform)-based compression is. It's a way of reducing the size of a digital image or video file by analyzing the data in small chunks (called "blocks") and then using mathematical formulas to transform that data into a more compact representation. This transformed data can then be stored or transmitted more efficiently.

However, this compression technique can sometimes lead to visible "distortions" in the compressed image or video. One of the most common types of distortion is the "blocking artifact," which occurs when the edges of the compressed blocks become visible, creating a grid-like pattern over the image or video.

In simpler terms, imagine if you take a picture and then try to compress it to make the file size smaller. The compression algorithm divides the picture into small blocks and tries to simplify the information in each block. But when there's not enough bitrate/data, the algorithm simplifies the information too much, resulting in visible blocks or a grid pattern on the picture. This is what we call the DCT blocking compression artifact.

## Banding/Contouring

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/colorbanding_ex1.jxl" type="image/jxl" />

</picture>

Banding or contouring is a specific form of posterization in which the color blocks form bands or stripes in the image. This occurs when the video is encoded with too coarse a quantization (Technique used in video compression that reduces the amount of data needed to store or transmit a video by rounding off certain values in the video's signal) configuration and a lack of available color gradient. As a result, the video's contents show a "layered" look, where instead of smooth gradients and transitions, the transitions from color to color are abrupt, causing strips of color to appear.

## Mosquito Noise

<picture>
    <source srcset="https://github.com/av1-community-contributors/images/raw/main/mosquito-noise.jxl" type="image/jxl" />

</picture>

Mosquito noise appears as small, moving, and blocky artifacts around sharp edges or fine details in videos or images. They are mostly found in videos that are compressed using the DCT algorithm we talked about earlier, such as MPEG videos or JPEG images.


title: Tools

description: Encoders, filters, players, inspectors, and related utilities.

markdown:
Tool pages document software and hardware encoders, filters, media players, muxers, inspectors, and
utilities. Version-sensitive instructions carry a last-verified date and tested version information.


title: Antialiasing

description: Codec Wiki reference material about Antialiasing.

markdown:
> **Danger — Help Wanted.**
>
> This section is in need of contributions. If you believe you can help, please see our [Contribution Guide](/reference/legacy/contribution-guide/) to get started as a contributor!


title: Basics of Filtering

text: Basics of Filtering with Vapoursynth

anchor: basics-of-filtering-with-vapoursynth

text: Intro to Filters

anchor: intro-to-filters

text: Filter Order

anchor: filter-order

text: Bit Depth and Colorimetry

anchor: bit-depth-and-colorimetry

text: Cropping

anchor: cropping

text: Resizing

anchor: resizing

text: Trimming

anchor: trimming

markdown:
# Basics of Filtering with Vapoursynth

## Intro to Filters

In order to do filtering, we are going to need some filters. Vapoursynth includes some basic filters, but there are a lot more useful filters
built by the community that can be incredibly useful to us.

For example, we are going to make use of some of the functions in [vs-tools](https://github.com/Jaded-Encoding-Thaumaturgy/vs-tools).
If you are on Arch, the recommended way to install this is via the AUR, using the `vapoursynth-plugin-vstools-git` package.
For other operating systems, it is simpler to install all of the JET plugins at once, using the [vs-jet utility](https://github.com/Jaded-Encoding-Thaumaturgy/vs-jet).
Note that these do require Python, which you should already have installed if you've installed Vapoursynth. You may have noticed in the previous section
that all Vapoursynth scripts use Python syntax. It is not necessary to know Python in order to write Vapoursynth scripts, although it can be helpful
for some more advanced techniques.

## Filter Order

Order matters when applying filters to a video. There are certain actions that must be taken at the very start, and certain actions that are better to be done
later in a filter chain. The following image shows a recommended order for many of the most common filter actions:

![filter chain](/img/vs2.webp)

If this seems a bit complicated at first, that's okay. We will learn about each of these steps one at a time, and when
each of these filters might be needed.

## Bit Depth and Colorimetry

In most cases, the first action we want to take on a video is to ensure the correct [matrix, primaries, and transfer characteristics](/reference/colorimetry/intro/) are set, and convert the video to 16-bit.
The colorimetry properties are important for ensuring accurate colors, and 16-bit helps filters to work in higher quality and create fewer artifacts.

The currently recommended way to handle this is with the `vstools.initialize_clip` function. This function will handle both ensuring that colorimetry is set, and converting to 16-bit with appropriate dithering if needed.
We can add it to a script in the following way:

```python
import vapoursynth as vs
core = vs.core
clip = core.lsmas.LWLibavSource(source="myinput.mkv")

import vstools
clip = vstools.initialize_clip(clip)

clip.set_output(0)
```

If the video has existing colorimetry data set on it, `initialize_clip` will keep the existing data. Otherwise, it will assume colorimetry based on the video resolution.
Usually, those assumptions will be correct. However, there may be cases where we know that the assumptions are incorrect, and we need to set them manually.
For example, we may have a 4k video that we know was upscaled from a 1080p video, and we want to force BT.709 colorimetry. In that case, we can specify the properties to `initialize_clip`,
and it will use the properties we specify instead of making guesses.

```python
from vstools import Primaries, Matrix, Transfer
clip = vstools.initialize_clip(clip, primaries=Primaries.BT709, matrix=Matrix.BT709, transfer=Transfer.BT709)
```

`initialize_clip` also has a companion function called `finalize_clip` which will, by default, convert the video down to 10-bit with dithering, and if the video is in limited color range,
clip all pixels to fit within the limited color range (this would be done by the player on playback, but if we do it now, we can save a few bits for the encoder). The function also takes
a `bits` parameter in case you would like to finalize to 8-bits instead.

As a result, a very basic template for a Vapoursynth script would be something like this:

```python
import vapoursynth as vs
core = vs.core
clip = core.lsmas.LWLibavSource(source="myinput.mkv")

import vstools
clip = vstools.initialize_clip(clip)
//
// Add filters inside this section
//
clip = vstools.finalize_clip(clip)

clip.set_output(0)
```

## Cropping

![screenshot](/img/crop_1.jpg)

Sometimes, you may encounter a source that has black bars on the sides or on the top and bottom. Generally, we don't like to keep these, and want to remove them. This process is called "cropping".

First, we need to figure out how many pixels to crop from each side. Fortunately, vs-preview has a tool to make this easy. Using this tool, which is found under the "Misc" section in the bottom right
of the app, allows us to experiment with crop values and find the correct ones without having to reload the Vapoursynth script.

![crop tool](/img/crop_tool.webp)

Once we flip the tool to "On", we can adjust each of the dimensions until we see the black borders disappear. We want to try to get as exact as possible, although the crop values must be a multiple of 2
when working with YUV420 video (which is the most common). In this case, we know we want to remove from the top and bottom, so we will adjust those values until we find the right ones. Note that the size of the bars
may often be the same on both sides, but not always, so do be careful to check both sides.

![zoom tool](/img/zoom_tool.avif)

vs-preview's zoom tool shown here can also be helpful to ensure we've gotten the right values.

Once we've found the correct values, we can click "Copy cropping command" to copy our value to the clipboard. We can then paste it into our Vapoursynth script, being sure to add the appropriate variable name for our clip.

```python
clip = clip.std.Crop(0, 0, 138, 138)
```

Remember to turn off the crop tool in vs-preview, or else we'll be doubling the crops from here on. Now, go ahead and reload the preview with Ctrl+R. No more black bars.

![screenshot](/img/crop_2.jpg)

## Resizing

Resizing is the process of changing the resolution of a video, whether that's making it bigger or smaller. Note that although the general idea is that "higher resolution is better",
we generally don't want to upscale content. This simply makes output videos larger in file size, but doesn't result in an increase in quality, because the higher resolution is artificial.
However, there are times when you may want or need to resize a video. Unfortunately, this is not as simple as it sounds, because we want to select a high-quality resizing method.

With this section, we'll introduce [slow.pics](https://slow.pics/), a very useful tool for comparing screenshots. Here, we'll be using it to show the differences between various resizers.

Vapoursynth comes with several resizers built-in, which will be the easiest to use. Of these, we'll be showcasing the following:

```python
clip = clip.resize.Bicubic(1920, 1080)
clip = clip.resize.Lanczos(1920, 1080)
clip = clip.resize.Spline36(1920, 1080)
```

Bicubic is the most widely used resizer. It is very fast and produces decent quality, but may produce less sharpness than more advanced scalers.
Lanczos produces more sharpness, but may also produce more [ringing](/tools/filtering/dehalo/).
Spline36 uses a spline-based resizing method, and produces results somewhere between Bicubic and Lanczos.

However, using plugins, we have access to some more advanced scalers.

```python
clip = vskernels.Catrom().scale(clip, 1920, 1080)
clip = vskernels.Catrom(sigmoid=True).scale(clip, 1920, 1080)
clip = vskernels.Hermite(linear=True).scale(clip, 1920, 1080)
clip = vsscale.SSIM().scale(clip, 1920, 1080)
```

Here we will look at downscalers.

Catrom, short for Catmull-Rom, is a variant of Bicubic which uses different parameters in order to achieve a better balance between sharpness and ringing.
It is often considered the best of the Bicubic kernels for image quality.

In the second example, you can see that we pass an additional `sigmoid=True` parameter to `Catrom`. This performs the scaling using a sigmoid function, which flattens the ends of the scaling curve to reduce ringing.

The third example here, Hermite, is another variant of Bicubic, and we are passing `linear=True` to perform the rescaling in linear light rather than in gamma light. This kernel is very good in regard to
not producing ringing, but it does have the effect of thinning line art. Therefore, although some users prefer it, other users dislike it.

Fourth, we have SSIM, which is a downscaler tuned around the SSIM video quality metric. This downscaler is intended to prioritize preserving detail and sharpness when downscaling.

In [this comparison](https://slow.pics/c/mb5Rm3k3), we can see the differences between the different scalers, on an image that was upscaled to 4k by nnedi3, then downscaled back using the scaler noted in the image title.
Here we have a couple of other [examples](https://slow.pics/c/kQvDOMYh) comparing in particular the [higher-quality scalers](https://slow.pics/c/UVwS62Vs).

Of these, my current recommendation would be `Catrom(sigmoid=True)` for anime, and `SSIM` for live action. Some users may prefer `Hermite(linear=True)` for anime.

For upscaling, we have the following high quality options:

```python
clip = vskernels.EwaLanczos().scale(clip, 3840, 2160)
clip = vsaa.Nnedi3().scale(clip, 3840, 2160)
clip = vsscale.Waifu2x().scale(clip, 3840, 2160)
```

Ewa-Lanczos is a variant of lanczos intended for high sharpness, making it good for upscaling, where we want an image to look sharper so that it does not appear upscaled.

Nnedi3 is a neural-network-based upscaler which attempts to interpolate every second line to produce a sharp, upscaled image. It is generally safe for use on all content, and produces good results.

Waifu2x is another AI-assisted upscaler, this one tuned specifically on anime. It produces very high quality and sharp images. However, the results of using it on live action content are unknown, so it is recommended to only use it on anime.

[Here](https://slow.pics/c/CpISIk1z) we can see the effects of these upscalers. My current recommendation would be `EwaLanczos` for live action, and `Waifu2x` for anime.

```python
clip = soifunc.good_resize(clip, 1920, 1080)
clip = soifunc.good_resize(clip, 1920, 1080, anime=True)
```

Lastly, we have `soifunc.good_resize`, which is a smart hybrid scaler which automatically chooses the best scaler depending on whether we are upscaling or downscaling,
and may use a different scaler for chroma than it does for luma for even better quality.
Note that the scalers chosen by this function may change over time as new knowledge surfaces, and are based on the above recommendations.
We can also add the `anime=True` parameter (false by default) to change the scalers to be tuned
more toward the fine line art that is present in anime, rather than the highly detailed elements such as skin and hair that are present in live action.
With these features, it is intended to be an easy-to-use general purpose high-quality upscaler.

## Trimming

Sometimes, we may need to cut out a portion of the video, for example, if we want to encode only the opening theme, or if our source has commercials that we want to remove.
Vapoursynth supports this using Python's slice syntax.

```python
clip = clip[1805:3185]
```

This snippet will trim the clip to include frames 1805-3184. Note that frame 3185 is _not_ included.

Now, let's say we wanted to _exclude_ this segment. We can do this by making clips of the segments we'd like to keep, and joining them together.

```python
clip = clip[:1805] + clip[3185:]
```

This snippet will create trims for the start of the clip to and including frame 1804, and from frame 3185 to the end of the clip. It then joins them together into one clip, in the order we specify.
If we wanted to reverse the segments, so that segment 2 comes first, we could do that simply by changing the order:

```python
clip = clip[3185:] + clip[:1805]
```


title: Deband

description: Codec Wiki reference material about Deband.


title: Decombing

description: Codec Wiki reference material about Decombing.


title: Dehalo

description: Codec Wiki reference material about Dehalo.


title: Deinterlace

description: Codec Wiki reference material about Deinterlace.