Spatial and immersive audio
“Spatial audio” covers several different layers, and most confusion comes from mixing them up:
- A codec compresses audio samples: Opus, AAC-LC, FLAC.
- An immersive format describes channels, objects, ambisonics and rendering metadata on top of a codec: IAMF, Dolby Atmos, MPEG-H.
- A renderer turns that description into output for a particular speaker layout or headphones.
- Binaural is a headphone rendering method, not a codec or container.
Converting the underlying codec usually throws away the object or scene metadata, even though a channel mix still plays. Keep the original immersive track and make any compatibility mix in addition. See Dolby for what FFmpeg can and cannot do with Atmos.
Comparison
Section titled “Comparison”| System | What it is | Carried by | Openness |
|---|---|---|---|
| IAMF / Eclipsa Audio | Open immersive format (channels, objects, ambisonics, mix metadata); Eclipsa is the consumer brand | Opus, AAC-LC, FLAC or LPCM in ISOBMFF | Open AOMedia specification, with a reference decoder and the Open Audio Renderer |
| MPEG-H 3D Audio | ISO standard for channels, objects and higher-order ambisonics, with interactivity and accessibility metadata | Its own codec | International standard; licensed |
| Sony 360 Reality Audio | Sony’s object-based music format | Built on MPEG-H 3D Audio | Proprietary ecosystem |
| Dolby Atmos | Channel bed plus objects and metadata | E-AC-3 JOC, TrueHD, AC-4 | Proprietary, licensed |
| DTS:X | DTS’s object-based format | An extension of DTS-HD Master Audio on Blu-ray | Proprietary |
| Auro-3D | Layered height-channel format | Height channels folded into a 5.1 or 7.1 PCM signal, on Blu-ray usually inside DTS-HD Master Audio | Proprietary |
| Binaural | HRTF-filtered stereo derived from a spatial scene | Any stereo codec | A technique, not a format |
IAMF and Eclipsa Audio
Section titled “IAMF and Eclipsa Audio”IAMF (Immersive Audio Model and Formats) is the open option from the Alliance for Open Media; the current specification is version 2.0. It is codec-agnostic: Opus, AAC-LC, FLAC and LPCM are the allowed codecs, and Opus is just one of them. It supports channel layouts such as 5.1.2 and 7.1.4, objects and ambisonics. Streams carry the coded audio plus metadata for mixing, loudspeaker rendering and binaural rendering, packaged in ISOBMFF (MP4). AOMedia publishes a reference decoder (libiamf) and the Open Audio Renderer. Eclipsa Audio is the consumer brand and certification program built on IAMF, backed by Google and Samsung. Device support is still limited.
| Relationship | Project or format | Evidence and scope | Status |
|---|---|---|---|
| Permits Codec | Opus | IAMF defines an Opus codec configuration alongside AAC-LC, FLAC, and LPCM configurations.Scope: Codec Binding · Sources: Immersive Audio Model and Formats v1.1 | Normative |
| Renders To | Binaural rendering | IAMF mix presentations can be rendered to binaural headphone output.Scope: Renderer · Sources: Immersive Audio Model and Formats v1.1, Open Audio Renderer 1.0.0 | Normative |
| Based On by | Eclipsa Audio | Eclipsa Audio builds consumer tools, certification, and device interoperability around IAMF.Scope: Ecosystem · Sources: Eclipsa Audio ecosystem and certification, Immersive Audio Model and Formats v1.1 | Documented |
MPEG-H and Sony 360 Reality Audio
Section titled “MPEG-H and Sony 360 Reality Audio”MPEG-H 3D Audio supports channels, objects and higher-order ambisonics with flexible layouts and headphone output. Sony describes 360 Reality Audio as an object-based music experience whose distribution format is based on MPEG-H 3D Audio.
Dolby Atmos, DTS:X and Auro-3D
Section titled “Dolby Atmos, DTS:X and Auro-3D”Atmos is covered on the Dolby page. DTS:X and Auro-3D tracks on Blu-ray are backward compatible: a player without the decoder plays the ordinary DTS-HD MA or PCM surround mix. FFmpeg decodes only that mix. Copy these tracks if you want to keep the immersive version.
Binaural
Section titled “Binaural”A binaural renderer filters a spatial scene with head-related transfer functions to make a two-channel headphone mix. Quality depends on the HRTF and whether head tracking is used. The resulting stereo file plays anywhere, but it no longer contains the original scene.
Playback
Section titled “Playback”No browser or OS exposes a single “spatial audio” capability. Playing the 5.1 mix of an E-AC-3 Atmos stream, or running a Web Audio PannerNode, does not mean an authored scene or its objects are being rendered. The image and audio support guide covers detection.