Virtual Acoustics for Immersive Audio

Virtual acoustics is the simulation of sound propagation in a virtual or real environment, involving technologies to model how sound travels, reflects, and interacts with the space. It is essential for creating immersive and convincing audio-visual experiences in extended reality (XR). This course will cover the fundamentals of spatial audio technology, with an emphasis on recreating virtual acoustics for AR and VR. The first part of the course will cover reverberation, which models the propagation of sound from a source placed in the environment to the listener, and the complex interactions of the sound waves with the environment. The second part of the course will focus on spatialisation, which is how sound is localised in space. We will discuss rendering out-loud through multiple loudspeakers and binaurally through headphones. There will be theory lectures and hands-on learning with lab assignments (in Python) and demos which will make use of CCRMA’s 3D listening room.

A limited number of scholarships are available for students and individuals from underrepresented backgrounds in the field.
The application deadline is May 16th at 23:59 (AoE).
If you're interested, please complete the questionnaire at this link.

Tentative schedule

Lectures will be divided into theoretical parts (morning) and hands-on exercises and demos (afternoon).

Week 1: Room Acoustics and Artificial Reverberation

Day 1: Introduction to room acoustics

Physics of sound

• Wave equations
• Room modes

Acoustic parameters

• Energy decay and reverberation time
• Directivity
• Echo density

Day 2: Artificial reverberation

Overview of the history of artificial reverberation (physical and analog reverbs)

Digital artificial reverberation methods

• Convolution
• Wave based methods
• Geometric methods
• Noise-based methods

Day 3: Artificial Reverberation with Delay Networks

Schroeder's allpass reverberator

Moorer's frequency-dependent reverberator

Jot and Chaigne's Feedback Delay Network (FDN)

• Designing FDN parameters
• Adding control over reverberation time in FDNs

Scattering Delay Networks

Day 4: Differentiable artificial reverberation

Overview of supervised learning and training pipeline

Differentiable DSP (DDSP)

• Audio loss functions
• Frequency domain optimization using the FLAMO library

Differentiable FDNs for artificial reverberation

Day 5: Modeling acoustics in coupled spaces

Multi-slope energy decay

Auralisation of coupled room acoustics

• Grouped Feedback Delay Networks
• Coupled Volume Scattering Delay Networks
• Hybrid image source + FDN

Room impulse response synthesis for 6DoF rendering in Augmented Reality

Common slopes model

Neural acoustical fields

Week 2: Spatial Audio

Day 6: 3D sound perception

• Sound localization and acoustic cues
• Externalization and collapse
• Head-related transfer functions (HRTFs) and binaural rendering
• Binaural room impulse responses (BRIRs)
• Spatial room impulse responses (SRIRs)

Day 7: Multichannel rendering

• Microphone arrays
• Vector-based amplitude panning (VBAP)
• Ambisonics

Day 8: Parametric spatial audio

• Spatial Decomposition Method (SDM)
• Directional Audio Coding (DiRAC)
• Spatial Impulse Response Rendering (SRIR)

Day 9: Spatial audio rendering with delay networks

• Directional Feedback Delay Network (DFDN)
• Binaural Scattering Delay Network
• Decorrelating FDNs for multichannel late reverb rendering

Day 10: Guest lecture and demo

Top figure: CCRMA stage https://music.stanford.edu/venues-facilities/venues/ccrma-stage

About the instructors...

Orchisama Das is a postdoctoral researcher at King’s College London. Prior to this, she was a Senior Audio Research Scientist at Sonos, and a Research Fellow at the Institute of Sound Recording at University of Surrey. She received her PhD from the Center for Computer Research in Music and Acoustics at Stanford University in 2021, during which she interned at Tesla and Meta Reality Labs. Her research interests are artificial reverberation and room acoustics modeling, with a focus on real-time room acoustics rendering with delay networks.

Gloria Dal Santo received the M.Sc. degree in electrical and electronic engineering from the Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland in 2022. She is currently working toward the Doctoral degree with the Acoustics Lab, Aalto University, Espoo, Finland. Her research interests include artificial reverberation and audio applications of machine learning.