Audio Presence in VR

Last week I participated in the evaluation of an omnidirectional speaker built by Parichat Songmuang – a master’s student in Music Technology. Listening to a recording of a cello through the speaker gave me the illusion that the musician was with me in the room. The experience was so realistic and inspiring that it made me think how it could change the audio reproduction systems of future VR.

“Presence”, which is often explained as “feeling of being there”, is one of the common measures used to describe the quality of a VR experience. This applies both to the visual and the audio layers of VR. A high-quality audio layer creates an illusion that the sound is truly in the room. Achieving this level of realism is difficult, and requires an appropriate sound reproduction system.

We experienced this when designing a VR theatrical experience – Holojam in Wonderland. Actors and audience members were in the same physical space wearing headsets. They could see each other as avatars in the virtual space, but could hear their real voices. There was a moment during the action when a character grew much bigger than the audience and his voice was played through the overhead speaker hanging from the ceiling.  Many people described that the most powerful moment of the whole experience was when the actor transformed back to his normal size and they could hear his real voice again. I think that this moment was so effective because they had an immediate comparison between the voice from the speaker and from the real person – the real voice was “present” with them.  They were convinced that the voice was truly coming from the person in front of them – in the virtual space.

This level of authenticity is very hard to achieve with 3D audio rendering on headphones (let alone any traditional stereo system). While binaural rendering software is constantly developing, it is still not perfect. There are so many subtleties in how we perceive sound in real environments that the technology is still far from being able to model all of the tiny characteristics of each factor. This is why I found it so powerful when listening to the omnidirectional speaker during Pari’s experiment. Traditional speakers project most of the sound energy in front of the speaker. Playing the voice through this kind of speaker does not make us believe that the person is with us in the room. The situation is different with the omnidirectional speaker – it projects the sound energy in all directions – creating a 3D sound source which is much more similar to the real source.

Now, imagine using this speaker in VR experiences – the speaker is placed in the space according to the position of a static object in VR. The audience could walk around and experience a very realistic sound without a real actor.  The speakers could also be mounted on robots which would move them through space according to the position of the visual object in VR. Complementing the system with environmental sounds coming through surround speakers and near-field speakers (mounted on headsets), we could create a very realistic scene with a sense of presence that would be hard to achieve with each of the systems separately.

 

Image: Omnidirectional speaker designed by Parichat Songmuang.