- An integrated 6DoF AR system of volumetric video and object-based audio
An integrated 6DoF AR system of volumetric video and object-based audio
NHK (Japan Broadcasting Corporation) Science & Technology Research Laboratories
The music performance of a six-piece band is converted into 3D data using volumetric capture and streamed to AR (Augmented Reality) viewer devices. The members of the band are placed in a 3D space by AR, and binaural sound is generated interactively according to the position of each performer and the viewer's 6DoF (6 Degrees of Freedom) position and gaze direction. By approaching the performer of your favor, you can not only observe the performance closely, but also focus on the tone, providing a deep immersion experience in the world of the content through the natural interplay of sight and sound.