Reverse passthrough VR shows a three-dimensional view of the wearer’s eyes to multiple outside viewers. Perspective-correct external eye views are achieved using a light field display comprised of an LCD and microlens array. These displays are driven using input from internal stereo pairs that view each eye, followed by a learned depth and color inference pipeline.

This system, in conjunction with existing forward passthrough technology, enables more seamless interactions between people with and without VR headsets in social and professional contexts. Reverse passthrough VR ties together research in social telepresence and copresence, autostereoscopic displays, and facial capture to enable natural eye contact and other important non-verbal cues in a wider range of interaction scenarios.

Nathan Matsuda, Facebook Reality Labs

Brian Wheelwright, Facebook Reality Labs

Joel Hegland, Facebook Reality Labs

Douglas Lanman, Facebook Reality Labs