Researchers at Facebook Reality Labs created a prototype headset with external displays to show the wearer’s eyes so other people can make eye contact with the wearer. The goal was to make conversations with someone wearing a virtual reality (VR) headset less awkward.
The first version of this technology, created in 2019, consisted of a simple screen on the front of the headset that displayed a floating 3D image of the user’s face. The latest development is 3D display tech that uses purpose build optics, electronics, software and other technology to capture and depict realistic 3D faces.
The proposed headset uses reverse passthrough VR. Currently, Quest headsets use Passthrough+ based on external-facing cameras to allow headset users to see external surroundings, but those in the room cannot see the wearer’s eyes. Reverse passthrough creates an externally facing display.
The new VR headset Source: Facebook Reality Labs
3D displays allow the user’s eyes and face to appear in the correct and natural position on the screen. This allows the face rendering to maintain alignment as people move in relation to the display.
The headset has two display pods that are mirror images of each other: an LCO panel and a lens for the base VR display. A ring of infrared LEDs illuminates the part of the face covered by the headset. The mirror is reflective of infrared light and sits between the lens and the screen so infrared cameras can view the eye from nearly head-on. This keeps the eye imaging system from distracting the user away from the VR display. The camera pairing produces the model of a face and projects the eye’s view.
The headset has a microlens array light field display that is thin and simple to construct based on existing consumer LCD tech. The display’s perspective shifts naturally which allows people in the room to see the correct perspective for their given location.
The headset is still in the experimental phase and has limitations. Currently, it is cumbersome and not a standalone headset because it is tethered by cables. The eye and facial renderings are not completely lifelike and the viewing angle is too severe. The prototype can show objects in sharp focus, but only within a few centimeters of the physical screen’s surface. The team notes that the display will work better as VR headsets shrink over time and with more development the eyes will become more realistic.
The team presented this tech at SIGGRAPH 2021.
