Facebook researchers have created a way to turn photos from any smartphone device into a 3D image within seconds.
A 2D to 3D photo feature has been available on Facebook since 2018. At first, users had to capture photos on a phone with a dual-lens camera. With the new system, users can take 3D photos from the Facebook app.
The new algorithm automates the depth estimation of a 2D photo. The framework provides a practical approach to 3D photography without needing sophisticated skills. The real-time conversion from 2D input image to 3D only takes a few seconds and works with almost any photo.
The convolutional neural network (CNN) was trained on millions of pairs of public 3D images with their depth maps and leveraged mobile optimization techniques developed by Facebook’s AI system. The system also incorporates texture inpainting and geometry capture of the 2D images and converts it to 3D. The automated system generates images in seconds.
In the future, the team will be looking at creating machine learning methods that enable high-quality depth estimation for videos on mobile devices.
This research will be presented at SIGGRAPH 2020.