TV shows and movies today often have computer-generated sequences that are made by rendering systems that simulate light flow in a 3-D scene. Computing a lot of light rays is very labor-intensive and takes a long time. The current alternative to this process is to produce these images with only a few light rays. But this method is a shortcut and often results in inaccuracies that show up as objectionable noise in the final image.
Researchers at Disney Research, Pixar Animation Studios and the University of California Santa Barbara are working on developing a new technology to solve this problem. This new technology is based on artificial intelligence (AI) and deep learning that eliminates the objectionable noise and allows production-quality rendering at much faster speeds.
The team used millions of examples from Finding Dory, a recent Pixar film, to train a deep learning model, known as Convolutional Neural Network. The system learned to transform the noisy images with few light rays into images that were noise-free images resembling images that were computed with more light rays. When the system was trained, it was able to successfully remove noise on test images from entirely different films, for example, “Cars 3” and an upcoming film called “Coco,” despite these two films having completely different styles and color palettes.
"Noise is a really big problem for production rendering," said Tony DeRose, head of research at Pixar. "This new technology allows us to automatically remove the noise while preserving the detail in our scenes."
This research has taken a significant step forward for the movie industry. Current state-of-the-art denoising methods leave artifacts or residual noise that artists either have to render more light rays or tweak the denoising filter to improve image quality. Disney and Pixar are planning to incorporate the new AI technology into their production pipelines in order to accelerate the movie-making process.
"Other approaches for removing image noise have grown increasingly complex, with diminishing returns," said Markus Gross, vice president for research at Disney Research. "By leveraging deep learning, this work presents an important step forward for removing undesirable artifacts from animated films."
This new research will be presented in July at the ACM SIGGRAPH 2017 conference. The team will make their code and trained weights available to the research community in order to facilitate further exploration of this new movie method.