Acquired Electronics360

Audio and Video

Deep Learning Removes Noise from Big Screen Animation Images

26 July 2017

Computer generated image and scenes are common place in modern films and TV shows. These images are computed by rendering systems that simulate the flow of light in a three-dimensional scene and convert the information into a two-dimensional image.

Computing the thousands of light rays per frame to achieve accurate color, shadows, reflectivity and other light-based characteristics is very labor-intensive, time-consuming and expensive undertaking. An alternative to this process is rendering the images using only a few light rays. While that saves time and labor, it can result in inaccuracies that show up as “noise” in the final image.

Researchers at UC Santa Barbra are working on a solution to this problem. Steve Bako, an electrical and computer engineering Ph.D. student, and Pradeep Sen, Bako’s advisor, have worked with a team at Disney Research and Pixar Animation Studios to develop a technology based on artificial intelligence (AI) and deep learning. This technology eliminates the noise and enables production-quality rendering at higher speeds.

The deep learning technology that was developed searched for and erased noise from the films "Finding Dory" and "Cars 3" (Disney, Pixar)The deep learning technology that was developed searched for and erased noise from the films "Finding Dory" and "Cars 3" (Disney, Pixar)

Bako worked with Pixar for a year. His team tested the software by using millions of examples from the Pixar film “Finding Dory” to train a deep-learning model called a convolutional neural network. The system learned to transform noisy images into noise-free versions that resemble the ones computed with more light rays through this process. When the system was properly trained, it successfully removed the noise on test images from different films, like Pixar’s latest film “Cars 3” and an upcoming film “Coco” even though they had completely different styles and color palettes.

"Noise is a really big problem for production rendering," said Tony DeRose, head of research at Pixar. "This new technology allows us to automatically remove the noise while preserving the detail in our scenes."

This research is a big step forward over previous state-of-the-art denoising methods. Those methods often left artifacts or residual noise that requires artists to render more light rays or tweak the denoising filter to improve the quality of an image. Disney and Pixar are planning to incorporate the technology in their production pipelines to accelerate the movie-making process.

"Other approaches for removing image noise have grown increasingly complex, with diminishing returns," said Markus Gross, vice president for research at Disney Research. "By leveraging deep learning, this work presents an important step forward for removing undesirable artifacts from animated films."

To further explore this, the team will make their code and the deep learning model’s trained weights are available to the entire research community.

To contact the author of this article, email Siobhan.Treacy@ieeeglobalspec.com


Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the Engineering360
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Advertisement
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter
Advertisement

CALENDAR OF EVENTS

Date Event Location
25-27 Sep 2017 Las Vegas, NV
02 Oct 2017 Barcelona, Spain
30 Nov-01 Dec 2017 Helsinki, Finland
23-27 Apr 2018 Oklahoma City, Oklahoma
Find Free Electronics Datasheets
Advertisement