Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed a machine learning system that can recreate the steps artists took to create a painting.
The system, called Timecraft, creates time-lapse videos that show how the paintings were most likely to have been painted by the original artists. Timecraft was trained on more than 200 existing time-lapse videos that people posted online of both digital and watercolor paintings. The team then created a neural network that can look at a new painting it has never seen before and figure out how it was most likely created.
Timecraft was then used to generate videos of paintings that have real time-lapse videos and then conducted an online survey asking which videos seemed to represent the most realistic painting. Timecraft outperformed existing benchmarks by more than 90% of the time and was confused by the real videos about half the time.
The time lapses show that painters work in a coarse-to-fine manner, starting with “the big picture” before filling in details. The AI can also paint within one single section of a scene, using only one or two colors at a time.
Learn more about the research with MIT CSAIL’s research paper.