For years researchers have been pondering the idea that artificial intelligence (AI) may be able to understand and generate narratives. But can AI recognize a good story when it sees one?
Source: D4N13l3/DeviantArt
Scientists at Disney Research and the University of Massachusetts, Boston have taken steps towards solving this problem. The team has created neural networks that can evaluate short narratives. These AIs don’t mimic the professional judgment of literary critics, but the technology can predict which stories will appeal to a large population of readers.
"Our neural networks had some success in predicting the popularity of stories," said Boyang "Albert" Li, a research scientist at Disney Research. "You can't yet use them to pick out winners for your local writing competition, but they can be used to guide future research."
"The ability to predict narrative quality impacts on both story creation and story understanding," said Markus Gross, vice president at Disney Research. "To evaluate quality, the AI needs some level of understanding of the text. And if AIs are to create narratives, they need to be able to judge the quality of what they are producing."
The lack of large databases of stories has proved to be a stumbling block for training literary AIs, but the scientists found that Quora is a good data source. The answers on Quora are in the form of stories, and the AI measures a story’s popularity by the number of upvotes it has.
The researchers gathered almost 55,000 answers and developed an algorithm to classify them as stories or non-stories. They ended up yielding more than 28,000 stories with an average of 369 words.
The team set out to understand the stories' complex semantics. They looked for ways to represent the influence of story structures in neural networks because a sequence of events can interact to reveal character intentions.
The research developed an AI that evaluated separate regions of each story, including the question that prompted each. The team created a network that looked at the regions independently and another network that took a holistic view of how the meaning of the events and story regions emerged. In each case, the AIs made predictions of which texts would prove popular with readers.
In experiments, neural networks showed an 18 percent improvement over a baseline text evaluation system.
The team will present the findings August 23rd at the International Joint Conference on Artificial Intelligence in Melbourne, Australia. To learn more about this study, click here.
