A new technique based on hashing drastically reduces the time and computation required for deep learning computation. Applications that rely on massive amounts of data—from self-driving cars to language translation—could take advantage of this breakthrough method.
Rice University assistant professor of computer science Anshumali Shrivastava and graduate student Ryan 
Ryan Spring and Anshumali Shrivastava. Credit: Jeff Fitlow/Rice UniversitySpring adapted hashing, a standard method for dealing with vast amounts of data, to reduce computational overhead for deep learning. Their technique combines a variant of locally-sensitive hashing and backpropagation. In small-scale tests, the researchers reduced computation as much as ninety-five percent with results within one percent of established accuracy.
Deep-learning networks consist of mathematical functions called artificial neurons. Adding neurons to a network creates different levels of neuron, with each level specializing in a particular function. Increasing the size of the network concomitantly increases its complexity and the time and energy needed to perform computations.
According to Shrivastava, Google is trying to train a 137-billion-neuron network. However, limits to the amount of available computational power restrict the size of neural networks. The Rice researchers’ technique could make such a large network feasible.
“Most machine-learning algorithms in use today were developed 30-50 years ago,” he said. “They were not designed with computational complexity in mind. But with ‘big data,’ there are fundamental limits on resources like compute cycles, energy and memory. Our lab focuses on addressing those limitations.”
Based on the team’s calculations, computation and energy savings should be larger on larger networks. They will present their results at the KDD2017 conference in Halifax, Nova Scotia, this August.
