Researchers from MIT have created a way to lower the carbon footprint of artificial intelligence (AI) systems.
The amount of power required in training and searching for AI has emissions of around 626,000 lb of carbon dioxide, or five times the lifetime emissions of the average U.S. car. Emissions become more severe in the model deployment phase. As such, deep neural networks need to be deployed on diverse hardware platforms with different properties and computational resources.
MIT’s new system, called a once-for-all (OFA) network, trains and runs certain neural networks, improving the computational efficiency of the system in key ways and cutting down on pounds of carbon emissions.
The OFA network trains one large neural network, which is comprised of many pre-trained subnetworks of different sizes. The subnetworks can be tailored to diverse hardware platforms without retraining. This dramatically reduces the energy required to train neural networks for new platforms.
With this system, a model could be trained using roughly 1/1,000th of the carbon emissions of a traditionally trained system, thereby reducing the inference time. This work was carried out on Satori, an efficient computing cluster that was donated to MIT by IBM. Satori is capable of performing two quadrillion calculations per second.
The team used AutoML (automatic machine learning) with the new advancements. AutoML eliminates manual network design. With AutoML, neural networks automatically search massive design spaces for network architectures tailored to specific hardware platforms. But AutoML has a training efficiency issue. Each model must be selected then trained from scratch for its platform architecture. That is where the new system comes in.
The team created an AutoML system that trains a single, large, OFA network that acts as the mother network. OFA shares learned weights with the subnetworks, pretraining them for tasks. Each subnetwork can operate independently at the inference time without retraining. The team also trained an OFA convolutional neural network (CNN) with versatile architecture configurations. This included various layers and neurons, diverse filter sizes and diverse input image resolutions.
When given a specific platform, the system uses the OFA as the search space to find the best subnetwork for a job. The OFA decouples model training and architecture search while spreading the one-time training cost over many interface hardware platforms.
The new method relies on a progressive shrinking algorithm that trains the OFA network to support all subnetworks at the same time. The full network is trained on the maximum size then shrinks the sizes to include smaller networks. It supports many hardware devices with zero training costs when adding a new device.
OFA can include more than 10 quintillion architectural settings, but does not compromise accuracy and interference efficiency.
A paper on this research was published in ICLR 2020 Conference Program Chairs.
