Illustration of 36-tile Jenga system running four applications, giving each a custom virtual cache hierarchy. (Credit: Research group at MIT)Computer chips save time and energy by using “caches” – small, local memory banks that store frequently-used data, cutting down on communication with off-chip memory and increasing efficiency.
At least, that’s the way they’ve done it for decades.
As the needs of different programs have become more diversified, the cache structure has become more complex. Today’s chips have three or four different levels of cache, each more spacious -- but also slower -- than the last.
Enter researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), who have designed a new system to reallocate cache access on the fly. The system, called Jenga, creates “cache hierarchies” designed to suit the needs of particular programs.
A test of Jenga, using a simulated chip with 36 processing units (or “cores”), was successful in increasing processing speed 20-30 percent, while also reducing energy consumption 30-85 percent.
The researchers presented their system at the International Symposium on Computer Architecture in late June 2017.
For the past decade or so, processing power improvements have been achieved by adding more cores to the chips. Most desktop computers today have four cores, but several chipmakers have announced plans to move to six cores within the next year or so, and industry watchers presume that the core count will continue to climb. Already, 16-core processors are not uncommon in high-end servers.
Within a multicore chip, two levels of “private” cache are typically built into each core. A third cache, broken into discrete memory banks scattered around the chip, is shared by all the cores. Some newer chips also include a fourth cache, called DRAM, etched into a second chip mounted on top of the first. For each core, accessing the cache within the closest physical distance maximizes efficiency. The amount of time it takes to retrieve information is referred to as latency.
That’s where Jenga comes in.
By distinguishing between the physical locations of the separate memory banks and calculating latency-space curves for every task, it can determine a space allocation that minimizes latency for the chip as a whole. The system builds on a previous system built by the researchers, called Jigsaw, which also allocated cache access on the fly but was unable to build cache hierarchies. A sampling algorithm built on top of Jigsaw makes it possible for Jenga to update memory allocations every 100 milliseconds and adjust priorities.
“There’s been a lot of work over the years on the right way to design a cache hierarchy,” said David Wood, a professor of computer science at the University of Wisconsin at Madison. “Jenga is different in that it really uses the software to try to characterize what the workload is and then do an optimal allocation of the resources between the competing processes. And that, I think, is fundamentally more powerful than what people have been doing before.”
