Reducing set-associative cache energy via way-prediction and selective direct-mapping

Abstract

Set-associative caches achieve low miss rates for typical applications but result in significant energy dissipation. Set-associative caches minimize access time by probing all the data ways in parallel with the tag lookup, although the output of only the matching way is used. The energy spent accessing the other ways is wasted. Eliminating the wasted energy by performing the data lookup sequentially following the tag lookup substantially increases cache access time, and is unacceptable for high-performance L1 caches. In this paper, we apply two previously-proposed techniques, way-prediction and selective direct-mapping, to reducing L1 cache dynamic energy while maintaining high performance. The techniques predict the matching way and probe only the predicted way and not all the ways, achieving energy savings. While these techniques were originally proposed to improve set-associative cache access times, this is the first paper to apply them to reducing cache energy. We evaluate the effectiveness of these techniques in reducing L1 d-cache, L1 i-cache, and overall processor energy. Using these techniques, our caches achieve the energy-delay of sequential access while maintaining the performance of parallel access. Relative to parallel access L1 i-and d-caches, the techniques achieve overall processor energy-delay reduction of 8%, while perfect way-prediction with no performance degradation achieves 10% reduction. The performance degradation of the techniques is less than 3%, compared to an aggressive, 1-cycle, 4-way, parallel access cache.

Recruiting Decay for Dynamic Power Reduction in Set-Associative Caches

“…These techniques also lead to power savings since they circumvent the associative search [12,13,17,30,31]. Way-prediction techniques aim to predict a single way where a reference might hit.…”

“…Some of the most sophisticated way-prediction techniques are those based on a combination of selective direct-mapping (DM) and way-prediction [13,31]. In this paper we compare against one such scheme called MMRU [31].…”

“…Furthermore, decay bits are typically embedded in the tag array, while we would like to have a separate array that we can access prior to the cache, similarly to other way-prediction structures [5,13,31]. We accomplish our goals using appropriately sized Bloom filters [3] instead of decay bits as our predictors.…”

“…1 Introduction dynamic power [13,30,14,31] but without a clear understanding of how to integrate them in a single implementation. We propose a new approach for reducing dynamic power in set-associative caches by exploiting a previous leakage-saving proposal.…”

“…To support inexpensive and power-efficient implementations of self-decaying Bloom filters we propose novel quasi-static cells designed to offer the necessary decay functionality for Bloom filters. We compare our proposal to a sophisticated and highly-accurate way-prediction scheme called Multi-MRU [31](of similar predictive power to other complex schemes [2,13]) and our results show:…”