With progressive generations and the ever-increasing promise of computing power, GPGPUs have been quickly growing in size, and at the same time, energy consumption has become a major bottleneck for them. The first level data cache and the scratchpad memory are critical to the performance of a GPGPU, but they are extremely energy inefficient due to the large number of cores they need to serve. This problem could be mitigated by introducing a cache higher up in hierarchy that services fewer cores, but this introduces cache coherency issues that may become very significant, especially for a GPGPU with hundreds of thousands of in-flight threads.In this paper, we propose adding incoherent tinyCaches between each lane in an SM, and the first level data cache that is currently shared by all the lanes in an SM. In a normal multiprocessor, this would require hardware cache coherence between all the SM lanes capable of handling hundreds of thousands of threads. Our incoherent tinyCache architecture exploits certain unique features of the CUDA/OpenCL programming model to avoid complex coherence schemes. This tinyCache is able to filter out 62% of memory requests that would otherwise need to be serviced by the DL1G, and almost 81% of scratchpad memory requests, allowing us to achieve a 37% energy reduction in the on-chip memory hierarchy. We evaluate the tinyCache for different memory patterns and show that it is beneficial in most cases.

Near Threshold Computing (NTC) has the potential to significantly improve efficiency in high throughput architectures like GPGPUs. Nevertheless, NTC is more sensitive to process variation (PV) as it complicates power delivery. We propose GPU Stacking, a novel method based on voltage stacking, to manage the effects of PV and improve the power delivery simultaneously. To evaluate our methodology, we first explore the design space of GPGPUs in the NTC to find a suitable baseline configuration and then apply GPU Stacking to mitigate the effects of PV. When comparing with an equivalent NTC GPGPU without process variation management, we achieve 37% more performance on average. When considering high production volume, our approach shifts all the chips closer to the nominal non-process variation case, delivering on average (across chips) ≈ 80% of the performance of nominal NTC GPGPU, whereas when not using our technique, chips would have ≈ 50% of the nominal performance. We also show that our approach can be applied on top of multi-frequency domain designs, improving the overall performance.