Mosaic

Ausavarungnirun, Rachata; Landgraf, Joshua; Miller, Vance; Ghose, Saugata; Gandhi, Jayneel; Rossbach, Christopher J.; Mutlu, Onur

doi:10.1145/3123939.3123975

Cited by 80 publications

(8 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also added TLB and MMU support to simulate unified memory. A two-level TLB design is used where each SM has its private L1 TLB and all SMs share an L2 TLB as in prior work [8,80,81,9,91]. A TLB miss triggers a page table walk upon a page fault; up to 64 concurrent page walkers are supported.…”

Section: Methodsmentioning

confidence: 99%

NUBA: Non-Uniform Bandwidth GPUs

Zhao¹,

Jahre

Tang

et al. 2023

Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems,

View full text Add to dashboard Cite

The parallel execution model of GPUs enables scaling to hundreds of thousands of threads, which is a key capability that many modern high-performance applications exploit. GPU vendors are hence increasing the compute and memory resources with every GPU generation Ð resulting in the need to efficiently stitch together a plethora of Symmetric Multiprocessors (SMs), Last-Level Cache (LLC) slices and memory controllers while maximizing bandwidth and keeping power consumption and design complexity in check. Conventional GPUs are Uniform Bandwidth Architectures (UBAs) as they provide equal bandwidth between all SMs and all LLC slices. UBA GPUs require a uniform high-bandwidth Network-on-Chip (NoC), and our key observation is that provisioning a NoC to match the LLC slice bandwidth incurs a hefty power and complexity overhead.We propose the Non-Uniform Bandwidth Architecture (NUBA), a GPU system architecture aimed at fully utilizing LLC slice bandwidth. A NUBA GPU consists of partitions that each feature a few SMs and LLC slices as well as a memory controller Ð hence exposing the complete LLC bandwidth to the SMs within a partition since they can be connected with point-to-point links Ð and a NoC between partitions Ð to enable access to remote data. Exploiting the potential of NUBA GPUs however requires carefully co-designing system software, the compiler and architectural policies. The critical system software component is our Local-And-Balanced (LAB) page placement policy which enables the GPU driver to place data in local partitions while avoiding load imbalance. Moreover, we propose Model-Driven Replication (MDR) which identifies readonly shared data with data-flow analysis at compile time. At run time, MDR leverages an architectural mechanism that replicates

show abstract

Section: Methodsmentioning

confidence: 99%

NUBA: Non-Uniform Bandwidth GPUs

Zhao¹,

Jahre

Tang

et al. 2023

Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems,

View full text Add to dashboard Cite

show abstract

“…GPU Memory Management. Many memory management optimizations have been proposed to pre-allocate most GPU memory and then manage the memory themselves, including paging [4], replacement caching [52] and memory pool [56]. Mosaic [4] provided application-transparent support for multiple page sizes to page-in and page-out.…”

Section: Related Workmentioning

confidence: 99%

“…Many memory management optimizations have been proposed to pre-allocate most GPU memory and then manage the memory themselves, including paging [4], replacement caching [52] and memory pool [56]. Mosaic [4] provided application-transparent support for multiple page sizes to page-in and page-out. MultiQx-GPU [52] designed a cost-driven replacement policy for efficient executions of concurrent queries in GPU databases.…”

Section: Related Workmentioning

confidence: 99%

Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent

Nie¹,

Liu²,

Fu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recent years have witnessed the unprecedented achievements of large-scale pre-trained models, especially the Transformer models. Many products and services in Tencent Inc., such as WeChat, QQ, and Tencent Advertisement, have been opted in to gain the power of pre-trained models. In this work, we present Angel-PTM, a productive deep learning system designed for pre-training and fine-tuning Transformer models. Angel-PTM can train extremely large-scale models with hierarchical memory efficiently. The key designs of Angel-PTM are the fine-grained memory management via the Page abstraction and a unified scheduling method that coordinate the computations, data movements, and communications. Furthermore, Angel-PTM supports extreme model scaling with SSD storage and implements the lock-free updating mechanism to address the SSD I/O bandwidth bottlenecks. Experimental results demonstrate that Angel-PTM outperforms existing systems by up to 114.8% in terms of maximum model scale as well as up to 88.9% in terms of training throughput. Additionally, experiments on GPT3-175B and T5-MoE-1.2T models utilizing hundreds of GPUs verify the strong scalability of Angel-PTM.

show abstract

“…This gap ( 7 , the green part in Figure 4) approximates the potential benefit of a huge page split. If the potential benefit is large, Memtis chooses huge pages with high access skew in their subpages as split candidates ( 8 ). Then, it splinters the huge pages in the background and places each split subpage into the appropriate memory tier by referring to subpage access information maintained in the huge page ( 9 ).…”

Section: Memtis Design Overviewmentioning

confidence: 99%

MEMTIS: Efficient Memory Tiering with Dynamic Page Classification and Page Size Determination

Lee,

Monga,

Min

et al. 2023

Proceedings of the 29th Symposium on Operating Systems Principles

View full text Add to dashboard Cite

The evergrowing memory demand fueled by datacenter workloads is the driving force behind new memory technology innovations (e.g., NVM, CXL). Tiered memory is a promising solution which harnesses such multiple memory types with varying capacity, latency, and cost characteristics in an effort to reduce server hardware costs while fulfilling memory demand. Prior works on memory tiering make suboptimal (often pathological) page placement decisions because they rely on various heuristics and static thresholds without considering overall memory access distribution. Also, deciding the appropriate page size for an application is difficult as huge pages are not always beneficial as a result of skewed accesses within them. We present Memtis, a tiered memory system that adopts informed decision-making for page placement and page size determination. Memtis leverages access distribution of allocated pages to optimally approximate the hot data set to the fast tier capacity. Moreover, Memtis dynamically determines the page size that allows applications to use huge pages while avoiding their drawbacks by detecting inefficient use of fast tier memory and splintering them if necessary. Our evaluation shows that Memtis outperforms state-of-the-art tiering systems by up to 169.0% and their best by up to 33.6%.CCS Concepts: • Software and its engineering → Memory management; • Computer systems organization → Heterogeneous (hybrid) systems.

show abstract

Mosaic

Cited by 80 publications

References 73 publications

NUBA: Non-Uniform Bandwidth GPUs

NUBA: Non-Uniform Bandwidth GPUs

Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent

MEMTIS: Efficient Memory Tiering with Dynamic Page Classification and Page Size Determination

Contact Info

Product

Resources

About