Compute Substrate for Software 2.0

Vasiljevic, Jasmina; Bajic, Ljubisa; Capalija, Davor; Sokorac, Stanislav; Ignjatovic, Dragoljub; Bajic, Lejla; Trajković, Miloš; Hamer, Ivan; Matosevic, Ivan; Cejkov, Aleksandar; Aydonat, Utku; Zhou, Tony; Gilani, Syed Zohaib; Paiva, Armond; Chu, Joseph Lin; Maksimovic, Djordje; Chin, Stephen; Moudallal, Zahi; Rakhmati, Akhmed; Nijjar, Sean; Bhullar, Almeet; Drazic, Boris; Lee, Charles; Sun, James; Kwong, Kei-Ming; Connolly, J.E.; Dooley, Miles; Farooq, Hassan; Chen, Joy Yu Ting; Walker, Matthew J.; Dabiri, Keivan; Mabee, Kyle; Lal, Rakesh Shaji; Rajatheva, Namal; Retnamma, Renjith; Karodi, Shripad; Rosen, Daniel; Muñoz, Emilio Ángel Villanueva; Lewycky, Andrew; Knežević, Aleksandar; Kim, Woojin; Rui, Allan; Drouillard, Alexander; Thompson, David

doi:10.1109/mm.2021.3061912

Cited by 15 publications

(11 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, most AI multi-core chips are commercial products and the architecture details are not available, for example, Cerebras WSE series [20] and Tenstorrent [21]. To our best knowledge, only Graphcore's Intelligent Processing Unit reveals that its parallel computing model [17] and only Tenstorrent claims the support of model-level parallelism [22].…”

Section: Upsurging Of Mimd Multi-core Acceleratorsmentioning

confidence: 99%

“…Uneven distribution of communication load in env groups will lead to a remarkable decrease in overall performance. A usual solution is shown in Figure 6, which pipelines upstream computing, communication, and downstream computing to maximize their overlap [22]. In Figure 6, C-1 means The 1st segment of the Conv in L1.…”

Section: Realizing Load Balance For High Hardware Utilizationmentioning

confidence: 99%

See 1 more Smart Citation

HASP: Hierarchical Asynchronous Parallelism for Multi-NN Tasks

Li¹,

Ma²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>The rapid development of deep learning has propelled many real-world artificial intelligence (AI) applications. Many of these applications integrate multiple neural network (multi-NN) models to cater to various functionalities. Although a number of multi-NN acceleration technologies have been explored, few can fully fulfill the flexibility and scalability required by emerging and diverse AI workloads, especially for mobile. Among these, homogeneous multi-core architectures have great potential to support multi-NN execution by leveraging decentralized parallelism and intrinsic scalability. However, the advantages of multi-core systems are underexploited due to the adoption of bulk synchronization parallelism (BSP), which is inefficient to meet the diversity of multi-NN workloads. This paper reports a hierarchical multi-core architecture with asynchronous parallelism to enhance multi-NN execution for higher performance and utilization. Hierarchical asynchronous parallel (HASP) is the theoretical foundation, which establishes a programmable and grouped dynamic synchronous-asynchronous framework for multi-NN acceleration. HASP can be implemented on a typical multi-core processor for multi-NN with minor modifications. We further developed a prototype chip to validate the hardware effectiveness of this design. A mapping strategy that combines spatial partitioning and temporal tuning is also developed, which allows the proposed architecture to promote resource utilization and throughput simultaneously.</p>

show abstract

Section: Upsurging Of Mimd Multi-core Acceleratorsmentioning

confidence: 99%

Section: Realizing Load Balance For High Hardware Utilizationmentioning

confidence: 99%

HASP: Hierarchical Asynchronous Parallelism for Multi-NN Tasks

Li¹,

Ma²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…As Deep Neural Networks (DNNs) tackle increasingly complex problems, their size and complexity grow rapidly, resulting in increased computing and storage demands [9], [10], [51], [56], [63]. While applying more advanced technology and enlarging single chip sizes have led to the development of many large-scale monolithic accelerators with tens of billions of transistors [23], [24], [28], [55], the end of Moore's Law [35] and limited photomask size pose significant challenges to further transistor integration. Chiplet technology, using advanced packaging to combine small functional dies, offers a promising solution to overcome these limitations and enable continuous transistor integration.…”

Section: Introductionmentioning

confidence: 99%

“…For the first challenge, maintaining high utilization and energy efficiency becomes increasingly difficult with the growing scale of accelerators. LP mapping, in which multiple layers are spatially mapped onto the accelerator, is widely employed by large-scale accelerators in both academia [15], [45], [46], [57] and industry [21], [52], [55] to relieve the challenge. The core of LP mapping is spatial mapping (SPM), which determines which part of which layer is allocated to which core, and significantly impacts the performance and energy efficiency of large-scale accelerators.…”

Section: Introductionmentioning

confidence: 99%

“…The core of LP mapping is spatial mapping (SPM), which determines which part of which layer is allocated to which core, and significantly impacts the performance and energy efficiency of large-scale accelerators. Despite the importance of LP SPM, most current strategies are still heuristic [15], [21], [46], [55], [57]. The problems and optimization space of LP SPM have not been clearly defined, exhaustively explored, or fully understood.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Airframe Damage Region Division Method Based on Structure Tensor Dynamic Operator

Cai

Shi

2022

J. Shanghai Jiaotong Univ. (Sci.)

View full text Add to dashboard Cite

Chiplet technology enables the integration of an increasing number of transistors on a single accelerator with higher yield in the post-Moore era, addressing the immense computational demands arising from rapid AI advancements. However, it also introduces more expensive packaging costs and costly Die-to-Die (D2D) interfaces, which require more area, consume higher power, and offer lower bandwidth than onchip interconnects. Maximizing the benefits and minimizing the drawbacks of chiplet technology is crucial for developing largescale DNN chiplet accelerators, which poses challenges to both architecture and mapping. Despite its importance in the post-Moore era, methods to address these challenges remain scarce.To bridge the gap, we first propose a layer-centric encoding method to encode Layer-Pipeline (LP) spatial mapping for largescale DNN inference accelerators and depict the optimization space of it. Based on it, we analyze the unexplored optimization opportunities within this space, which play a more crucial role in chiplet scenarios. Based on the encoding method and a highly configurable and universal hardware template, we propose an architecture and mapping co-exploration framework, Gemini, to explore the design and mapping space of large-scale DNN chiplet accelerators while taking monetary cost (MC), performance, and energy efficiency into account. Compared to the state-of-the-art (SOTA) Simba architecture with SOTA Tangram LP Mapping, Gemini's co-optimized architecture and mapping achieve, on average, 1.98× performance improvement and 1.41× energy efficiency improvement simultaneously across various DNNs and batch sizes, with only a 14.3% increase in monetary cost. Moreover, we leverage Gemini to uncover intriguing insights into the methods for utilizing chiplet technology in architecture design and mapping DNN workloads under chiplet scenarios. The Gemini framework is open-sourced at https://github.com/SET-Scheduling-Project/GEMINI-HPCA2024.

show abstract

NoC-based hardware software co-design framework for dataflow thread management

et al. 2023

View full text Add to dashboard Cite

Applications running in a large and complex manycore system can significantly benefit from adopting the dataflow model of computation. In a dataflow execution environment, a thread can run only if all its required inputs are available. While the potential benefits are large, it is not trivial to improve resource utilization and energy efficiency by focusing on dataflow thread execution models (i.e., the ways specifying how the threads adhering to a dataflow model of computation execute on a given compute/communication architecture). This paper proposes and implements a hardware-software co-design-based dataflow threads management framework. It works at the Network-on-Chip (NoC) level and consists of three stages. The first stage focuses on a fast and effective thread distribution policy. The next stage proposes an approach that adds reconfigurability to a 2D mesh NoC via customized instructions to manage the dataflow thread distribution. Finally, a 2D mesh and ring-based hybrid NoC is proposed for better scalability and higher performance. This work can be considered a primary reference framework from which extensions can be carried out.

show abstract

Compute Substrate for Software 2.0

Cited by 15 publications

References 2 publications

HASP: Hierarchical Asynchronous Parallelism for Multi-NN Tasks

HASP: Hierarchical Asynchronous Parallelism for Multi-NN Tasks

Airframe Damage Region Division Method Based on Structure Tensor Dynamic Operator

NoC-based hardware software co-design framework for dataflow thread management

Contact Info

Product

Resources

About