Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures 2020
DOI: 10.1145/3350755.3400267
|View full text |Cite
|
Sign up to set email alerts
|

Communication Lower Bounds of Convolutions in CNNs

Abstract: Convolution is the most time-consuming part in the computation of convolutional neural networks (CNNs), which have achieved great successes in numerous practical applications. Due to the complex data dependency and the increase in the amount of model samples, the convolution suffers from high overhead on data movement (i.e., memory access). This work provides comprehensive analysis and methodologies to minimize the communication for the convolution in CNNs. With an in-depth analysis of the recent I/O complexit… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1

Citation Types

0
1
0

Year Published

2020
2020
2024
2024

Publication Types

Select...
2
1

Relationship

1
2

Authors

Journals

citations
Cited by 3 publications
(1 citation statement)
references
References 24 publications
0
1
0
Order By: Relevance
“…As the recent methodology mainly focuses on the workflow's specific properties which do not translate across different computational patterns, the recent lower bound theory seems hard to be applied to arbitrary computations such as convolutions, in which different sub-computations involve different computational patterns. How to establish a systematic I/O lower bound theory for convolutions based on the red-blue pebble game model is a big challenge [32]. Even if the lower bounds could be obtained, the theoretical minimum of I/O complexity is not easy to directly yield an efficient dataflow strategy.…”
Section: Introductionmentioning
confidence: 99%
“…As the recent methodology mainly focuses on the workflow's specific properties which do not translate across different computational patterns, the recent lower bound theory seems hard to be applied to arbitrary computations such as convolutions, in which different sub-computations involve different computational patterns. How to establish a systematic I/O lower bound theory for convolutions based on the red-blue pebble game model is a big challenge [32]. Even if the lower bounds could be obtained, the theoretical minimum of I/O complexity is not easy to directly yield an efficient dataflow strategy.…”
Section: Introductionmentioning
confidence: 99%