Time-series ML-regression on Graphcore IPU-M2000 and Nvidia A100

Balewski, J.; Liu, Zhenying; Tsyplikhin, Alexander; Roland, Manuel Lopez; Bouchard, Kristofer E.

doi:10.1109/pmbs56514.2022.00019

Cited by 1 publication

(4 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The study by Balewski et al [2] compares time-series ML-based regression on IPUs against GPUs. They conduct a comparison of training performance between an M2000 system and an NVIDIA A100 GPU system, including multiple convolutional and fullyconnected models with batch normalization.…”

Section: Related Work On the Ipumentioning

confidence: 99%

“…separated into two halves by some permutation (2) Therefore, equation 1 is a special case of 2 where…”

Section: 31mentioning

confidence: 99%

“…We follow the methodology of [14] by using a single-hidden layer (SHL) neural architecture on the CIFAR10 image classification task, and employ a variety of methods for sparsity. In more detail, we vary the mini-batch size as well to allow a wider range of comparisons 2 . The experiments are conducted by iterating 1000 times and the average execution time is used to avoid warm-up issues and to stabilize the results.…”

Section: Cifar-10 Image Classificationmentioning

confidence: 99%

“…Unlike previous experiments, we incorporate a single iteration, due to the overhead of profiling and unpredictable IPU-Fabric errors resulting from slurm integration issues 2. In contrast, in[4] the mini-batch size is set to 2 8 .…”

mentioning

confidence: 99%

See 3 more Smart Citations

Reducing Memory Requirements for the IPU using Butterfly Factorizations

Shekofteh,

Alles,

Fröning

2023

Proceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analys

View full text Add to dashboard Cite

High Performance Computing (HPC) benefits from different improvements during last decades, specially in terms of hardware platforms to provide more processing power while maintaining the power consumption at a reasonable level. The Intelligence Processing Unit (IPU) is a new type of massively parallel processor, designed to speedup parallel computations with huge number of processing cores and on-chip memory components connected with high-speed fabrics. IPUs mainly target machine learning applications, however, due to the architectural differences between GPUs and IPUs, especially significantly less memory capacity on an IPU, methods for reducing model size by sparsification have to be considered. Butterfly factorizations are well-known replacements for fully-connected and convolutional layers. In this paper, we examine how butterfly structures can be implemented on an IPU and studiy their behavior and performance compared to a GPU. Experimental results indicate that these methods can provide 98.5% compression ratio to decrease the immense need for memory, the IPU implementation can benefit from 1.3x and 1.6x performance improvement for butterfly and pixelated butterfly, respoectively. We also reach to 1.62x training time speedup on a real-word dataset such as CIFAR10.

show abstract

Section: Related Work On the Ipumentioning

confidence: 99%

“…separated into two halves by some permutation (2) Therefore, equation 1 is a special case of 2 where…”

Section: 31mentioning

confidence: 99%

Section: Cifar-10 Image Classificationmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations