ZAKI+: A Machine Learning Based Process Mapping Tool for SpMV Computations on Distributed Memory Architectures

Abstract

Smart cities and other cyber-physical systems (CPSs) rely on various scientific, engineering, business, and social applications that provide timely intelligence for their design, operations, and management. Many of these scientific and analytics applications require the solution of sparse linear equation systems, where sparse matrix-vector (SpMV) product is a key computing operation. Several factors determine the performance of parallel SpMV computations, including matrix characteristics, storage formats, and the rising complexity and heterogeneity of computer systems. There is a pressing need for new ways of exploiting parallelism, and mapping data and applications to the computing resources. We propose here ZAKI+ , a data-driven machine-learning approach, allowing users to automatically, effortlessly, and speedily obtain the best configuration (the data distribution, the optimal number of processes, and mapping strategy) and performance for the execution of the parallel SpMV computations on distributed memory machines. We train and test the tool using three machine learning methods-decision trees, random forest, and Xtreme boosting-and nearly 2000 real-world matrices obtained from 45 application domains, including computer vision and robotics. ZAKI+ provides optimal process mapping and outperforms the MPI default mapping policy by a factor of 4.24. This is the first work where the sparsity structure of matrices has been exploited to predict the optimal mapping of processes and data in distributed-memory environments by using different base and ensemble machine learning methods. Various CPSs comprise compute-intensive machine learning applications, such as the SpMV, and hence, the process and data mapping contributions of this paper would be of paramount impact for the CPSs. INDEX TERMS Cyber-physical systems, SpMV, sparse linear algebra, sparse matrices, machine learning, MPI, process affinity, compressed sparse row (CSR), decision trees, random forest, Xtreme boosting, parallel computing, high performance computing (HPC), smart cities analytics, exascale systems, OpenMPI. I. INTRODUCTION Cyber-Physical Systems (CPS) comprises ''interacting digital, analog, physical, and human components engineered for function through integrated physics and logic. These systems will provide the foundation of our critical infrastructure, form the basis of emerging and future smart services, and improve our quality of life in many areas'' [1], including transportation, healthcare, smart grid, disaster management and many other areas. A CPS uses Internet of Things (IoT), The associate editor coordinating the review of this manuscript and approving it for publication was Wei Yu.

COVID-19: Detecting Government Pandemic Measures and Public Concerns from Twitter Arabic Data Using Distributed Machine Learning

“…For larger datasets, executing sequential codes may not even be possible, or distributed computing could save years of development time. How to select the number of cores for a given job that could save experimental time and energy itself is a challenge and has been addressed in our other works [ 78 , 79 ].…”

Can Building “Artificially Intelligent Cities” Safeguard Humanity from Natural Disasters, Pandemics, and Other Catastrophes? An Urban Scholar’s Perspective

“…Additionally, machine learning has been applied along with distributed computing to improve basic scientific computing operations that are fundamental to urban design modeling methodologies [87]. Moreover, machine learning is paired with IoT for human activity recognition [88], smart farming [89], and developing next-generation distance learning systems [90].…”

Sehaa: A Big Data Analytics Tool for Healthcare Symptoms and Diseases Detection Using Twitter, Apache Spark, and Machine Learning

“…A range of technologies is contributing to the development of these smart systems. These include the Internet of Things (IoT) [32][33][34][35][36], social media [21][22][23]37,38], big data [39][40][41][42][43][44], high performance computing (HPC) [45][46][47][48], cloud, fog, and edge computing [34,[49][50][51][52], and machine learning [36,53]. The applications include healthcare [34,39,[54][55][56], transportation [57,58], and others [59,60].…”