EVEREST: A design environment for extreme-scale big data analytics on heterogeneous platforms

Abstract

High-Performance Big Data Analytics (HPDA) applications are characterized by huge volumes of distributed and heterogeneous data that require efficient computation for knowledge extraction and decision making. Designers are moving towards a tight integration of computing systems combining HPC, Cloud, and IoT solutions with artificial intelligence (AI). Matching the application and data requirements with the characteristics of the underlying hardware is a key element to improve the predictions thanks to high performance and better use of resources.We present EVEREST, a novel H2020 project started on October 1st, 2020 that aims at developing a holistic environment for the co-design of HPDA applications on heterogeneous, distributed, and secure platforms. EVEREST focuses on programmability issues through a data-driven design approach, the use of hardwareaccelerated AI, and an efficient runtime monitoring with virtualization support. In the different stages, EVEREST combines state-of-the-art programming models, emerging communication standards, and novel domain-specific extensions. We describe the EVEREST approach and the use cases that drive our research.

Monitoring performance computing environments and autoscaling using AI

“…Sentiment analysis may be used to examine how people or groups feel about other people, products, services, or social activities. Thanks to advances in deep learning, the abundance of information available online (particularly on social media), as well as rapid processing equipment, AI structures will eventually penetrate every aspect of human life and inspire us to think more profoundly on our own lives [5] . Cloud computing, enormous amounts of data, data centres, VR/AR, 5G, AI, the IoT, and optical fibre sensing are some of the newer sectors that have arisen in recent years.…”

Dynamically-Tunable Dataflow Architectures Based on Markov Queuing Models

“…To enable runtime management, as described previously in the paper, the controller has to maintain and dynamically evaluate the expected average response time. If we consider that the job arrival times can be modeled as a continuous-time Markov process, and, in particular, job interarrival times are exponentially distributed with the mean λ = 1, we can produce a prediction model for R by modeling the problem as an M/D/1 Markov process, i.e., arrivals are determined by a Poisson process (M), job service times are deterministic (D), and there is a single resource service station (1).…”

“…Modern applications require the elaboration of massive amounts of data, e.g., in realtime video streaming for entertainment or surveillance applications, or network communications [1,2]. To achieve high performance, such applications demand heterogeneous System-on-Chip (SoC) architectures with specialized hardware components.…”

“…When applied to hardware, approximation can either save resources (i.e., less logic is used to perform the same computation) or improve the performance (i.e., some computation can be executed faster, improving the hardware microarchitecture or performing the operations in a different way). While many software approximation techniques can be easily applied to hardware accelerators (e.g., variable-to-constant optimizations [9]), multi-variant hardware systems are more difficult to be designed since they need to (1) design efficient hardware modules able to support all variants and (2) detect the proper variant efficiently and correctly based on the given workload.…”

Automatic Creation of High-Bandwidth Memory Architectures from Domain-Specific Languages: The Case of Computational Fluid Dynamics

“…Additionally, designers can use high-level synthesis (HLS) tools to raise the abstraction level of hardware design [26,32]. However, designing efficient architectures for such systems is complex as it requires a concurrent optimization of communication, computation, and storage [34]. These optimizations may be limited by platform constraints, like the physical architecture, which can make the routing stages more difficult, or the number of physical resources, which can limit the number of parallel executions.…”