The Performance Wall of Large Parallel Computing Systems

The Need for Modern Computing Paradigm: Science Applied to Computing

“…As Figure 4 shows (and discussed in details in [8]), the deviation from the linear dependence is unnoticeable at low performance values: here the "classic speed addition" is valid. At extremely large performance values, however, the dependence is strongly non-linear and specific on the measurement conditions: here the performance limits manifest.…”

“…increases linearly and the performance decreases exponentially with the number of cores, at some critical value where an inflection point occurs, the resulting performance starts to decrease. The resulting large non-parallelizable fraction strongly decreases the efficacy (or in other words: the performance gain or speedup) of the system [8], [39]. The effect was noticed early [22], under different technical conditions but forgotten due to the successes of the development of the parallelization technology.…”

“…This phenomenon cannot be explained in the frames of the "classic computing". The limits of single-processor performance enforced by the laws of nature [7] are topped by the limitation of parallel computing [8], [31], and further limited through introducing the "biological clock period" [32]. Notice that these contributions are competing with each other, the actual circumstances decide which of them will dominate.…”

“…• as a new QT receives a new Processing Unit (PU)(s), there is no need to save/restore registers and return address (less memory utilization and less instruction cycles) • the OS can receive its own PU, which is initialized in kernel mode and can promptly (i.e. without the need of context change) service the requests from the requestor core • for resource sharing, temporarily a PU can be delegated to protect the critical section; the next call to run the code fragment with the same offset will be delayed until the processing by the first PU terminates • the processor can natively accommodate to the variable need of parallelization • the actually out-of-use cores are waiting in low energy consumption mode • the hierarchic core-to-core communication greatly increases the memory throughput • the asynchronous-style computing [57] largely reduces the loss due to the gap [58] between speed of the processor and that of the memory • the direct core-to-core connection (more dynamic than in [46]) greatly enhances efficacy in large systems [59] • the thread-like feature to fork() and the hierarchic buses change the dependence of on the number of cores from linear to logarithmic [8] (enables to build really exa-scale supercomputers) The very first version of EMPA [11] has been implemented in a form of simple (practically untimed) simulator [60], now an advanced (Transaction Level Modelled) simulator is prepared in SystemC. The initial version adapted Y86 cores [61], the new one RISC-V cores.…”

“…Not only that in extreme conditions quickly reaches its performance limit [6], [7], but this performance degradation is even more accelerated in systems comprising parallelized sequential processors [8]. Consequently, in real-time multitasking systems can be encountered the following situations: and the systems comprising parallelized sequential processors also introduce their own limits [8].…”