CATS: cache-aware task scheduling for Hadoop-based systems

Lim, Byungnam; Kim, Jong Wook; Chung, Yon Dohn

doi:10.1007/s10586-017-0920-6

Cited by 4 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 2017, Cache Aware Task Scheduling (CATS) [14] was introduced to exploit the operating system's buffer cache and take into account cached data in scheduling decisions. This technique includes two phases: a buffer cache probe, and a task scheduler.…”

Section: Related Workmentioning

confidence: 99%

CLQLMRS: improving cache locality in MapReduce job scheduling using Q-learning

et al. 2022

View full text Add to dashboard Cite

Scheduling of MapReduce jobs is an integral part of Hadoop and effective job scheduling has a direct impact on Hadoop performance. Data locality is one of the most important factors to be considered in order to improve efficiency, as it affects data transmission through the system. A number of researchers have suggested approaches for improving data locality, but few have considered cache locality. In this paper, we present a state-of-the-art job scheduler, CLQLMRS (Cache Locality with Q-Learning in MapReduce Scheduler) for improving both data locality and cache locality using reinforcement learning. The proposed algorithm is evaluated by various experiments in a heterogeneous environment. Experimental results show significantly decreased execution time compared with FIFO, Delay, and the Adaptive Cache Local scheduler.

show abstract

Section: Related Workmentioning

confidence: 99%