Cross-Layer Optimization of Big Data Transfer Throughput and Energy Consumption

Tacchio, Luigi Di; Nine, S Q Zulkar; Kosar, Tevfik; Bulut, Muhammed Fatih; Hwang, Jinho

doi:10.1109/cloud.2019.00017

Cited by 5 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experimental results in Figure 2 show how our proposed decision tree-based max throughput and min energy algorithms compare to Di Tacchio's algorithm implemented in [29]. Di Tacchio et al developed real-time tuning heuristics to optimize throughput and minimize energy consumption by tuning both application-level data transfer parameters and kernel-layer parameters during HTTP transfers.…”

Section: E Ementioning

confidence: 99%

“…Kim et al [38] considered the combined effect of parallelism, pipelining, and concurrency on end-to-end data transfer throughput. In our prior work [29] and [39], we developed a combination of model-free and empirical algorithms to maximize throughput by utilizing both machine learning techniques and online dynamic parameter tuning. This paper extends our prior work by using a novel decision treebased model capable of dealing with the aleatoric uncertainty presented in the historical logs.…”

Section: R Wmentioning

confidence: 99%

See 1 more Smart Citation

Energy-Efficient Data Transfer Optimization via Decision-Tree Based Uncertainty Reduction

Hasibul¹,

Rodolph²,

Goldverg³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The increase and rapid growth of data produced by scientific instruments, the Internet of Things (IoT), and social media is causing data transfer performance and resource consumption to garner much attention in the research community. The network infrastructure and end systems that enable this extensive data movement use a substantial amount of electricity, measured in terawatt-hours per year. Managing energy consumption within the core networking infrastructure is an active research area, but there is a limited amount of work on reducing power consumption at the end systems during active data transfers. This paper presents a novel two-phase dynamic throughput and energy optimization model that utilizes an offline decision-search-tree based clustering technique to encapsulate and categorize historical data transfer log information and an online search optimization algorithm to find the best application and kernel layer parameter combination to maximize the achieved data transfer throughput while minimizing the energy consumption. Our model also incorporates an ensemble method to reduce aleatoric uncertainty in finding optimal application and kernel layer parameters during the offline analysis phase. The experimental evaluation results show that our decision-tree based model outperforms the state-of-the-art solutions in this area by achieving 117% higher throughput on average and also consuming 19% less energy at the end systems during active data transfers.

show abstract

Section: E Ementioning

confidence: 99%