Runtime and memory consumption analyses for machine learning R programs

Kotthaus, Helena; Korb, Ingo; Lang, Michel; Bischl, Bernd; Rahnenführer, Jörg; Marwedel, Peter

doi:10.1080/00949655.2014.925192

Cited by 18 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of those machine learning classification algorithms is based on both the method's popularity and the availability of an implementation. These benchmarks were also used in [9] and are publicly available [10]. Most of the listed algorithms allow the user to adjust several parameters to increase predictive performance.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic page sharing optimization for the R language

Kotthaus

Korb

Engel

et al. 2014

Proceedings of the 10th ACM Symposium on Dynamic Languages

Self Cite

View full text Add to dashboard Cite

Dynamic languages such as R are increasingly used to process large data sets. Here, the R interpreter induces a large memory overhead due to wasteful memory allocation policies. If an application's working set exceeds the available physical memory, the OS starts to swap, resulting in slowdowns of a several orders of magnitude. Thus, memory optimizations for R will be beneficial to many applications.Existing R optimizations are mostly based on dynamic compilation or native libraries. Both methods are futile when the OS starts to page out memory. So far, only a few, data-type or application specific memory optimizations for R exist. To remedy this situation, we present a low-overhead page sharing approach for R that significantly reduces the interpreter's memory overhead. Concentrating on the most rewarding optimizations avoids the high runtime overhead of existing generic approaches for memory deduplication or compression. In addition, by applying knowledge of interpreter data structures and memory allocation patterns, our approach is not constrained to specific R applications and is transparent to the R interpreter.Our page sharing optimization enables us to reduce the memory consumption by up to 53.5% with an average of 18.0% for a set of real-world R benchmarks with a runtime overhead of only 5.3% on average. In cases where page I/O can be avoided, significant speedups are achieved.

show abstract

Section: Methodsmentioning

confidence: 99%

“…As shown by Morandat et al [8] and our previous work [9], the interpreter-based original R implementation has several drawbacks leading to slow and memory inefficient program execution.…”

Section: Related Workmentioning

confidence: 95%

Dynamic page sharing optimization for the R language

Kotthaus

Korb

Engel

et al. 2014

Proceedings of the 10th ACM Symposium on Dynamic Languages

Self Cite

View full text Add to dashboard Cite

show abstract

“…In particular, the work in [ 13 ] compared 293 classifiers using various sensor placements and window sizes, concluding on the superiority of k nearest neighbors ( k -NN) and pointing out a trade-off between runtime and classification performance. Resource consumption, including memory and runtime, was also studied for offline classifiers, such as in [ 14 ] for the particular case of the R programming language.…”

Section: Related Workmentioning

confidence: 99%

A Benchmark of Data Stream Classification for Human Activity Recognition on Connected Objects

Khannouz

Glatard

2020

Sensors

View full text Add to dashboard Cite

This paper evaluates data stream classifiers from the perspective of connected devices, focusing on the use case of Human Activity Recognition. We measure both the classification performance and resource consumption (runtime, memory, and power) of five usual stream classification algorithms, implemented in a consistent library, and applied to two real human activity datasets and three synthetic datasets. Regarding classification performance, the results show the overall superiority of the Hoeffding Tree, the Mondrian forest, and the Naïve Bayes classifiers over the Feedforward Neural Network and the Micro Cluster Nearest Neighbor classifiers on four datasets out of six, including the real ones. In addition, the Hoeffding Tree and—to some extent—the Micro Cluster Nearest Neighbor, are the only classifiers that can recover from a concept drift. Overall, the three leading classifiers still perform substantially worse than an offline classifier on the real datasets. Regarding resource consumption, the Hoeffding Tree and the Mondrian forest are the most memory intensive and have the longest runtime; however, no difference in power consumption is found between classifiers. We conclude that stream learning for Human Activity Recognition on connected objects is challenged by two factors which could lead to interesting future work: a high memory consumption and low F1 scores overall.

show abstract

“…In particular, the work in [14] compared 293 classifiers using various sensor placements and window sizes, concluding on the superiority of k nearest neighbors (k-NN) and pointing out a trade-off between runtime and classification performance. Resource consumption, including memory and runtime, was also studied for offline classifiers, such as in [18] for the particular case of the R programming language.…”

Section: B Offline and Data Stream Classifiers For Harmentioning

confidence: 99%

A benchmark of data stream classification for human activity recognition on connected objects

Khannouz¹,

Glatard²

2020

Preprint

View full text Add to dashboard Cite

This paper evaluates data stream classifiers from the perspective of connected devices, focusing on the use case of HAR. We measure both classification performance and resource consumption (runtime, memory, and power) of five usual stream classification algorithms, implemented in a consistent library, and applied to two real human activity datasets and to three synthetic datasets. Regarding classification performance, results show an overall superiority of the HT, the MF, and the NB classifiers over the FNN and the Micro Cluster Nearest Neighbor (MCNN) classifiers on 4 datasets out of 6, including the real ones. In addition, the HT, and to some extent MCNN, are the only classifiers that can recover from a concept drift. Overall, the three leading classifiers still perform substantially lower than an offline classifier on the real datasets. Regarding resource consumption, the HT and the MF are the most memory intensive and have the longest runtime, however, no difference in power consumption is found between classifiers. We conclude that stream learning for HAR on connected objects is challenged by two factors which could lead to interesting future work: a high memory consumption and low F1 scores overall.

show abstract

Runtime and memory consumption analyses for machine learning R programs

Cited by 18 publications

References 5 publications

Dynamic page sharing optimization for the R language

Dynamic page sharing optimization for the R language

A Benchmark of Data Stream Classification for Human Activity Recognition on Connected Objects

A benchmark of data stream classification for human activity recognition on connected objects

Contact Info

Product

Resources

About