Energy-aware very fast decision tree

Abstract

Recently machine learning researchers are designing algorithms that can run in embedded and mobile devices, which introduces additional constraints compared to traditional algorithm design approaches. One of these constraints is energy consumption, which directly translates to battery capacity for these devices. Streaming algorithms, such as the Very Fast Decision Tree (VFDT), are designed to run in such devices due to their high velocity and low memory requirements. However, they have not been designed with an energy efficiency focus. This paper addresses this challenge by presenting the nmin adaptation method, which reduces the energy consumption of the VFDT algorithm with only minor effects on accuracy. nmin adaptation allows the algorithm to grow faster in those branches where there is more confidence to create a split, and delays the split on the less confident branches. This removes unnecessary computations related to checking for splits but maintains similar levels of accuracy. We have conducted extensive experiments on 29 public datasets, showing that the VFDT with nmin adaptation consumes up to 31% less energy than the original VFDT, and up to 96% less energy than the CVFDT (VFDT adapted for concept drift scenarios), trading off up to 1.7 percent of accuracy.

A systematic review of Green AI

“…Some works propose small changes to the algorithms that make a big difference in the final energy consumption. For example, García‐Martín et al (2021) approximate the splitting criteria by selecting branches that require less computational effort. Results showcase decision trees that are up to 31% more energy efficient and with minimal impact on accuracy.…”

Investigating hardware and software aspects in the energy consumption of machine learning: A green AI‐centric analysis

“…However, results are focused on the strategies of mini-batch for the ensembles and not on the architecture or energy efficiency. Software approach: Some software approaches for energy savings are the investigation of computational requirements and their influence on energy consumption with code optimizations, the influence of the number of instances and features in the overall energy consumption, 26 identifying which part of the algorithm is consuming most of the energy (hotspots), 27 hyperparameters choice and its impact on energy. 28 Regarding the library, Holt & Sievert 29 Another aspect related to the algorithms that have emerged as a promising design alternative to better performance and energy efficiency in ML is Approximate Computing (AC).…”

“…Software approach: Some software approaches for energy savings are the investigation of computational requirements and their influence on energy consumption with code optimizations, the influence of the number of instances and features in the overall energy consumption, 26 identifying which part of the algorithm is consuming most of the energy (hotspots), 27 hyperparameters choice and its impact on energy 28 …”

Towards a sustainable artificial intelligence: A case study of energy efficiency in decision tree algorithms

“…Works [24][25][26][27] propose different approaches to reduce EC of the VFDT algorithm and advocate for energy efficiency as an essential metric in ML.…”

“…They have similarities with ours in the general proposal of reducing and estimating energy. In particular, such as assessing energy efficiency, 25,26 identifying hotspots in the codes, 24,27 and adjusting parameters. 24,26 So, concerning related works, our main contributions are: our work combines these different approaches and others to reduce EC in an extensive experimental set (focused on structured data).…”