Hard-ODT: Hardware-Friendly Online Decision Tree Learning Algorithm and System

Abstract

Decision trees are machine learning models commonly used in various application scenarios. In the era of big data, traditional decision tree induction algorithms are not suitable for learning large-scale datasets due to their stringent data storage requirement. Online decision tree learning algorithms have been devised to tackle this problem by concurrently training with incoming samples and providing inference results. However, even the most up-to-date online tree learning algorithms still suffer from either high memory usage or high computational intensity with dependency and long latency, making them challenging to implement in hardware. To overcome these difficulties, we introduce a new quantile-based algorithm to improve the induction of the Hoeffding tree, one of the state-of-the-art online learning models. The proposed algorithm is lightweight in terms of both memory and computational demand, while still maintaining high generalization ability. A series of optimization techniques dedicated to the proposed algorithm have been investigated from the hardware perspective, including coarse-grained and finegrained parallelism, dynamic and memory-based resource sharing, pipelining with data forwarding. Following this, we present Hard-ODT, a high-performance, hardware-efficient and scalable online decision tree learning system on a field-programmable gate array (FPGA) with system-level optimization techniques. Performance and resource utilization are modeled for the complete learning system for early and fast analysis of the tradeoff between various design metrics. Finally, we propose a design flow in which the proposed learning system is applied to FPGA run-time power monitoring as a case study. Experimental results show that our proposed algorithm outperforms the state-of-theart Hoeffding tree learning method, leading to 0.05% to 12.3% improvement in inference accuracy. Real implementation of the complete learning system on the FPGA demonstrates a 384× to 1581× speedup in execution time over the state-of-the-art design. The power modeling strategy with Hard-ODT achieves an average power prediction error within 4.93% of a commercial gate-level power estimation tool.

A Data Classifier Based on Maximum Likelihood Evidential Reasoning Rule

“…Benchmarks. In order to further verify the efectiveness of the newly proposed classifer, this section compares it with seven classic classifers, namely, decision trees [40], naive Bayes [6], k nearest neighbors (k-NN) [7], Bayes net [41], random forest [8], DC rule-based classifer (DC-core sample) [42], and the original ER classifer [43].…”

Research and Implementation of Real-time Render Optimization Algorithm Based on GPU

“…The regular mesh used in this case is simpler and more effective than the irregular mesh and has a stable rendering rate in practice. For example, terrain data is stored as a 2D height map in the framework designed by the algorithm, which can be filtered into a mipmap pyramid containing Llayer after effective processing [11][12].…”