Parallel boosted regression trees for web search ranking

Tyree, Stephen; Weinberger, Kilian Q.; Agrawal, Kunal; Paykin, Jennifer

doi:10.1145/1963405.1963461

Cited by 131 publications

(91 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such static histograms overlooks the variation of underlying data distribution as the tree grows and therefore could lead to biased splits. [2,26] put forward to construct dynamic histograms rebuilt for each layer of tree nodes and used for deliberately approximating the exact splits. [2,26] communicate the histograms re-built for each layer of tree nodes to a master worker for tree induction.…”

Section: Related Workmentioning

confidence: 99%

“…[2,26] put forward to construct dynamic histograms rebuilt for each layer of tree nodes and used for deliberately approximating the exact splits. [2,26] communicate the histograms re-built for each layer of tree nodes to a master worker for tree induction. [1] is a MapReduce algorithm which builds multiple random forest ensembles on distributed blocks of data and merges them into a mega-ensemble.…”

Section: Related Workmentioning

confidence: 99%

“…Data summarization in distributed classification/regression trees: Data summarization in distributed regression trees [2,17,22,26] serves for data compression to facilitate the communication between workers and the master and supports mergeable operations for building a global picture about the data distribution on the master to grow the tree. Meanwhile, [2,17,26] build histograms over the feature value domain to provide splits candidates in growing the tree.…”

Section: Related Workmentioning

confidence: 99%

“…Meanwhile, [2,17,26] build histograms over the feature value domain to provide splits candidates in growing the tree. Our proposed data summarization borrows ideas from [2] and is able to support efficient estimation of robust loss criteria.…”

Section: Related Workmentioning

confidence: 99%

“…Their popularity stems from the ability to (a) select, from the set of all attributes, a subset that is most relevant for the regression and classification problem at hand; (b) identify complex, non-linear correlations between attributes; and to (c) provide highly interpretable and human-readable models [7,17,19,25]. Recently due to the increasing amount of available data and the ubiquity of distributed computation platforms and clouds, there is a rapidly growing interest in designing distributed versions of regression and classification trees [1,2,17,21,26,28], for instance, the decision/regression tree in Apache Spark MLlib machine learning package 3 . Meanwhile, since many of the large datasets are from observations and measurements of physical entities and events, such data is inevitably noisy and skewed in part due to equipment malfunctions or abnormal events [10,12,27].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Efficient Distributed Decision Trees for Robust Regression

Guo

Kutzkov

Ahmed

et al. 2016

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Abstract. The availability of massive volumes of data and recent advances in data collection and processing platforms have motivated the development of distributed machine learning algorithms. In numerous real-world applications large datasets are inevitably noisy and contain outliers. These outliers can dramatically degrade the performance of standard machine learning approaches such as regression trees. To this end, we present a novel distributed regression tree approach that utilizes robust regression statistics, statistics that are more robust to outliers, for handling large and noisy data. We propose to integrate robust statistics based error criteria into the regression tree. A data summarization method is developed and used to improve the efficiency of learning regression trees in the distributed setting. We implemented the proposed approach and baselines based on Apache Spark, a popular distributed data processing platform. Extensive experiments on both synthetic and real datasets verify the effectiveness and efficiency of our approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Efficient Distributed Decision Trees for Robust Regression

Guo

Kutzkov

Ahmed

et al. 2016

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

show abstract

Parallel construction of classification trees on a GPU

Strnad

Nerat

2015

Concurrency and Computation

View full text Add to dashboard Cite

Summary Algorithms for constructing tree‐based classifiers are aimed at building an optimal set of rules implicitly described by some dataset of training samples. As the number of samples and/or attributes in the dataset increases, the required construction time becomes the limiting factor for interactive or even functional use. The problem is emphasized if tree derivation is part of an iterative optimization method, such as boosting. Attempts to parallelize the construction of classification trees have therefore been presented in the past. This paper describes a parallel method for binary classification tree construction implemented on a graphics processing unit (GPU) using compute unified device architecture (CUDA). By employing node‐level, attribute‐level, and split‐level parallelism, the task parallel and data parallel sections of tree induction are mapped to the architecture of a modern GPU. The GPU‐based solution is compared with the sequential and multi‐threaded CPU versions on public access datasets, and it is shown that an order of magnitude acceleration can be achieved on this data‐intensive task using inexpensive commodity hardware. The influence of dataset characteristics on the efficiency of parallel implementation is also analyzed. Copyright © 2015 John Wiley & Sons, Ltd.

show abstract

Prediction of Customer Purchasing Power of Google Merchandise Store

Feng

Gao

2019

Advanced Data Mining and Applications

View full text Add to dashboard Cite

Parallel boosted regression trees for web search ranking

Cited by 131 publications

References 25 publications

Efficient Distributed Decision Trees for Robust Regression

Efficient Distributed Decision Trees for Robust Regression

Parallel construction of classification trees on a GPU

Prediction of Customer Purchasing Power of Google Merchandise Store

Contact Info

Product

Resources

About