The family of decision tree learning algorithms is among the most widespread and studied. Motivated by the desire to develop learning algorithms that can generalize when learning highly varying functions such as those presumably needed to achieve artificial intelligence, we study some theoretical limitations of decision trees. We demonstrate formally that they can be seriously hurt by the curse of dimensionality in a sense that is a bit different from other nonparametric statistical methods, but most importantly, that they cannot generalize to variations not seen in the training set. This is because a decision tree creates a partition of the input space and needs at least one example in each of the regions associated with a leaf to make a sensible prediction in that region. A better understanding of the fundamental reasons for this limitation suggests that one should use forests or even deeper architectures instead of trees, which provide a form of distributed representation and can generalize to variations not encountered in the training data.

No abstract

Abstract:In this paper we present a forecasting method for time series using copula-based models for multivariate time series. We study how the performance of the predictions evolves when changing the strength of the di erent possible dependencies, as well as the structure of the dependence. We also look at the impact of the marginal distributions. The impact of estimation errors on the performance of the predictions is also considered. In all the experiments, we compare predictions from our multivariate method with predictions from the univariate version which has been introduced in the literature recently. To simplify implementation, a test of independence between univariate Markovian time series is proposed. Finally, we illustrate the methodology by a practical implementation with nancial data.

No abstract

No abstract