Isotonic Regression via Partitioning

Abstract

Algorithms are given for determining weighted isotonic regressions satisfying order constraints specified via a directed acyclic graph (DAG). For the L 1 metric a partitioning approach is used which exploits the fact that L 1 regression values can always be chosen to be data values. Extending this approach, algorithms for binary-valued L 1 isotonic regression are used to find L p isotonic regressions for 1 < p < ∞. Algorithms are given for trees, 2-dimensional and multidimensional orderings, and arbitrary DAGs. Algorithms are also given for L p isotonic regression with constrained data and weight values, L 1 regression with unweighted data, and L 1 regression for DAGs where there are multiple data values at the vertices.

Isotonic Regression for Multiple Independent Variables

“…Algorithms for isotonic regressions on multidimensional orderings, without imposing restrictions such as additive models, have concentrated on 2 dimensions [5,12,14,25,30,31,36]. For the L 2 metric and 2-dimensional points, the fastest known algorithms take Θ(n 2 ) time if the points are in a grid [36] and Θ(n 2 log n) time for points in general position [39], while for L 1 the times are Θ(n log n) and Θ(n log 2 n), respectively [39]. These algorithms, which are based on dynamic programming, are significantly faster than merely utilizing the best algorithm for arbitrary dags.…”

“…Recall that for d = 2 there are dynamic programming algorithms which take Θ(n log 2 n) and Θ(n 2 log n) time for L 1 and L 2 , respectively [39], and Θ(n log n) [39] and Θ(n 2 ) [36] time if they form a grid. When d ≥ 3, for a set of n d-dimensional points, applying the algorithms for arbitrary dags results in Θ(n 3 ) time for…”

“…There is an extensive literature on statistical uses of isotonic regression going back to the 1950's [2,3,14,32,41]. It has also been applied to optimization [4,20,23] and classification [8,9,39] problems. Some recent applications involve very large data sets, including learning [19,26,29], analyzing data from microarrays [1] and general data mining [42].…”

“…Some recent applications involve very large data sets, including learning [19,26,29], analyzing data from microarrays [1] and general data mining [42]. While the earliest algorithms appeared in the statistical literature, most of the advances in exact algorithms have appeared in discrete mathematics, operations research, and computer science forums [1,20,23,36,39], using techniques such as network flows, dynamic programming, and parametric search. We consider the case where V consists of points in d-dimensional space, d ≥ 2, where point p = (p 1 , .…”

“…For L 2 , the algorithm of Maxwell and Muckstadt [23], with the modest correction by Spouge, Wan, and Wilber [36], takes Θ(n 4 ) time. This has recently been reduced to Θ(n 2 m + n 3 log n) [39]. Figure 2 shows that multidimensional ordering on n points can result in Θ(n 2 ) edges.…”