A Fast Scaling Algorithm for Minimizing Separable Convex Functions Subject to Chain Constraints

Abstract

We consider the problem of minimizing Σj∈N Cj(xj), subject to the following chain constraints x1 ≤ x2 ≤ x3 ≤ … ≤ xn, where Cj(xj) is a convex function of xj for each j ∈ N = {1,2, … ,n}. This problem is a generalization of the isotonic regression problems with complete order, an important class of problems in regression analysis that has been examined extensively in the literature. We refer to this problem as the generalized isotonic regression problem. In this paper, we focus on developing a fast-scaling algorithm to obtain an integer solution of the generalized isotonic regression problem. Let U denote the difference between an upper bound on an optimal value of xn and a lower bound on an optimal value of x1. Under the assumption that the evaluation of any function Cj (xj) takes O(1) time, we show that the generalized isotonic regression problem can be solved in O(n log U) time. This improves by a factor of n the previous best running time of O(n2 log U) to solve the same problem. In addition, when our algorithm is specialized to the isotonic median regression problem (where Cj(xj) = cj|xj−aj|) for specified values of cjs and ajs, the algorithm obtains a real-valued optimal solution in O(n log n) time. This time bound matches the best available time bound to solve the isotonic median regression problem, but our algorithm uses simpler data structures and may be easier to implement.

Isotonic Regression via Partitioning

“…Since the subgraph is the original graph, using the median of the data values in the subgraph results in an infinite loop. While having some similarities, Ahuja and Orlin's scaling for L 1 regression [1] is quite different from partitioning. In their approach, all vertices are initially in their own level set, and whenever two vertices are placed in the same level set then they remain in the same level set.…”

“…For example, suppose the values and weights on a linear ordering are (4,1), (3,10), (1,1), (2,1). The first partitioning involves values 2 and 3, and results in all vertices being assigned to the interval [3,4].…”

“…While it has been mentioned in the literature, few concrete algorithms have appeared for p = 2. The only ones we are aware of involve linear orders [1,7,26], with the fastest being Ahuja and Orlin's algorithm [1] which takes Θ(n log U ) time, where U is the maximum difference of any two data values and the results are restricted to integers.…”

“…For example, one may believe that average weight is an increasing function of height and of S < M < L < XL shirt size. That is, if the shirt size is held constant the average weight ordered metric set L 1 L 2 L ∞ linear Θ(n log n) Θ(n) Θ(n log n) [1,22] PAV * tree Θ(n 2 ) Θ(n log n) Θ(n log n) [5] [15] * 2-dim Θ(n 2 log n) Θ(n 2 ) Θ(n log n) grid * [21] * 2-dim Θ(n 3 ) Θ(n 3 ) Θ(n log 2 n) arbitrary * [21] [24] d ≥ 3 dim Θ(n 2 log n) Θ(n 4 ) Θ(n log n) grid * * * d ≥ 3 dim Θ(n 3 ) Θ(n 4 ) Θ(n log d n) arbitrary * * [24] arbitrary Θ(nm + n 2 log n) Θ(n 4 ) Θ(m log n) [2] [14] [24] * indicates that it is implied by the result for arbitrary ordered sets Table 1: Times of fastest previously known isotonic regression algorithms increases with height, and similarly the weight increases if the height is fixed and the shirt size increases. However, there are no assumptions of what happens if height increases but shirt size decreases.…”

“…No matter what order the sets are merged in, for any p the result is optimal. Simple left-right parsing yields the L 2 isotonic regression in Θ(n) time, L 1 in Θ(n log n) time [1,22], and L ∞ in Θ(n log n) time [24]. Thompson's extension of PAV to trees [27] is discussed in Section 3.…”