On the Optimality of the Backward Greedy Algorithm for the Subset Selection Problem

Abstract

The following linear inverse problem is considered: given a full column rank m n data matrix A and a length m observation vector b, nd the best least squares solution to Ax = b with at most r < n nonzero components. The backward greedy algorithm computes a sparse solution to Ax = b by removing greedily columns from A until r columns are left. A simple implementation based on a QR downdating scheme by Givens rotations is described. The backward greedy algorithm is shown to be optimal for this problem in the sense that it selects the \correct" subset of columns from A if the perturbation of the data vector b is small enough.

Realization of desired digital spike-trains by a simple evolutionary algorithm

“…5(a) is a = (5,6,7,6,7,11,12,13) and that in Fig. 5(b) is a = (7,6,7,6,5,11,13,12). Each wiring activates either element of the X-cells.…”

“…5(a) gives the characteristic vector of Dmap in Fig. 1: 7,6,7,6,5,11,13,12) The Dmap generates stable PEO and the DSN generates a stable PST as shown in Fig. 6(a).…”

“…Basically, the evolutionary algorithms use individuals which are potential solutions of an optimization problem for a cost function [10][11][12][13]. The individuals refer to their past history, communicate to each other, and try to find a solution.…”

“…As is well known, various evolutionary algorithms have been presented and have been applied to optimization problems in various systems. For example, analog/digital filters, switching power converters, and neural networks [11][12][13][14][15][16][17][18][19]. The design of dynamic digital circuit is an important/effective object in the applications.…”

Sparse Basis Selection: New Results and Application to Adaptive Prediction of Video Source Traffic

“…Such an approach is termed the forward greedy selection approach, and the majority of the proposed methods are related to this type [3], [12], [32], [35]. Another approach, the backward greedy selection approach, starts with the full dictionary, and sequentially eliminates one vector after the other [13], [14], [40]. Other approaches are based on minimizing the norm as well as some entropy-based criterion [39], minimizing the norm [17], or based on some algebraic approach [33].…”