Optimization and Dynamical Systems

Abstract

PrefaceThis work is aimed at mathematics and engineering graduate students and researchers in the areas of optimization, dynamical systems, control systems, signal processing, and linear algebra. The motivation for the results developed here arises from advanced engineering applications and the emergence of highly parallel computing machines for tackling such applications. The problems solved are those of linear algebra and linear systems theory, and include such topics as diagonalizing a symmetric matrix, singular value decomposition, balanced realizations, linear programming, sensitivity minimization, and eigenvalue assignment by feedback control.The tools are those, not only of linear algebra and systems theory, but also of differential geometry. The problems are solved via dynamical systems implementation, either in continuous time or discrete time , which is ideally suited to distributed parallel processing. The problems tackled are indirectly or directly concerned with dynamical systems themselves, so there is feedback in that dynamical systems are used to understand and optimize dynamical systems. One key to the new research results has been the recent discovery of rather deep existence and uniqueness results for the solution of certain matrix least squares optimization problems in geometric invariant theory. These problems, as well as many other optimization problems arising in linear algebra and systems theory, do not always admit solutions which can be found by algebraic methods. Even for such problems that do admit solutions via algebraic methods, as for example the classical task of singular value decomposition, there is merit in viewing the task as a certain matrix optimization problem, so as to shift the focus from algebraic methods to geometric methods. It is in this context that gradient flows on manifolds appear as a natural approach to achieve construction methods that complement the existence and uniqueness results of geometric invari- There has been an attempt to bridge the disciplines of engineering and mathematics in such a way that the work is lucid for engineers and yet suitably rigorous for mathematicians. There is also an attempt to teach, to provide insight, and to make connections, rather than to present the results as a fait accompli.The research for this work has been carried out by the authors while visiting each other's institutions. Some of the work has been written in conjunction with the writing of research papers in collaboration with PhD students Jane Perkins and Robert Mahony, and post doctoral fellow Weiyong Yan. Indeed, the papers benefited from the book and vice versa, and consequently many of the paragraphs are common to both. Uwe Helmke has a background in global analysis with a strong interest in systems theory, and John Moore has a background in control systems and signal processing. AcknowledgementsThis work was partially supported by grants from Boeing Commercial Airplane Company, the Cooperative Research Centre for Robust and Adaptive Systems, and the German-Israel...

Least-squares approximation by elements from matrix orbits achieved by gradient flows on compact lie groups

“…These gradient flows follow the extension of the original idea on the orthogonal group [3,16] to the unitary group [14], where now we have introduced a larger system of coupled flows. …”

Gradient Adaptive Paraunitary Filter Banks for Spatio-Temporal Subspace Analysis and Multichannel Blind Deconvolution

“…where R xx is a symmetric positive-definite matrix, then (5) and (6) forms the framework for principal subspace analysis (PSA) or minor subspace analysis (MSA) [23][24][25][26][27][28][29][30][31][32]. Alternatively, if J (W) is a contrast function, then (5) and (6) forms the framework for contrast-based blind source separation of prewhitened instantaneous mixtures [33][34][35][36].…”

“…For this reason, modifications of the updates are required. For PSA, an approximation to (26) and (27) given by…”

“…Equations (38), (43) and (44) describe the proposed spatio-temporal extension of the principal subspace rule in (26). This algorithm requires (3 mn + n)(L + 1) + m multiply/accumulates (MACs) per iteration to implement, and its average complexity per adaptive parameter is about the same as that in the scalar case (L = 0).…”

“…Although other adaptive methods could be used, gradient methods represent the simplest to implement and thus form the baseline to which others are often compared. Our algorithms are spatio-temporal extensions of gradient techniques on the Grassmann and Stiefel manifolds [21][22][23][24][25][26][27][28][29][30] and can be applied to any appropriatelydefined cost function. We prove that our algorithms in differential form preserve (2) or (4) over time, and thus they adjust W p in the impulse response space of paraunitary systems.…”