In this paper we review 30 years of developments and applications of the variable
projection method for solving separable nonlinear least-squares problems. These
are problems for which the model function is a linear combination of
nonlinear functions. Taking advantage of this special structure, the method of
variable projections eliminates the linear variables obtaining a somewhat
more complicated function that involves only the nonlinear parameters.
This procedure not only reduces the dimension of the parameter space
but also results in a better-conditioned problem. The same optimization
method applied to the original and reduced problems will always converge
faster for the latter. We present first a historical account of the basic
theoretical work and its various computer implementations, and then
report on a variety of applications from electrical engineering, medical and
biological imaging, chemistry, robotics, vision, and environmental sciences.
An extensive bibliography is included. The method is particularly well
suited for solving real and complex exponential model fitting problems,
which are pervasive in their applications and are notoriously hard to solve.