Abstract-In this paper, we introduce concepts to reduce the computational complexity of regression, which are successfully used for Support Vector Machines. To the best of our knowledge, we are the first to publish the use of a cascaded Reduced Set Vector approach for regression. The WaveletApproximated Reduced Vector Machine classifiers for face and facial feature point detection are extended to regression for efficient and robust head pose estimation. We use synthetic data, generated by the 3D Morphable Model, for optimal training sets and demonstrate results superior to state-ofthe-art techniques. The new Wavelet Reduced Vector Regression shows similarly good results on natural data, gaining a reduction of the complexity by a factor of up to 560. The introduced Evolutionary Regression Tree uses coarse-tofine loops of strongly reduced regression and classification up to most accurate complex machines. We demonstrate the Cascaded Condensation Tracking for head pose estimation for a large pose range up to ±90 degrees on videostreams.