On incremental and robust subspace learning

Abstract

Principal Component Analysis (PCA) has been of great interest in computer vision and pattern recognition. In particular, incrementally learning a PCA model, which is computationally efficient for large scale problems as well as adaptable to reflect the variable state of a dynamic system, is an attractive research topic with numerous applications such as adaptive background modelling and active object recognition. In addition, the conventional PCA, in the sense of least mean squared error minimisation, is susceptible to outlying measurements. To address these two important issues, we present a novel algorithm of incremental PCA, and then extend it to robust PCA. Compared with the previous studies on robust PCA, our algorithm is computationally more efficient. We demonstrate the performance of these algorithms with experimental results on dynamic background modelling and multi-view face modelling.

MOSES: A Streaming Algorithm for Linear Dimensionality Reduction

“…On the one hand, (20) states that MOSES successfully reduces the dimension of streaming data, namely, (14) holds under certain conditions: Loosely-speaking, (20)…”

“…Moreover, MOSES approximately solves Program (7). In other words, MOSES satisfies both (14,15). These statements are made concrete below and proved in Section C of the supplementary material.…”

“…with high probability over the choice of {y t } T t=1 . To summarise, the key questions are whether (11,14,15) hold. We answer these questions for the important case where µ is a zero-mean Gaussian probability measure with covariance matrix Ξ ∈ R n×n .…”

“…The well-known iSVD is a special case of MOSES. At iteration t and given the (truncated) SVD of Y t−1 , iSVD aims to compute the (truncated) SVD of Y t = [Y t−1 y t ] ∈ R n×t , where y t ∈ R n is the newly arrived data vector and Y t−1 is the matrix formed by concatenating the previous data vectors [10], [11], [12], [13], [14]. MOSES generalises iSVD to handle data blocks; see Algorithm 1.…”

“…Indeed, incremental SVD (iSVD) is a successful streaming algorithm that updates its estimate of the truncated SVD of the data matrix with every new incoming vector [10], [11], [12], [13], [14]. However, to the best of our knowledge and despite its popularity and empirical success, iSVD lacks comprehensive statistical guarantees.…”