Online Low-Rank Representation Learning for Joint Multi-Subspace Recovery and Clustering

Abstract

Benefiting from global rank constraints, the low-rank representation (LRR) method has been shown to be an effective solution to subspace learning. However, the global mechanism also means that the LRR model is not suitable for handling large-scale data or dynamic data. For large-scale data, the LRR method suffers from high time complexity, and for dynamic data, it has to recompute a complex rank minimization for the entire data set whenever new samples are dynamically added, making it prohibitively expensive. Existing attempts to online LRR either take a stochastic approach or build the representation purely based on a small sample set and treat new input as out-of-sample data. The former often requires multiple runs for good performance and thus takes longer time to run, and the latter formulates online LRR as an out-of-sample classification problem and is less robust to noise. In this paper, a novel online LRR subspace learning method is proposed for both large-scale and dynamic data. The proposed algorithm is composed of two stages: static learning and dynamic updating. In the first stage, the subspace structure is learned from a small number of data samples. In the second stage, the intrinsic principal components of the entire data set are computed incrementally by utilizing the learned subspace structure, and the LRR matrix can also be incrementally solved by an efficient online singular value decomposition algorithm. The time complexity is reduced dramatically for large-scale data, and repeated computation is avoided for dynamic problems. We further perform theoretical analysis comparing the proposed online algorithm with the batch LRR method. Finally, experimental results on typical tasks of subspace recovery and subspace clustering show that the proposed algorithm performs comparably or better than batch methods, including the batch LRR, and significantly outperforms state-of-the-art online methods.

Dynamic Sparse Subspace Clustering for Evolving High-Dimensional Data Streams

“…where • * means the nuclear norm, which is the best convex envelope of the rank [24]. The optimization problem 5is convex and can be solved by several methods.…”

“…On the other hand, the DSC algorithms usually discard many points to fulfill the restrictions on computing and storage usage. Recently, research on online SC algorithms using the selfexpressiveness property has grown in popularity with some representative algorithms, such as SSSC [23], SLRR [23], online LRR [24], OLRSC [25], and SLSR [23], proposed. Most of these approaches achieve online SC based on twophase frameworks, that is, the static phase for global subspace learning and the online phase for subspace classifying (see Fig.…”

“…Contributions: In this article, a novel online SC method, called evolutionary dynamic SSC (EDSSC), is proposed for evolving high-dimensional DSC tasks. Like the recent approaches SSSC [23], SLSR [23], SLRR [23], and online LRR [24], our proposed EDSSC consists of two phases: 1) static learning and 2) online clustering, but fundamental differences exist: EDSSC does not assume that the global subspace structure must be covered by the data points in the static learning phase and must remain unchanged in the online clustering phase. Instead, EDSSC can detect and adapt to the evolving subspace structure based on the proposed subspace evolution detection strategy.…”

“…Instead, EDSSC can detect and adapt to the evolving subspace structure based on the proposed subspace evolution detection strategy. In the online clustering phase, EDSSC identifies the representative points of the newly appearing subspaces and then updates the models with these new points, instead of keeping all the points (such as online LRR [24]) or discarding all the points (such as SSSC [23]). The statistic information of the data streams with identified representative points are stored in a data structure that is referred to as EDSSC summary.…”

MADNet: A Fast and Lightweight Network for Single-Image Super Resolution

“…To mitigate this problem, numerous SR approaches have been proposed from different perspectives, including interpolation-based [17], reconstruction-based [33], and example-based methods [23], [25], [40], [41], [49]. The former two kinds of methods are simple and efficient but suffer a dramatic drop in restoration performance as the scale factors increase, and the example-based methods that try to analyze relationships between LR and HR pairs achieve satisfactory performance but involve time-consuming operations.…”