On low-rank updates to the singular value and Tucker decompositions

Abstract

The singular value decomposition is widely used in signal processing and data mining. Since the data often arrives in a stream, the problem of updating matrix decompositions under low-rank modification has been widely studied. Brand developed a technique in 2006 that has many advantages. However, the technique does not directly approximate the updated matrix, but rather its previous low-rank approximation added to the new update, which needs justification. Further, the technique is still too slow for large information processing problems. We show that the technique minimizes the change in error per update, so if the error is small initially it remains small. We show that an updating algorithm for large sparse matrices should be sub-linear in the matrix dimension in order to be practical for large problems, and demonstrate a simple modification to the original technique that meets the requirements. We extend Brand's method to tensors, the multi-way generalization of matrices. The few published comparable techniques either focus on small-magnitude changes, are iterative, or have no published error properties. We show that the technique of Brand for updating the truncated singular value decomposition can be redesigned as a low-rank update scheme for the Tucker decomposition, with analogous run-time and error properties.

Redundancy and Optimization of tANS Entropy Encoders

“…Traditionally, measuring the effects of any replacement or permutation of the key elements has been computationally challenging. In particular, obtaining the dominant eigenvector after a change in a state transition matrix is a difficult and wellstudied problem [37], [39]. Existing solutions are significantly more computationally expensive than the straightforward application of our redundancy calculation method, or are just not very effective.…”