A Coordinate Descent Method for Robust Matrix Factorization and Applications

Abstract

Matrix factorization methods are widely used for extracting latent factors for low rank matrix completion and rating prediction problems arising in recommender systems of on-line retailers. Most of the existing models are based on L2 fidelity (quadratic functions of factorization error). In this work, a coordinate descent (CD) method is developed for matrix factorization under L1 fidelity so that the related minimization is done one variable at a time and the factorization error is sparsely distributed. In low rank random matrix completion and rating prediction of MovieLens-100k datasets, the CDL1 method shows remarkable stability and accuracy under gross corruption of training (observation) data while the L2 fidelity based methods rapidly deteriorate. A closed form analytical solution is found for the one-dimensional L1-fidelity subproblem, and is used as a building block of CDL1 algorithm whose convergence is analyzed. The connection with the well-known convex method, the robust principal component analysis (RPCA), is made. A comparison with RPCA on recovering low rank Gaussian matrices under sparse and independent Gaussian noise shows that CDL1 maintains accuracy at much lower sampling ratios (from much fewer observed entries) than that for RPCA.

Median Binary-Connect Method and a Binary Convolutional Neural Network for Word Recognition

“…has closed form solution (eq. (2.3), [6]). Let us rename the sequence {j : |q l j | > 0} as {h i } I i=1 , where I equals the cardinality of {j : |q l j | > 0}.…”

“…For a derivation of the median solution in (2.6), see section 2 of [6] where a robust low rank matrix factorization in the 1 sense is studied. We shall call (2.5)-(2.6) the median based binary projection.…”

“…The 1 projector is found in closed analytical form, and it differs from the 2 projector in that the median operation replaces an arithmatic average operation. It is known that 1 norm as a measure of error is more robust to outliers than 2 , see [6] for a case study on matrix completion. Our experiments on TensoFlow show that BC with 1 projector (called median BC) always outperforms BC under either cold (random) or warm start.…”