Robust Factorization Machine: A Doubly Capped Norms Minimization

Abstract

Factorization Machine (FM) is a general supervised learning framework for many AI applications due to its powerful capability of feature engineering. Despite being extensively studied, existing FM methods have several limitations in common. First of all, most existing FM methods often adopt the squared loss in the modeling process, which can be very sensitive when the data for learning contains noises and outliers. Second, some recent FM variants often explore the low-rank structure of the feature interactions matrix by relaxing the low-rank minimization problem as a trace norm minimization, which cannot always achieve a tight approximation to the original one. To address the aforementioned issues, this paper proposes a new scheme of Robust Factorization Machine (RFM) by exploring a doubly capped norms minimization approach, which employs both a capped squared trace norm in achieving a tighter approximation of the rank minimization and a capped 1norm loss to enhance the robustness of the empirical loss minimization from noisy data. We develop an efficient algorithm with a rigorous convergence proof of RFM. Experiments on public real-world datasets show that our method outperforms the state-of-the-art FM methods significantly.

Causal factorization machine for robust recommendation

“…generalizability of the model [45]. FM models against adversarial perturbations have been studied recently [24,34]. Punjabi and Bhatt [34] model the perturbation when there is data uncertainty through Gaussian or Poisson perturbations on the input signals.…”

“…Punjabi and Bhatt [34] model the perturbation when there is data uncertainty through Gaussian or Poisson perturbations on the input signals. Liu et al [24] consider the situation where there are noisy training samples by making some labels of the input features wrong. Liu et al [27] consider discrete adversarial perturbation on instance features since the considered FM features are binary.…”

Deep Self-representative Concept Factorization Network for Representation Learning

“…Representation learning from high-dimensional complex data is always an important and fundamental problem in the fields of pattern recognition and data mining [40][41][42][43][44][45][46][47][48][49][50]. To represent data, lots of feasible and effective approaches can be used, of which Matrix Factorization (MF) based models have been proven to be effective for low-dimensional feature extraction and cluster-ing [24][25][26][27][28][29][30][31][32] [36][37][38][39].…”