Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems

Yao, Quanming; Kwok, James T.; Gao, Fei; Chen, Wei; Liu, Tie-Yan

doi:10.24963/ijcai.2017/462

Cited by 52 publications

(42 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results on the Movielens dataset are on-par with some of the recent literature [28]. Their authors use a completely different setting (a non-convex matrix factorization approach) to reach a RMSE of 0.785 on the same dataset.…”

Section: Recommendation Resultssupporting

confidence: 72%

Augmenting content-based rating prediction with link stream features

Viard

Fournier-S’niehotta

2019

Computer Networks

View full text Add to dashboard Cite

While graph-based collaborative filtering recommender systems have been introduced several years ago, there are still several shortcomings to deal with, the temporal information being one of the most important. The new link stream paradigm is aiming at extending graphs for correctly modelling the graph dynamics, without losing crucial information. We investigate the impact of such link stream features for recommender systems. We design link stream features, that capture the intrinsic structure and dynamics of the data. We show that such features encode a fine-grained and subtle description of the underlying system. We focused on a traditional recommender system context, the rating prediction on the MovieLens20M movie dataset and the Goodreads book dataset. We input link stream features along with some content-based ones into a gradient boosting machine (XGBoost) and show that it outperforms significantly a sole content-based solution. These encouraging results call for further exploration of this original modelling and its integration to complete state-of-the-art recommender systems algorithms. Link streams and graphs, as natural visualizations of recommender systems, may offer more interpretability in a time when algorithm transparency is an increasingly important topic of discussion. We also hope that these results will sparkle interesting discussions in the community about the connections between link streams and traditional methods (matrix factorization, deep learning).

show abstract

Section: Recommendation Resultssupporting

confidence: 72%

Augmenting content-based rating prediction with link stream features

Viard

Fournier-S’niehotta

2019

Computer Networks

View full text Add to dashboard Cite

show abstract

“…The convergence studies for AAPCD, through a novel perspective, characterize the stepsize based on the momentum parameter. This fills the void in previous analyses such as (Li and Lin 2015;Yao et al 2017), where the effect of the exact value of the momentum parameter on the acceleration of convergence were not observed. As the stability of the algorithm is highly affected by asynchronism, by allowing negative momentum for high staleness values we will show the reduction in the objective function will be increased significantly and accelerates convergence.…”

Section: Introductionsupporting

confidence: 67%

Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problems

Kazemi

Wang

2019

AAAI

View full text Add to dashboard Cite

Nonconvex and nonsmooth problems have recently attracted considerable attention in machine learning. However, developing efficient methods for the nonconvex and nonsmooth optimization problems with certain performance guarantee remains a challenge. Proximal coordinate descent (PCD) has been widely used for solving optimization problems, but the knowledge of PCD methods in the nonconvex setting is very limited. On the other hand, the asynchronous proximal coordinate descent (APCD) recently have received much attention in order to solve large-scale problems. However, the accelerated variants of APCD algorithms are rarely studied. In this paper, we extend APCD method to the accelerated algorithm (AAPCD) for nonsmooth and nonconvex problems that satisfies the sufficient descent property, by comparing between the function values at proximal update and a linear extrapolated point using a delay-aware momentum value. To the best of our knowledge, we are the first to provide stochastic and deterministic accelerated extension of APCD algorithms for general nonconvex and nonsmooth problems ensuring that for both bounded delays and unbounded delays every limit point is a critical point. By leveraging Kurdyka-Łojasiewicz property, we will show linear and sublinear convergence rates for the deterministic AAPCD with bounded delays. Numerical results demonstrate the practical efficiency of our algorithm in speed.

show abstract

“…However, such results required the computation of exact proximal steps (22) in the PGD iterations, and do not apply to CPGD which leverages the inexact proximal step (33). Convergence of PGD in non convex setups with inexact proximal steps was studied in [26], [39]. The results established in both papers require the proximal step approximation errors incurred at each iteration to be decreasing and summable, which may not necessarily be the case for the MAP approximation (32).…”

Section: Local Fixed-point Convergence Of Cpgdmentioning

confidence: 99%

CPGD: Cadzow Plug-and-Play Gradient Descent for Generalised FRI

Simeoni

Besson²,

Hurley

et al. 2021

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

Finite rate of innovation (FRI) is a powerful reconstruction framework enabling the recovery of sparse Dirac streams from uniform low-pass filtered samples. An extension of this framework, called generalised FRI (genFRI), has been recently proposed for handling cases with arbitrary linear measurement models. In this context, signal reconstruction amounts to solving a joint constrained optimisation problem, yielding estimates of both the Fourier series coefficients of the Dirac stream and its so-called annihilating filter, involved in the regularisation term. This optimisation problem is however highly non convex and non linear in the data. Moreover, the proposed numerical solver is computationally intensive and without convergence guarantee. In this work, we propose an implicit formulation of the genFRI problem. To this end, we leverage a novel regularisation term which does not depend explicitly on the unknown annihilating filter yet enforces sufficient structure in the solution for stable recovery. The resulting optimisation problem is still non convex, but simpler since linear in the data and with less unknowns. We solve it by means of a provably convergent proximal gradient descent (PGD) method. Since the proximal step does not admit a simple closed-form expression, we propose an inexact PGD method, coined Cadzow plug-and-play gradient descent (CPGD). The latter approximates the proximal steps by means of Cadzow denoising, a well-known denoising algorithm in FRI. We provide local fixed-point convergence guarantees for CPGD. Through extensive numerical simulations, we demonstrate the superiority of CPGD against the state-of-the-art in the case of non uniform time samples.

show abstract

Efficient Inexact Proximal Gradient Algorithm for Nonconvex Problems

Cited by 52 publications

References 15 publications

Augmenting content-based rating prediction with link stream features

Augmenting content-based rating prediction with link stream features

Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for Nonconvex Nonsmooth Problems

CPGD: Cadzow Plug-and-Play Gradient Descent for Generalised FRI

Contact Info

Product

Resources

About