A difference-of-convex approach for split feasibility with applications to matrix factorizations and outlier detection

Abstract

The split feasibility problem is to find an element in the intersection of a closed set C and the linear preimage of another closed set D, assuming the projections onto C and D are easy to compute. This class of problems arises naturally in many contemporary applications such as compressed sensing. While the sets C and D are typically assumed to be convex in the literature, in this paper, we allow both sets to be possibly nonconvex. We observe that, in this setting, the split feasibility problem can be formulated as an optimization problem with a difference-of-convex objective so that standard majorization-minimization type algorithms can be applied. Here we focus on the nonmonotone proximal gradient algorithm with majorization studied in [21, Appendix A]. We show that, when this algorithm is applied to a split feasibility problem, the sequence generated clusters at a stationary point of the problem under mild assumptions. We also study local convergence property of the sequence under suitable assumptions on the closed sets involved. Finally, we perform numerical experiments to illustrate

Variable Smoothing for Convex Optimization Problems Using Stochastic Gradients

“…, which is evidently fulfilled. Thus, the choices in (16) are indeed feasible for our algorithm. Now we want to prove the claimed convergence rates.…”

“…In this section we want to solve a nonconvex and nonsmooth optimization problem of completely positive matrix factorization, see [16,19,27]. For an observed matrix A ∈ R d×d we want to find a completely positive low rank factorization, meaning we are looking for x ∈ R r ×d ≥0 with r d such that x T x = A.…”

Non-Convex Split Feasibility Problems: Models, Algorithms and Theory

“…We first prove that the sequence x k , u k , y k k∈N is a Lagrangian sequence as defined in [12].This requires to prove four conditions. In order to prove the first condition we use the optimality of u k+1 and x k+1 in (17) and (18), respectively, to obtain that…”

“…which proves the first condition. For proving the second and third conditions we use the optimality conditions associated with the two steps (17) and (18), meaning…”

“…While the literature on iterative methods for solving the SCF Problem is focused on the optimization model (SF3) and to the best of our understanding most algorithms are modifications and/or generalizations of the CQ Algorithm for other settings of the problem or other related problems with similar affinities, there is one recent paper [18] (as far as we know) that study the CQ Algorithm in the non-convex setting. We will discuss this work in details, later on, after giving the needed notations and relevant definitions.…”