Stability and optimization error of stochastic gradient descent for pairwise learning

Shen, Wei; Yang, Zhenhuan; Ying, Yiming; Yuan, Xiaoming

doi:10.1142/s0219530519400062

Cited by 10 publications

(7 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, on average, the solution was not optimal, because of the interaction between stochasticity and the nonlinearities of the models. Errors of various forms of stochastic optimization have been described in other studies (Ingber, 1993;Shen et al, 2020). For example, errors could arise if the sampling were not truly stochastic.…”

Section: Minimization Of Regretmentioning

confidence: 91%

Stochasticity, Nonlinear Value Functions, and Update Rules in Learning Aesthetic Biases

Grzywacz

2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

A theoretical framework for the reinforcement learning of aesthetic biases was recently proposed based on brain circuitries revealed by neuroimaging. A model grounded on that framework accounted for interesting features of human aesthetic biases. These features included individuality, cultural predispositions, stochastic dynamics of learning and aesthetic biases, and the peak-shift effect. However, despite the success in explaining these features, a potential weakness was the linearity of the value function used to predict reward. This linearity meant that the learning process employed a value function that assumed a linear relationship between reward and sensory stimuli. Linearity is common in reinforcement learning in neuroscience. However, linearity can be problematic because neural mechanisms and the dependence of reward on sensory stimuli were typically nonlinear. Here, we analyze the learning performance with models including optimal nonlinear value functions. We also compare updating the free parameters of the value functions with the delta rule, which neuroscience models use frequently, vs. updating with a new Phi rule that considers the structure of the nonlinearities. Our computer simulations showed that optimal nonlinear value functions resulted in improvements of learning errors when the reward models were nonlinear. Similarly, the new Phi rule led to improvements in these errors. These improvements were accompanied by the straightening of the trajectories of the vector of free parameters in its phase space. This straightening meant that the process became more efficient in learning the prediction of reward. Surprisingly, however, this improved efficiency had a complex relationship with the rate of learning. Finally, the stochasticity arising from the probabilistic sampling of sensory stimuli, rewards, and motivations helped the learning process narrow the range of free parameters to nearly optimal outcomes. Therefore, we suggest that value functions and update rules optimized for social and ecological constraints are ideal for learning aesthetic biases.

show abstract

Section: Minimization Of Regretmentioning

confidence: 91%

Stochasticity, Nonlinear Value Functions, and Update Rules in Learning Aesthetic Biases

Grzywacz

2021

Front. Hum. Neurosci.

View full text Add to dashboard Cite

show abstract

“…A (randomized) algorithm A for pairwise learning is called ε-uniformly argument stable if for all neighboring datasets S, S ∈ Z n we have [7]. The connection between the uniform stability for pairwise learning and its generalization has been established in the literature [1,34]. Lemma 1.…”

Section: Stability and Generalization Errorsmentioning

confidence: 99%

Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning

Yang¹,

Lei²,

Wang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Pairwise learning refers to learning tasks where the loss function depends on a pair of instances. It instantiates many important machine learning tasks such as bipartite ranking and metric learning. A popular approach to handle streaming data in pairwise learning is an online gradient descent (OGD) algorithm, where one needs to pair the current instance with a buffering set of previous instances with a sufficiently large size and therefore suffers from a scalability issue. In this paper, we propose simple stochastic and online gradient descent methods for pairwise learning. A notable difference from the existing studies is that we only pair the current instance with the previous one in building a gradient direction, which is efficient in both the storage and computational complexity. We develop novel stability results, optimization, and generalization error bounds for both convex and nonconvex as well as both smooth and nonsmooth problems. We introduce novel techniques to decouple the dependency of models and the previous instance in both the optimization and generalization analysis. Our study resolves an open question on developing meaningful generalization bounds for OGD using a buffering set with a very small fixed size. We also extend our algorithms and stability analysis to develop differentially private SGD algorithms for pairwise learning which significantly improves the existing results.

show abstract

“…[32], [89] consider differential privacy problems in pairwise setting. [77] uses stability to study the tradeoff between the generalization error and optimization error for a variant of pairwise SGD. [39] starts the studying of pairwise learning framework via algorithmic stability.…”

Section: Related Workmentioning

confidence: 99%

“…Secondly, they mostly require convexity conditions [41]. In the related work of learning the unified pairwise framework, [34], [52], [85] investigate the online pairwise learning, which is different from the offline setting of this paper, while [67], [77] study the variants of stochastic gradient descent (SGD). The most related work to this paper is [39], [40], [41].…”

Section: Introductionmentioning

confidence: 99%

“…While in [41], the authors initialize a systematic generalization analysis of SGD under milder assumptions via algorithmic stability and uniform convergence of gradients. However, the above works [34], [39], [40], [41], [52], [67], [77] are almost limited to the convex learning, and even often requiring the restrictive strong convexity condition. An exception is [41], where nonconvex learning is involved.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Rates for Nonconvex Pairwise Learning

Li¹,

Liu²

2021

Preprint

View full text Add to dashboard Cite

Pairwise learning is receiving increasing attention since it covers many important machine learning tasks, e.g., metric learning, AUC maximization, and ranking. Investigating the generalization behavior of pairwise learning is thus of significance. However, existing generalization analysis mainly focuses on the convex objective functions, leaving the nonconvex learning far less explored. Moreover, the current learning rates derived for generalization performance of pairwise learning are mostly of slower order. Motivated by these problems, we study the generalization performance of nonconvex pairwise learning and provide improved learning rates. Specifically, we develop different uniform convergence of gradients for pairwise learning under different assumptions, based on which we analyze empirical risk minimizer, gradient descent, and stochastic gradient descent pairwise learning. We first successfully establish learning rates for these algorithms in a general nonconvex setting, where the analysis sheds insights on the trade-off between optimization and generalization and the role of early-stopping. We then investigate the generalization performance of nonconvex learning with a gradient dominance curvature condition. In this setting, we derive faster learning rates of order O(1/n), where n is the sample size. Provided that the optimal population risk is small, we further improve the learning rates to O(1/n 2 ), which, to the best of our knowledge, are the first O(1/n 2 )-type of rates for pairwise learning, no matter of convex or nonconvex learning. Overall, we systematically analyzed the generalization performance of nonconvex pairwise learning.

show abstract

Stability and optimization error of stochastic gradient descent for pairwise learning

Cited by 10 publications

References 33 publications

Stochasticity, Nonlinear Value Functions, and Update Rules in Learning Aesthetic Biases

Stochasticity, Nonlinear Value Functions, and Update Rules in Learning Aesthetic Biases

Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning

Learning Rates for Nonconvex Pairwise Learning

Contact Info

Product

Resources

About