Online regularized pairwise learning with non-i.i.d. observations

Abstract

In this paper, we consider the online regularized pairwise learning (ORPL) algorithm with least squares loss function for non-independently and identically distribution (non-i.i.d.) observations. We first establish new Bennett’s inequalities for [Formula: see text]-mixing sequence, geometrically [Formula: see text]-mixing sequence, [Formula: see text]-geometrically ergodic Markov chain and uniformly ergodic Markov chain. Then we establish the convergence rates for the last iterate of the ORPL algorithm with the polynomially decaying step sizes and varying regularization parameters for non-i.i.d. observations. These established results in this paper extend the previously known results of ORPL from i.i.d. observations to the case of non-i.i.d. observations, and the established result of ORPL for [Formula: see text]-mixing can be nearly optimal rate of ORPL for i.i.d. observations with [Formula: see text]-norm.