Differentially Quantized Gradient Descent

Abstract

Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay constraints. The only information that the server knows about the problem instance is what it receives from the worker via a rate-limited noiseless communication channel. We introduce the principle we call differential quantization (DQ) that prescribes that the past quantization errors should be compensated in such a way as to direct the descent trajectory of a quantized algorithm towards that of its unquantized counterpart. Assuming that the objective function is smooth and strongly convex, we prove that differentially quantized gradient descent (DQ-GD) attains a linear contraction factor of max{σGD, ρn2 −R }, where σGD is the contraction factor of unquantized gradient descent (GD), ρn ≥ 1 is the covering efficiency of the quantizer, and R is the bitrate per problem dimension n. Thus at any R ≥ log 2 ρn/σGD bits, the contraction factor of DQ-GD is the same as that of unquantized GD, i.e., there is no loss due to quantization. We show a converse demonstrating that no GD-like quantized algorithm can converge faster than max{σGD, 2 −R }. Since quantizers exist with ρn → 1 as n → ∞ (Rogers, 1963), this means that DQ-GD is asymptotically optimal. In contrast, naively quantized GD where the worker directly quantizes the gradient barely attains σGD+ρn2 −R . The principle of differential quantization continues to apply to gradient methods with momentum such as Nesterov's accelerated gradient descent, and Polyak's heavy ball method. For these algorithms as well, if the rate is above a certain threshold, there is no loss in contraction factor obtained by the differentially quantized algorithm compared to its unquantized counterpart, and furthermore, the differentially quantized heavy ball method attains the optimal contraction achievable among all (even unquantized) gradient methods. Experimental results on both simulated and real-world least-squares problems validate our theoretical analysis.

Linear Stochastic Bandits over a Bit-Constrained Channel

“…We note that results of a similar conceptual flavor are established in[29] and[23] in the context of stabilization of an LTI system, and optimization, respectively. While the results in these papers pertain to deterministic settings, Lemma 1 carefully exploits statistical concentration bounds specific to the stochastic process we study.…”

“…A common abstraction for analyzing optimization under limited communication is one where a worker agent transmits quantized gradients to a server over a finite bit-rate communication channel [21][22][23]. Inspired by this model, for our problem of interest, we introduce and study a new linear stochastic bandit formulation comprising of an agent connected to a decision-making entity (server) by a noiseless communication channel of finite capacity B; see Fig.…”

Distributed Adaptive Learning Under Communication Constraints

“…However, calling upon the theory of predictive quantization (e.g., the sigmadelta modulation adopted in PCM [44]), we see that the impact of quantization errors on convergence can be reduced by properly leveraging the inherent memory arising in recursive implementations such as gradient descent implementations. Two canonical paradigms to achieve this goal are error-feedback management [45], [46], [47], [48] and differential quantization [49], [50], which, perhaps surprisingly, have been applied to distributed optimization quite recently.…”

“…This is because: i) the innovation typically exhibits a reduced range as compared to the entire sample; and ii) owing to the correlation between consecutive samples, quantizing the entire sample will waste resources by transmitting redundant information. The information-theoretic fundamental limits of (non-stochastic) gradient descent under differential quantization have been recently established in [50].…”

“…for a suitable choice of the matrices X and Y , having made explicit the definition of H i in (50). The bound in (138) follows readily from the Lipschitz property in (2).…”