Stochastic Geometric Programming

Avriel, Mordecai; Wilde, Douglass J.

doi:10.1515/9781400869930-007

Cited by 11 publications

(13 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We consider widely adopted gradient descent (GD) algorithm (Avriel, 2003) in full-graph GNN training. Similar to previous studies (Fu et al, 2020;Chen et al, 2020), the fullgraph training can be modeled as the following non-convex empirical risk minimization problem:…”

Section: Impact Of Message Quantization On Training Convergencementioning

confidence: 99%

DGS: Communication-Efficient Graph Sampling for Distributed GNN Training

Wan

Chen

Zhang

2022

2022 IEEE 30th International Conference on Network Protocols (ICNP)

View full text Add to dashboard Cite

Distributed full-graph training of Graph Neural Networks (GNNs) over large graphs is bandwidth-demanding and time-consuming. Frequent exchanges of node features, embeddings and embedding gradients (all referred to as messages) across devices bring significant communication overhead for nodes with remote neighbors on other devices (marginal nodes) and unnecessary waiting time for nodes without remote neighbors (central nodes) in the graph. This paper proposes an efficient GNN training system, AdaQP, to expedite distributed full-graph GNN training. We stochastically quantize messages transferred across devices to lower-precision integers for communication traffic reduction and advocate communication-computation parallelization between marginal nodes and central nodes. We provide theoretical analysis to prove fast training convergence (at the rate of O(T −1 ) with T being the total number of training epochs) and design an adaptive quantization bit-width assignment scheme for each message based on the analysis, targeting a good trade-off between training convergence and efficiency. Extensive experiments on mainstream graph datasets show that AdaQP substantially improves distributed fullgraph training's throughput (up to 3.01×) with negligible accuracy drop (at most 0.30%) or even accuracy improvement (up to 0.19%) in most cases, showing significant advantages over the state-of-the-art works. The code is available at https://github.com/raywan-110/AdaQP.

show abstract

Section: Impact Of Message Quantization On Training Convergencementioning

confidence: 99%

DGS: Communication-Efficient Graph Sampling for Distributed GNN Training

Wan

Chen

Zhang

2022

2022 IEEE 30th International Conference on Network Protocols (ICNP)

View full text Add to dashboard Cite

show abstract

“…Next, the component goodness values are calculated such that they maximize Pr((A, e)|d k ). This is calculated by applying a gradient ascent procedure [9] (lines 9-11).…”

Section: Algorithmmentioning

confidence: 99%

Model-Based Software Debugging

Ceballos

Abreu

Varela-Vaca

et al. 2019

Fault Diagnosis of Dynamic Systems

View full text Add to dashboard Cite

“…It should be noted that the authors, Khanjani, Tavana, DiCaprio, and Fukuyama reference Baoding Liu's extensive work [6] on uncertainty which they use to derive deterministic equivalents for chance-constrained geometric programs in the form of chance constraints, see for example, [7][8][9][10][11][12][13]. Theirs [1] is the only application of which we are aware for uncertain geometric programming models whose deterministic equivalents are standard posynomial geometric programs.…”

Section: Introductionmentioning

confidence: 99%

Perfect Duality in Solving Geometric Programming Problems Under Uncertainty

Bricker

Kortanek

2017

J Optim Theory Appl

View full text Add to dashboard Cite

We examine computational solutions to all of the geometric programming problems published in a recent paper in the Journal of Optimization Theory and Applications. We employed three implementations of published algorithms interchangeably to obtain "perfect duality" for all of these problems. Perfect duality is taken to mean that a computed solution of an optimization problem achieves two properties: (1) primal and dual feasibility and (2) equality of primal and dual objective function values, all within the accuracy of the machine employed. Perfect duality was introduced by Duffin (Math Program 4: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143]1973). When primal and dual objective values differ, we say there is a duality gap.

show abstract

Stochastic Geometric Programming

Cited by 11 publications

References 11 publications

DGS: Communication-Efficient Graph Sampling for Distributed GNN Training

DGS: Communication-Efficient Graph Sampling for Distributed GNN Training

Model-Based Software Debugging

Perfect Duality in Solving Geometric Programming Problems Under Uncertainty

Contact Info

Product

Resources

About