Model-Free Unsupervised Learning for Optimization Problems with Constraints

Sun, Chengjian; Yang, Chenyang

doi:10.1109/apcc47188.2019.9026527

Cited by 11 publications

(11 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the gradients of the OF and CFs w.r.t. x are derivable, then the policy network and multiplier network can be trained by the primal-dual stochastic gradient method that iteratively updates the PNP and MNP along the ascent and descent directions of the Langrangian function's gradients, respectively [14]. The distribution of the environment's status does not have to be known, because the sampled-averaged gradients can be used for replacing the true gradients in the sense of ensemble average.…”

Section: B Learning Generic Functional Optimization Without Labelsmentioning

confidence: 99%

“…2. The value networks take x and h as the input and then they output the approximated values of the OF and CFs at h. The observed values of the OF and CFs can be used for supervising the training procedure [14]. Specifically, the value networks are trained by minimizing the L 2 -norm loss between their outputs and the observed values of the OF and CFs using SGD, as shown in Fig.…”

Section: A Learning Deterministic Policymentioning

confidence: 99%

“…In contrast to the deterministic policy scenario, the sampled gradients of the OF and CFs w.r.t. the executed action are no longer necessary for the primal-dual update in learning a stochastic policy [14]. To train the policy and multiplier networks, the sampled gradients of the Lagrangian function w.r.t.…”

Section: B Learning Stochastic Policymentioning

confidence: 99%

“…To train the policy and multiplier networks, the sampled gradients of the Lagrangian function w.r.t. the PNP and MNP are used for the primal-dual updates, which can be readily obtained via back-propagation [14]. Nevertheless, the sampled gradient in the primal-dual update of the stochastic policy may exhibit large variance [11], because x also has to be averaged for obtaining the gradients of the Lagrangian function, which results in slow convergence.…”

Section: B Learning Stochastic Policymentioning

confidence: 99%

“…Analogous to the advantage actor-critic approach, a baseline term that averages the Lagrangian function over the distribution of the action can be subtracted from the sampled gradient for reducing the variance [14]. To obtain the actionaveraged Lagrangian function, the average value of the OF and CFs are again approximated by value networks, which are trained by minimizing the L 2 -norm loss using SGD, as shown in Fig.…”

Section: B Learning Stochastic Policymentioning

confidence: 99%

See 4 more Smart Citations

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

Resource allocation and transceivers in wireless networks are usually designed by solving optimization problems subject to specific constraints, which can be formulated as variable or functional optimization. If the objective and constraint functions of a variable optimization problem can be derived, standard numerical algorithms can be applied for finding the optimal solution, which however incur high computational cost when the dimension of the variable is high. To reduce the online computational complexity, learning the optimal solution as a function of the environment's status by deep neural networks (DNNs) is an effective approach. DNNs can be trained under the supervision of optimal solutions, which however, is not applicable to the scenarios without models or for functional optimization where the optimal solutions are hard to obtain. If the objective and constraint functions are unavailable, reinforcement learning can be applied to find the solution of a functional optimization problem, which is however not tailored to optimization problems in wireless networks. In this article, we introduce unsupervised and reinforced-unsupervised learning frameworks for solving both variable and functional optimization problems without the supervision of the optimal solutions. When the mathematical model of the environment is completely known and the distribution of environment's status is known or unknown, we can invoke unsupervised learning algorithm. When the mathematical model of the environment is incomplete, we introduce reinforcedunsupervised learning algorithms that learn the model by interacting with the environment. Our simulation results confirm the applicability of these learning frameworks by taking a user association problem as an example.

show abstract

Section: B Learning Generic Functional Optimization Without Labelsmentioning

confidence: 99%

Section: A Learning Deterministic Policymentioning

confidence: 99%

Section: B Learning Stochastic Policymentioning

confidence: 99%

Section: B Learning Stochastic Policymentioning

confidence: 99%

Section: B Learning Stochastic Policymentioning

confidence: 99%

See 3 more Smart Citations

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

Novel Distributed Beamforming Algorithms for Heterogeneous Space Terrestrial Integrated Network

Shi

Liu

Thompson

2022

IEEE Internet Things J.

View full text Add to dashboard Cite

An integrated space-terrestrial network based on the ultra-dense low-earth-orbit (LEO) satellite constellations has been envisioned in both 5G and beyond 5G (B5G) networks. This approach is a powerful solution to some key challenges from Internet-of-Thing (IoT) services, such as the lack of link capacity to deal with large data transfer or coverage in the remote areas. This paper focuses on the beamforming design for the transmissions from multiple LEO satellites, equipped with massive phased array antenna, to a large number of heterogeneous terrestrial terminals. Superposition coding based beamforming is efficient in dealing with the receiver heterogeneity, but at the cost of higher computational complexity. Based on the dual decomposition theory as well as deep-neural-networks (DNNs), this paper proposes to combine the non-linear approximation ability of DNNs with distributed algorithms. This combination not only supports advanced non-orthogonal beamforming algorithms for achieving superior throughput performance, but also keeps the overall computational complexity low and enables the beamforming process to be speed up dramatically through parallel computing.

show abstract

A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning

et al. 2021

View full text Add to dashboard Cite

Model-Free Unsupervised Learning for Optimization Problems with Constraints

Cited by 11 publications

References 11 publications

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning

Novel Distributed Beamforming Algorithms for Heterogeneous Space Terrestrial Integrated Network

A Tutorial on Ultrareliable and Low-Latency Communications in 6G: Integrating Domain Knowledge Into Deep Learning

Contact Info

Product

Resources

About