Optimization of Graph Neural Networks with Natural Gradient Descent

Izadi, Mohammad; Fang, Yihao; Stevenson, Robert Louis; Lin, Lizhen

doi:10.1109/bigdata50022.2020.9378063

Cited by 20 publications

(3 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gradient descent is used to train deep networks in a bid to reduce a predefined cost function of the output layer and is shown as the negative log-likelihood function. Among a plentitude of deep network models, the type known as deep [24]. Since both kinds, i.e., DBNs and RBMs, do not work with the 2D structure of images to extract a given feature, the weights that are required need to be learned one by one for each pixel.…”

Section: Proposed Deep Belief Learning Networkmentioning

confidence: 99%

Dermoscopic Image Classification Using Deep Belief Learning Network Architecture

Farhi

Kazmi

Imam

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

In this paper, deep belief learning network architecture (DBL) is proposed for medical image classification in a bid to improve the diagnostics of dermal melanoma as an alternative to traditional dermoscopy. Preprocessing was carried out by using a linear Gaussian filter by eliminating high-frequency artifacts and distortion. The K -means segmentation technique was used to extract the region of interest. The DBL network was then applied to the segmented image for classification. The DBL architecture disperses the weights and hyperparameters to all positions in an image, making it possible to scale to various image sizes. The effects of overfitting were mitigated for small datasets and were achieved by optimizing the proposed network. The algorithm works effectively by fine-tuning constraints. The results showed an increase in the accuracy between the proposed model and AlexNet and LeeNet for segmented images from 8% to 47%, respectively. Similarly, an increase for nonsegmented images was observed between 2% and 48%. An average reduction of 47.8% and 41.5% in error for both segmented and nonsegmented images was recorded for dermal images. The execution time also decreased in comparison with the other architectures averaged by 8-13%, since the weights were distributed only on the clustered regions in the segmented image, as compared to the whole image thus allowing the network to classify it faster with improved accuracy.

show abstract

Section: Proposed Deep Belief Learning Networkmentioning

confidence: 99%

Dermoscopic Image Classification Using Deep Belief Learning Network Architecture

Farhi

Kazmi

Imam

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…NGD transforms gradients into so-called natural gradients that have proved to be much faster compared to the stochastic gradient descent (SGD). Recently, the work in [15] used NGD for a semi-supervised classification task in GCN, and it showed encouraging results in both accuracy and convergence speed on some benchmark Algorithm 1 Preconditioning using NGD Input: Gradient of parameters ∇W l for l = 1, ..., m, adjacency matrix A, degree matrix D, training mask z, regularization hyper-parameters λ,ǫ 1: Derive the numbers of labeled and unlabeled vertices via n = (z) and n = dim(z). And let [∆a ij ] represent the entry of ∆A.…”

Section: Graph Reconstructionmentioning

confidence: 99%

“…However, any extra information about gradients is often impossible or hard to obtain. Motivated by NGD, we introduce a preconditioning algorithm that uses the second moment of gradient to approximate the parameters' Fisher information matrix in the prediction distribution [15].…”

mentioning

confidence: 99%

Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networks

Liu¹,

Xiao²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

The effective deployment of connected vehicular networks is contingent upon maintaining a desired performance across spatial and temporal domains. In this paper, a graphbased framework, called SMART, is proposed to model and keep track of the spatial and temporal statistics of vehicleto-infrastructure (V2I) communication latency across a large geographical area. SMART first formulates the spatio-temporal performance of a vehicular network as a graph in which each vertex corresponds to a subregion consisting of a set of neighboring location points with similar statistical features of V2I latency and each edge represents the spatio-correlation between latency statistics of two connected vertices. Motivated by the observation that the complete temporal and spatial latency performance of a vehicular network can be reconstructed from a limited number of vertices and edge relations, we develop a graph reconstructionbased approach using a graph convolutional network integrated with a deep Q-networks algorithm in order to capture the spatial and temporal statistic of feature map pf latency performance for a large-scale vehicular network. Extensive simulations have been conducted based on a five-month latency measurement study on a commercial LTE network. Our results show that the proposed method can significantly improve both the accuracy and efficiency for modeling and reconstructing the latency performance of large vehicular networks.

show abstract

Subgraph Reconstruction via Reversible Subgraph Embedding

Yang¹,

Zheng²

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Optimization of Graph Neural Networks with Natural Gradient Descent

Cited by 20 publications

References 10 publications

Dermoscopic Image Classification Using Deep Belief Learning Network Architecture

Dermoscopic Image Classification Using Deep Belief Learning Network Architecture

Spatio-temporal Modeling for Large-scale Vehicular Networks Using Graph Convolutional Networks

Subgraph Reconstruction via Reversible Subgraph Embedding

Contact Info

Product

Resources

About