Classifying Big Data Over Networks Via The Logistic Network Lasso

Ambos, H.Dieter; Tran, Nguyen; Jung, Alexander

doi:10.1109/acssc.2018.8645260

Cited by 10 publications

(22 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper substantially extends our prior work on networked linear models for regression and classification [1], [28], [29], [47]. We have recently derived conditions on the data network structure such that nLasso accurately learns a clustered graph signal [29].…”

Section: Introductionmentioning

confidence: 66%

“…Algorithm 1 can be implemented as message passing over the empirical graph G (see [1]). During each iteration, messages are passed over each edge {i, j} ∈ E in the empirical graph.…”

Section: B Computational Complexitymentioning

confidence: 99%

“…2 depicts the normalized mean squared error (NMSE) ε := w − w 2 2 / w 2 2 incurred by Algorithm 1 (averaged over 10 i.i.d. simulation runs) for varying connectivity, as measured by the empirical averageρ of ρ (1) and ρ (2) (having same distribution). According to Fig.…”

Section: A Two-cluster Datasetmentioning

confidence: 99%

See 2 more Smart Citations

Networked Exponential Families for Big Data Over Networks

Jung

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

The data generated in many application domains can be modeled as big data over networks, i.e., massive collections of high-dimensional local datasets related via an intrinsic network structure. Machine learning for big data over networks must jointly leverage the information contained in the local datasets and their network structure. We propose networked exponential families as a novel probabilistic modeling framework for machine learning from big data over networks. We interpret the high-dimensional local datasets as the realizations of a random process distributed according to some exponential family. Networked exponential families allow us to jointly leverage the information contained in local datasets and their network structure in order to learn a tailored model for each local dataset. We formulate the task of learning the parameters of networked exponential families as a convex optimization problem. This optimization problem is an instance of the network Lasso and enforces a data-driven pooling (or clustering) of the local datasets according to their corresponding parameters for the exponential family. We derive an upper bound on the estimation error of network Lasso. This upper bound depends on the network structure and the information geometry of the node-wise exponential families. These insights provided by this bound can be used for determining how much data needs to be collected or observed to ensure network Lasso to be accurate. We also provide a scalable implementation of the network Lasso as a message-passing between adjacent local datasets. Such message passing is appealing for federated machine learning relying on edge computing. We finally note that the proposed method is also privacy-preserving because no raw data but only parameter (estimates) are shared among different nodes.

show abstract

Section: Introductionmentioning

confidence: 66%

“…Algorithm 1 can be implemented as message passing over the empirical graph G (see [1]). During each iteration, messages are passed over each edge {i, j} ∈ E in the empirical graph.…”

Section: B Computational Complexitymentioning

confidence: 99%

See 1 more Smart Citation

Networked Exponential Families for Big Data Over Networks

Jung

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…It can be shown that Algorithm 1 can be implemented as message passing over the empirical graph G (see [1]). During each iteration, messages are passed over each edge {i, j} ∈ E in the empirical graph.…”

Section: Computational Complexitymentioning

confidence: 99%

“…2, we depict the normalized mean squared error (NMSE) ε := w − w 2 2 / w 2 2 incurred by Algorithm 1 (averaged over 10 i.i.d. simulation runs) for varying connectivity, as measured by the empirical averageρ of ρ (1) and ρ (2) (having same distribution). According to Fig.…”

Section: Two-cluster Datasetmentioning

confidence: 99%

Networked Exponential Families For Big Data Over Networks

Jung¹

2020

Preprint

Self Cite

View full text Add to dashboard Cite

We propose networked exponential families for non-parametric machine learning from massive network-structured datasets (“big data over networks”). High-dimensional data points are interpreted as the realizations of a random process distributed according to some exponential family. Networked exponential families allow to jointly leverage the information contained in high-dimensional data points and their network structure. For data points representing individuals, we obtain perfectly personalized models which enable high-precision medicine or more general recommendation systems.We learn the parameters of networked exponential families, using the network Lasso which implicitly pools (or clusters) the data points according to the intrinsic network structure and a local likelihood function. Our main theoretical result characterizes how the accuracy of network Lasso depends on the network structure and the information geometry of the node-wise exponential families. The network Lasso can be implemented as highly scalable message-passing over the data network. Such message passing is appealing for federated machine learning relying on edge computing. The proposed method is also privacy preserving in the sense that no raw data but only parameter (estimates) are shared among different nodes.

show abstract

Federated Learning from Big Data Over Networks

SarcheshmehPour

Leinonen

Jung

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper formulates and studies a novel algorithm for federated learning from large collections of local datasets. This algorithm capitalizes on an intrinsic network structure that relates the local datasets via an undirected "empirical" graph. We model such big data over networks using a networked linear regression model. Each local dataset has individual regression weights. The weights of close-knit sub-collections of local datasets are enforced to deviate only little. This lends naturally to a network Lasso problem which we solve using a primal-dual method. We obtain a distributed federated learning algorithm via a message passing implementation of this primal-dual method. We provide a detailed analysis of the statistical and computational properties of the resulting federated learning algorithm.

show abstract

Classifying Big Data Over Networks Via The Logistic Network Lasso

Cited by 10 publications

References 7 publications

Networked Exponential Families for Big Data Over Networks

Networked Exponential Families for Big Data Over Networks

Networked Exponential Families For Big Data Over Networks

Federated Learning from Big Data Over Networks

Contact Info

Product

Resources

About