Towards Non-I.I.D. Image Classification: A Dataset and Baselines

He, Yue; Shen, Zuo‐Jun Max; Cui, Peng

doi:10.48550/arxiv.1906.02899

Cited by 4 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…NICO: we evaluate the cat/dog classification in "Animal" dataset in NICO, a benchmark for non-i.i.d problem in He et al (2019). Each animal is associated with "grass","snow" contexts with different proportions, denoted as d s ∈ R 4 (cat,dog in grass,snow).…”

Section: Real-world Ood Challengementioning

confidence: 99%

See 1 more Smart Citation

Latent Causal Invariant Model

Sun,

Wu,

Zheng

et al. 2020

Preprint

View full text Add to dashboard Cite

Current supervised learning can learn spurious correlation during the data-fitting process, imposing issues regarding interpretability, out-of-distribution (OOD) generalization, and robustness. To avoid spurious correlation, we propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction. Specifically, we introduce latent variables that are separated into (a) output-causative factors and (b) others that are spuriously correlated to the output via confounders, to model the underlying causal factors. We further assume the generating mechanisms from latent space to observed data to be causally invariant. We give the identifiable claim of such invariance, particularly the disentanglement of output-causative factors from others, as a theoretical guarantee for precise inference and avoiding spurious correlation. We propose a Variational-Bayesian-based method for estimation and to optimize over the latent space for prediction. The utility of our approach is verified by improved interpretability, prediction power on various OOD scenarios (including healthcare) and robustness on security.

show abstract

Section: Real-world Ood Challengementioning

confidence: 99%

“…On NICO, we implement ConvNet with Batch Balancing as a specifically benchmark inHe et al (2019). The results are 60 ± 1 on m = 8 and 62.33 ± 3.06 on m = 14.…”

mentioning

confidence: 99%

Latent Causal Invariant Model

Sun,

Wu,

Zheng

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…First of all, we prefer to accurately and intuitively quantify the degree of distribution shift each dataset in the federated learning environment. Based on NI [5], we found that different datasets correspond to different trained feature extractors g ϕ (•) and classifiers f θ (•), and get the corresponding results in the NI equation. In the actual case of federal learning, we found that this equation is not practical and explored a simpler and clearer model to replace the model in the equation.…”

Section: Non-iid Equation Definitionmentioning

confidence: 99%

“…Based on the [5], we will represent the label of dataset image in two dimensions: main concept and contexts which provides a novel perspective on the classification of dataset images. For example, in CIFAR10 [10] dataset, in the category of 'dog', images are divided into different contexts such as 'grass', 'car', 'beach', meaning the 'dog' is on the grass, in the car, or on the beach respectively.…”

Section: Concept Shiftmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation Framework For Large-scale Federated Learning

Liu¹,

Zhang²,

Xiao³

et al. 2020

Preprint

View full text Add to dashboard Cite

Federated learning is proposed as a machine learning setting to enable distributed edge devices, such as mobile phones, to collaboratively learn a shared prediction model while keeping all the training data on device, which can not only take full advantage of data distributed across millions of nodes to train a good model but also protect data privacy. However, learning in scenario above poses new challenges. In fact, data across a massive number of unreliable devices is likely to be non-IID (identically and independently distributed), which may make the performance of models trained by federated learning unstable. In this paper, we introduce a framework designed for large-scale federated learning which consists of approaches to generating dataset and modular evaluation framework. Firstly, we construct a suite of open-source non-IID datasets by providing three respects including covariate shift, prior probability shift, and concept shift, which are grounded in real-world assumptions. In addition, we design several rigorous evaluation metrics including the number of network nodes, the size of datasets, the number of communication rounds and communication resources etc. Finally, we present an open-source benchmark for large-scale federated learning research.

show abstract