Survey on Multi-Output Learning

Xu, Donna; Shi, Yaxin; Tsang, Ivor W.; Ong, Yew-Soon; Gong, Chen; Shen, Xiaobo

doi:10.1109/tnnls.2019.2945133

Cited by 170 publications

(111 citation statements)

References 201 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To date, scalable MTGPs are mainly studied in the scenario where the tasks have well defined labels and share the input space with modest dimensions. Many efforts are required for extending current MTGPs to handle the 4V challenges in the regime of multi-task (multi-output) learning [205].…”

Section: Scalable Multi-task Gpmentioning

confidence: 99%

When Gaussian Process Meets Big Data: A Review of Scalable GPs

Liu

Ong

Shen

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

564

320

View full text Add to dashboard Cite

The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a well-known non-parametric and interpretable Bayesian model, which suffers from cubic complexity to data size. To improve the scalability while retaining desirable prediction quality, a variety of scalable GPs have been presented. But they have not yet been comprehensively reviewed and analyzed in order to be well understood by both academia and industry. The review of scalable GPs in the GP community is timely and important due to the explosion of data size. To this end, this paper is devoted to the review on state-of-the-art scalable GPs involving two main categories: global approximations which distillate the entire data and local approximations which divide the data for subspace learning. Particularly, for global approximations, we mainly focus on sparse approximations comprising prior approximations which modify the prior but perform exact inference, posterior approximations which retain exact prior but perform approximate inference, and structured sparse approximations which exploit specific structures in kernel matrix; for local approximations, we highlight the mixture/product of experts that conducts model averaging from multiple local experts to boost predictions. To present a complete review, recent advances for improving the scalability and capability of scalable GPs are reviewed. Finally, the extensions and open issues regarding the implementation of scalable GPs in various scenarios are reviewed and discussed to inspire novel ideas for future research avenues.Index Terms-Gaussian process regression, big data, scalability, sparse approximations, local approximations Haitao Liu is with the Rolls-

show abstract

Section: Scalable Multi-task Gpmentioning

confidence: 99%

When Gaussian Process Meets Big Data: A Review of Scalable GPs

Liu

Ong

Shen

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

564

320

View full text Add to dashboard Cite

show abstract

“…Notably, this paper presents a detailed analysis of artificial NNs with varying numbers of hidden layers L for predicting various QoIs. This problem is known as multi-target regression [57] in the machine learning literature, where the goal is to simultaneously predict multiple outputs given an input vector. This work investigates the performance of NNs with the number of neurons in the output layer set to the number of QoIs.…”

Section: Proposed Two-stage Framework For Predictive Modeling Anmentioning

confidence: 99%

Neural Networks and Imbalanced Learning for Data-Driven Scientific Computing With Uncertainties

Pourkamali-Anaraki

Hariri-Ardebili

2021

IEEE Access

View full text Add to dashboard Cite

Uncertainty quantification in complex engineering problems is challenging because of necessitating large numbers of expensive model evaluations. This paper proposes a two-stage framework for developing accurate machine learning-based surrogate models in structural engineering. The studied numerical model considers aleatory and epistemic uncertainties, i.e., ground motion features and material properties. Our framework's first step trains classification algorithms on the collected data from our numerical model with a disproportionate ratio of observations from two categories, i.e., failed and safe simulations. We investigate the performance of imbalanced learning strategies along with artificial neural networks to achieve high classification accuracy. The second step of our framework aims to estimate three quantities of interest using the same network architecture, comparing our approach with regularized linear regression models. Moreover, we present a new approach to reducing the number of numerical simulations for developing machine learning-based surrogate models with limited training data. This approach employs Gaussian processes as a powerful probabilistic technique, providing an inherent uncertainty measure to determine the quality of estimated response values. Extensive numerical experiments demonstrate the superior performance of neural networks with three hidden layers compared to traditional machine learning algorithms for both classification and regression tasks. Also, empirical investigations corroborate that Gaussian processes enable us to predict the values of missing simulations for reducing the computational cost associated with numerical models. To conclude this work, we present several applications and future research directions. INDEX TERMS Uncertainty, Gaussian processes, neural networks, imbalanced classification, regression.

show abstract

“…After finding the appropriate locations of UAVs by RL, the 3D locations of UAVs are determined relying on detailed traffic patterns. More precisely, users interference sensitivity, density, data rate, delay sensitivity, and reliability, are considered as features to build the input dataset for a multi-output SL (MOSL) box, with the output being 3D locations of the UAVs [72]. Multi-output learning subsumes many learning problems in multiple disciplines and provides complex decision making in many real-world applications.…”

Section: Positioning Of Uavs Acting As Bssmentioning

confidence: 99%

On the Application of Machine Learning to the Design of UAV-Based 5G Radio Access Networks

et al. 2020

View full text Add to dashboard Cite

A groundbreaking design of radio access networks (RANs) is needed to fulfill 5G traffic requirements. To this aim, a cost-effective and flexible strategy consists of complementing terrestrial RANs with unmanned aerial vehicles (UAVs). However, several problems must be solved in order to effectively deploy such UAV-based RANs (U-RANs). Indeed, due to the high complexity and heterogeneity of these networks, model-based design approaches, often relying on restrictive assumptions and constraints, exhibit severe limitation in real-world scenarios. Moreover, design of a set of appropriate protocols for such U-RANs is a highly sophisticated task. In this context, machine learning (ML) emerges as a useful tool to obtain practical and effective solutions. In this paper, we discuss why, how, and which types of ML methods are useful for designing U-RANs, by focusing in particular on supervised and reinforcement learning strategies.

show abstract

Survey on Multi-Output Learning

Cited by 170 publications

References 201 publications

When Gaussian Process Meets Big Data: A Review of Scalable GPs

When Gaussian Process Meets Big Data: A Review of Scalable GPs

Neural Networks and Imbalanced Learning for Data-Driven Scientific Computing With Uncertainties

On the Application of Machine Learning to the Design of UAV-Based 5G Radio Access Networks

Contact Info

Product

Resources

About