An Efficient Approach to Informative Feature Extraction from Multimodal Data

Wang, Lichen; Wu, Jiaxiang; Huang, Shao-Lun; Zheng, Lizhong; Xu, Xiangxiang; Zhang, Lin; Huang, Junzhou

doi:10.1609/aaai.v33i01.33015281

Cited by 58 publications

(25 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proof: See Appendix B. Note that the H-score can be efficiently estimated from data samples, via, e.g., [10,Algorithm 1]. Then, we can set the negative H-score −H (f θ , g) as our loss function, and use SGD to train the optimal parameters θ H and g H .…”

Section: B Mcr With Deep Learningmentioning

confidence: 99%

“…is a normalized measure of the dependence between two random variables in the k-dimensional functional spaces. In particular, the functions f and g that achieve the maximum value of (17), called maximal correlation functions, have been shown to take important roles in information theory [11], [15] and machine learning [10], [16].…”

Section: Relation To Hgr Maximal Correlationmentioning

confidence: 99%

See 1 more Smart Citation

Maximal Correlation Regression

Huang

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a novel regression analysis approach, called maximal correlation regression, by exploiting the ideas from the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation. We show that in supervised learning problems, the optimal weights in maximal correlation regression can be expressed analytically with the relationships to the HGR maximal correlation functions, which reveals theoretical insights for our approach. In addition, we apply the maximal correlation regression to deep learning, in which efficient training algorithms are proposed for learning the weights in hidden layers. Furthermore, we illustrate that the maximal correlation regression is deeply connected to several existing approaches in information theory and machine learning, including the universal feature selection problem, linear discriminant analysis, and the softmax regression. Finally, experiments on real datasets demonstrate that our approach can obtain performance comparable to the widely used softmax regression based-method. INDEX TERMS Artificial neural networks, HGR maximal correlation, linear discriminant analysis, machine learning algorithms, regression analysis, softmax regression.

show abstract

Section: B Mcr With Deep Learningmentioning

confidence: 99%

Section: Relation To Hgr Maximal Correlationmentioning

confidence: 99%

Maximal Correlation Regression

Huang

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, these good detection capabilities come with the drawback of high computational requirements. Many different approaches have been proposed in the state of the art to decrease the computational requirements by means of input dimensional reduction [41,42]. The proposed approach follows these main guidelines, but it makes use of a low dimensonality input (80×60 pixels) to avoid high computational requirements.…”

Section: Manhole Detectionmentioning

confidence: 99%

A Robust Localization System for Inspection Robots in Sewer Networks

Alejo

Caballero

Merino

2019

Sensors

View full text Add to dashboard Cite

Sewers represent a very important infrastructure of cities whose state should be monitored periodically. However, the length of such infrastructure prevents sensor networks from being applicable. In this paper, we present a mobile platform (SIAR) designed to inspect the sewer network. It is capable of sensing gas concentrations and detecting failures in the network such as cracks and holes in the floor and walls or zones were the water is not flowing. These alarms should be precisely geo-localized to allow the operators performing the required correcting measures. To this end, this paper presents a robust localization system for global pose estimation on sewers. It makes use of prior information of the sewer network, including its topology, the different cross sections traversed and the position of some elements such as manholes. The system is based on a Monte Carlo Localization system that fuses wheel and RGB-D odometry for the prediction stage. The update step takes into account the sewer network topology for discarding wrong hypotheses. Additionally, the localization is further refined with novel updating steps proposed in this paper which are activated whenever a discrete element in the sewer network is detected or the relative orientation of the robot over the sewer gallery could be estimated. Each part of the system has been validated with real data obtained from the sewers of Barcelona. The whole system is able to obtain median localization errors in the order of one meter in all cases. Finally, the paper also includes comparisons with state-of-the-art Simultaneous Localization and Mapping (SLAM) systems that demonstrate the convenience of the approach.

show abstract

“…Therefore, from (22) and (16), the characterization of the sample complexity (7) can be reduced to the minimization of Γ XY 2 F subjected to (11) and (23). Since (23) is a quadratic function of Γ XY , this is equivalent to the optimization problem:…”

Section: A the Sample Complexity For The Case σ K > σ K+1mentioning

confidence: 99%

“…In machine learning problems, the variable Y can be viewed as the label, and X is the data variable that is used to infer or predict about attributes of Y . It can be shown that the optimal functions f * , g * maximizing (1), called the maximal correlation functions, extract the most correlated aspects between X, Y , which take important roles in statistics [7], [8], information theory [6], [9], machine learning [10], [11], and recently in interpreting deep neural networks [12]. Therefore, efficiently and effectively computing maximal correlation functions from data appears to be important in information theory and machine learning.…”

Section: Introductionmentioning

confidence: 99%

On The Sample Complexity of HGR Maximal Correlation Functions

Huang

2019

2019 IEEE Information Theory Workshop (ITW)

Self Cite

View full text Add to dashboard Cite

The Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the corresponding functions have been shown useful in many machine learning scenarios. In this paper, we study the sample complexity of estimating the HGR maximal correlation functions by the alternative conditional expectation (ACE) algorithm from a sequence of training data in the asymptotic regime. Specifically, we develop a mathematical framework to characterize the learning errors between the maximal correlation functions computed from the true distribution, and the functions estimated from the ACE algorithm. For both supervised and semi-supervised learning scenarios, we establish the analytical expressions for the error exponents of the learning errors, which indicate the number of training samples required for estimating the HGR maximal correlation functions by the ACE algorithm. Moreover, with our theoretical results, we investigate the sampling strategy for different types of samples in semi-supervised learning with a total sampling budget constraint, and an optimal sampling strategy is developed to maximize the error exponent of the learning error. Finally, the numerical simulations are presented to support our theoretical results.

show abstract

An Efficient Approach to Informative Feature Extraction from Multimodal Data

Cited by 58 publications

References 21 publications

Maximal Correlation Regression

Maximal Correlation Regression

A Robust Localization System for Inspection Robots in Sewer Networks

On The Sample Complexity of HGR Maximal Correlation Functions

Contact Info

Product

Resources

About