Taking into Account Mutual Correlations during Selection of Significant Input Features in Neural Network Solution of Inverse Problems of Spectroscopy

Abstract

In the neural network solution of many physical problems, it becomes necessary to reduce the dimension of the input data in order to achieve a more accurate and stable solution while reducing computational complexity. When solving the inverse problem of spectroscopy, high multicollinearity between input features is often observed, as spectral lines may be much wider than the spectral channel width. This leads to the need to use a feature selection method that takes into account this characteristic. The method discussed in this article is based on iterative selection of input features with the highest Pearson correlation with the target variable and elimination of input features with high cross-correlation. This study compares the quality of the neural network solution to the problem of determining the concentration of heavy metal ions in water by Raman and absorption spectra on the full feature set and on its subsets produced by the considered feature selection method and by conventional methods of selection of significant input features.