Deep Neural Network Regularization for Feature Selection in Learning-to-Rank

Abstract

Learning-to-rank is an emerging area of research for a wide range of applications. Many algorithms are devised to tackle the problem of learning-to-rank. However, very few existing algorithms deal with deep learning. Previous research depicts that deep learning makes significant improvements in a variety of applications. The proposed model makes use of the deep neural network for learning-to-rank for document retrieval. It employs a regularization technique particularly suited for the deep neural network to improve the results significantly. The main aim of regularization is optimizing the weight of neural network, selecting the relevant features with active neurons at the input layer, and pruning of the network by selecting only active neurons at hidden layer while learning. Specifically, we use group 1 regularization in order to induce the group level sparsity on the network's connections. Set of outgoing weights from each hidden layer represents the group here. The sparsity of network is measured by the sparsity ratio and it is compared with learning-torank models, which adopt the embedded method for feature selection. An extensive experimental evaluation considers the performance of the extended 1 regularization technique against classical regularization techniques. The empirical results confirm that sparse group 1 regularization is able to achieve competitive performance while simultaneously making the network compact with less number of input features. The model is analyzed with respect to evaluating measures, such as prediction accuracy, NDCG@n, MAP, and Precision on benchmark datasets, which demonstrate improved results over other state-of-the-art methods.

Generalized Zero Shot Learning via Synthesis Pseudo Features

“…Existing studies [32], [33], [37]- [39] have proven that semantic similarities among classes are consistent with the visual features. This notion indicates that if two semantic data belonging to two different classes are similar, then the visual instances should also be similar to each other.…”

A graph-based feature selection method for learning to rank using spectral clustering for redundancy minimization and biased PageRank for relevance analysis

“…EGRank [18] uses exponentiated gradient updates to solve a convex optimization problem on a sparsity-promoting 1 constraint and a pairwise ranking loss. In [53], a deep neural LTR model was provided. The authors used group 1 regularization to optimize the weights of a neural network, select the relevant features with active neurons at the input layer, and remove inactive neurons from hidden layers.…”

Short-Term Load Forecasting Using Convolutional Neural Networks in COVID-19 Context: The Romanian Case Study

“…The L 1 regularization method aims at minimizing the neural network's weights with respect to the λ 1 coefficient, representing a feature selection mechanism. As the input of the proposed model consists of a significant number of categorical variables, leading to a degree of sparsity in the data, the L 1 regularization may reduce the value of weights associated with insignificant features [54]. Described by the sum of squared weights, the L 2 regularization technique prevents overfitting of data during training by hindering the weights from reaching high values if it is not necessary [55].…”