Factorized weight interaction neural networks for sparse feature prediction

Zou, Dafang; Sheng, Mengmeng; Mao, Jiafa; Chen, Shengyong; Sheng, Weiguo

doi:10.1007/s00521-019-04470-9

Cited by 8 publications

(7 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning has achieved great success in many research fields such as computer vision [35], [36] and natural language processing [37]. As a result, many deep learning based CTR models have also been proposed in recent years [38], [39], [40]. How to effectively model the feature interactions is the key factor for most of these neural network based models.…”

Section: B Neural Network Based Methodsmentioning

confidence: 99%

Feature Interaction based Neural Network for Click-Through Rate Prediction

Zou,

Zhang,

Mao

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Click-Through Rate (CTR) prediction is one of the most important and challenging in calculating advertisements and recommendation systems. To build a machine learning system with these data, it is important to properly model the interaction among features. However, many current works calculate the feature interactions in a simple way such as inner product and element-wise product. This paper aims to fully utilize the information between features and improve the performance of deep neural networks in the CTR prediction task. In this paper, we propose a Feature Interaction based Neural Network (FINN) which is able to model feature interaction via a 3-dimention relation tensor. FINN provides representations for the feature interactions on the the bottom layer and the non-linearity of neural network in modelling higher-order feature interactions. We evaluate our models on CTR prediction tasks compared with classical baselines and show that our deep FINN model outperforms other state-of-the-art deep models such as PNN and DeepFM. Evaluation results demonstrate that feature interaction contains significant information for better CTR prediction. It also indicates that our models can effectively learn the feature interactions, and achieve better performances in real-world datasets.

show abstract

Section: B Neural Network Based Methodsmentioning

confidence: 99%

Feature Interaction based Neural Network for Click-Through Rate Prediction

Zou,

Zhang,

Mao

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…FwFMs can outperform LR, Poly2 and FMs, and achieve comparable performance with FFMs in AUC, while with fewer parameters to be estimated (Pan et al, 2018). In order to reduce the dimension of sparse data, Zou et al (2020a)…”

Section: Field-weighted Factorization Machines (Fwfms)mentioning

confidence: 99%

Click-through rate prediction in online advertising: A literature review

Yang

Zhai

2022

Information Processing & Management

View full text Add to dashboard Cite

“…However, the features of the width part need to be manually designed because it will directly participate in the prediction. If the structure of this part is not accurate, it will affect the accuracy of the entire model [29]. Deep&Cross and Wide&Deep share a similar framework [30], but it uses a residual network instead of a width model to obtain low-level interactive information.…”

Section: B Development Of Fm and Deep Learning Integrated Modelmentioning

confidence: 99%

“…FM, the integrated model of FM and Deep Learning are mainly used for recommendation systems, especially CTR problems. Many features in this type of data are discrete [29], and are very sparse after one-hot encoding. Other fields also have the same features, so many scholars have applied FM, DeepFM, etc.…”

Section: Medical Applications Of Fm Modelsmentioning

confidence: 99%

Research on Disease Prediction Based on Improved DeepFM and IoMT

Amin

Alhussein

et al. 2021

IEEE Access

View full text Add to dashboard Cite

In recent years, with the increase of computer computing power, Deep Learning has begun to be favored. Its learning of non-linear feature combinations has played a role that traditional machine learning cannot reach in almost every field. The application of Deep Learning has also driven the advancement of Factorization Machine (FM) in the field of recommendation systems, because Deep Learning and FM can learn high-order and low-order features combinations respectively, and FM's hidden vector system enables it to learn information from sparse data. The integration of them has attracted the attention of many scholars. They have researched many classic models such as Factorization-supported Neural Network (FNN), Product-based Neural Networks (PNN), Inner PNN (IPNN), Wide&Deep, Deep&Cross, DeepFM, etc. for the Click-Through-Rate (CTR) problem, and their performance is getting better and better. This kind of model is also suitable for agriculture, meteorology, disease prediction and other fields due to the above advantages. Based on the DeepFM model, we predicts the incidence of hepatitis in each sample in the structured disease prediction data of the 2020 Artificial Intelligence Challenge Preliminary Competition, and make minor improvements and parameter adjustments to DeepFM. Compared with other models, the improved DeepFM has excellent performance in AUC. This research can be applied to electronic medical records to reduce the workload of doctors and make doctors focus on the samples with higher predicted incidence rates. For some changing data, such as blood pressure, height, weight, cholesterol, etc., we can introduce the Internet of Medical Things (IoMT). IoMT's sensors can be used to conduct transmission to ensure that the disease can be predicted in time, just in case. After joining IoMT, a healthcare system is formed, which is superior in forecasting and time performance.

show abstract

Factorized weight interaction neural networks for sparse feature prediction

Cited by 8 publications

References 26 publications

Feature Interaction based Neural Network for Click-Through Rate Prediction

Feature Interaction based Neural Network for Click-Through Rate Prediction

Click-through rate prediction in online advertising: A literature review

Research on Disease Prediction Based on Improved DeepFM and IoMT

Contact Info

Product

Resources

About