A Convolutional Click Prediction Model

Liu, Qiang; Yu, Feng; Wu, Shu; Wang, Liang

doi:10.1145/2806416.2806603

Cited by 115 publications

(69 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We take FM, KFM, and NIFM as feature extractors in PNN, leading to Inner Product-based Neural Network (IPNN), Kernel Product-based Neural Network (KPNN), and Product-network In Network (PIN).CTR estimation is a fundamental task in personalized advertising and recommender systems, and we take CTR estimation as the working example to evaluate our models. Extensive experiments on 4 large-scale real-world datasets and 1 contest dataset demonstrate the consistent superiority of our models over 8 baselines [6,15,21,25,27,36,46,51] on both AUC and log loss. Besides, PIN makes great CTR improvement (34.67%) in online A/B test.…”

mentioning

confidence: 70%

“…Convolutional Click Prediction Model (CCPM) [27] ( Fig. 3(b)) uses convolutional layers to explore local-global dependencies.…”

Section: Dnn-based Modelsmentioning

confidence: 99%

“…However, without the interaction module, FNN may fail to learn expected feature interactions automatically. Similarly, Convolutional Click Prediction Model (CCPM) [27] ( Fig. 3(b)) is formulated aŝ…”

Section: Dnn-based Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data

Fang

Zhang

et al. 2018

ACM Trans. Inf. Syst.

189

196

View full text Add to dashboard Cite

User response prediction is a crucial component for personalized information retrieval and filtering scenarios, such as recommender system and web search. The data in user response prediction is mostly in a multi-field categorical format and transformed into sparse representations via one-hot encoding. Due to the sparsity problems in representation and optimization, most research focuses on feature engineering and shallow modeling. Recently, deep neural networks have attracted research attention on such a problem for their high capacity and end-to-end training scheme. In this paper, we study user response prediction in the scenario of click prediction. We first analyze a coupled gradient issue in latent vector-based models and propose kernel product to learn field-aware feature interactions. Then we discuss an insensitive gradient issue in DNN-based models and propose Product-based Neural Network (PNN) which adopts a feature extractor to explore feature interactions. Generalizing the kernel product to a net-in-net architecture, we further propose Product-network In Network (PIN) which can generalize previous models. Extensive experiments on 4 industrial datasets and 1 contest dataset demonstrate that our models consistently outperform 8 baselines on both AUC and log loss. Besides, PIN makes great CTR improvement (relatively 34.67%) in online A/B test.Many machine learning models are leveraged or proposed to work on such a problem, including linear models, latent vector-based models, tree models, and DNN-based models. Linear models, such as Logistic Regression (LR) [25] and Bayesian Probit Regression [14], are easy to implement and with high efficiency. A typical latent vector-based model is Factorization Machine (FM) [36]. FM uses weights and latent vectors to represent categories. According to their parametric representations, LR has a linear feature extractor, and FM has a bi-linear 2 feature extractor. The prediction of LR and FM are simply based on the sum over weights, thus their classifiers are linear. FM works well on sparse data, and inspires a lot of extensions, including Field-aware FM (FFM) [21]. FFM introduces field-aware latent vectors, which gain FFM higher capacity and better performance. However, FFM is restricted by space complexity. Inspired by FFM, we find a coupled gradient issue of latent vector-based models and refine feature interactions 3 as field-aware feature interactions. To solve this issue as well as saving memory, we propose kernel product methods and derive Kernel FM (KFM) and Network in FM (NIFM).Trees and DNNs are potent function approximators. Tree models, such as Gradient Boosting Decision Tree (GBDT) [6], are popular in various data science contests as well as industrial applications. GBDT explores very high order feature combinations in a non-parametric way, yet its exploration ability is restricted when feature space becomes extremely high-dimensional and sparse. DNN has also been preliminarily studied in information system literature [8,33,40,51]. In [51], FM supported Neura...

show abstract

mentioning

confidence: 70%

“…Convolutional Click Prediction Model (CCPM) [27] ( Fig. 3(b)) uses convolutional layers to explore local-global dependencies.…”

Section: Dnn-based Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data

Fang

Zhang

et al. 2018

ACM Trans. Inf. Syst.

189

196

View full text Add to dashboard Cite

show abstract

“…Factorization-machine supported neural networks (FNN) was proposed in [12] to learn embedding vectors of categorical data via pre-trained FM. Convolutional Click Prediction Model (CCPM) was proposed in [13] to predict ad click by convolutional neural networks (CNN). However, in CCPM the convolutions are only performed on the neighbor fields in a certain alignment, which fails to model the full interactions among non-neighbor features.…”

Section: Related Workmentioning

confidence: 99%

Product-Based Neural Networks for User Response Prediction

Cai

Ren

et al. 2016

2016 IEEE 16th International Conference on Data Mining (ICDM)

599

468

View full text Add to dashboard Cite

Abstract-Predicting user responses, such as clicks and conversions, is of great importance and has found its usage in many Web applications including recommender systems, web search and online advertising. The data in those applications is mostly categorical and contains multiple fields; a typical representation is to transform it into a high-dimensional sparse binary feature representation via one-hot encoding. Facing with the extreme sparsity, traditional models may limit their capacity of mining shallow patterns from the data, i.e. low-order feature combinations. Deep models like deep neural networks, on the other hand, cannot be directly applied for the high-dimensional input because of the huge feature space. In this paper, we propose a Product-based Neural Networks (PNN) with an embedding layer to learn a distributed representation of the categorical data, a product layer to capture interactive patterns between interfield categories, and further fully connected layers to explore high-order feature interactions. Our experimental results on two large-scale real-world ad click datasets demonstrate that PNNs consistently outperform the state-of-the-art models on various metrics.

show abstract

“…From the methodology view, linear models such as logistic regression [14] and non-linear models such as tree-based model [10] and factorization machines [19,21] are commonly used. Other variants include Bayesian probit regression [9], FTRFL [24] in factorization machine, and convolutional neural network learning framework [17]. Normally, area under ROC curve (AUC) and relative information gain (RIG) are common evaluation metrics for CTR prediction accuracy [9].…”

Section: Related Workmentioning

confidence: 99%

User Response Learning for Directly Optimizing Campaign Performance in Display Advertising

Ren

Zhang

Rong

et al. 2016

Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Learning and predicting user responses, such as clicks and conversions, are crucial for many Internet-based businesses including web search, e-commerce, and online advertising. Typically, a user response model is established by optimizing the prediction accuracy, e.g., minimizing the error between the prediction and the ground truth user response. However, in many practical cases, predicting user responses is only part of a rather larger predictive or optimization task, where on one hand, the accuracy of a user response prediction determines the final (expected) utility to be optimized, but on the other hand, its learning may also be influenced from the follow-up stochastic process. It is, thus, of great interest to optimize the entire process as a whole rather than treat them independently or sequentially. In this paper, we take real-time display advertising as an example, where the predicted user's ad click-through rate (CTR) is employed to calculate a bid for an ad impression in the second price auction. We reformulate a common logistic regression CTR model by putting it back into its subsequent bidding context: rather than minimizing the prediction error, the model parameters are learned directly by optimizing campaign profit. The gradient update resulted from our formulations naturally fine-tunes the cases where the market competition is high, leading to a more costeffective bidding. Our experiments demonstrate that, while maintaining comparable CTR prediction accuracy, our proposed user response learning leads to campaign profit gains as much as 78.2% for offline test and 25.5% for online A/B test over strong baselines.

show abstract

A Convolutional Click Prediction Model

Cited by 115 publications

References 7 publications

Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data

Product-Based Neural Networks for User Response Prediction over Multi-Field Categorical Data

Product-Based Neural Networks for User Response Prediction

User Response Learning for Directly Optimizing Campaign Performance in Display Advertising

Contact Info

Product

Resources

About