Deep Interest Network for Click-Through Rate Prediction

Abstract

Click-through rate prediction is an essential task in industrial applications, such as online advertising. Recently deep learning based models have been proposed, which follow a similar Embed-ding&MLP paradigm. In these methods large scale sparse input features are first mapped into low dimensional embedding vectors, and then transformed into fixed-length vectors in a group-wise manner, finally concatenated together to fed into a multilayer perceptron (MLP) to learn the nonlinear relations among features. In this way, user features are compressed into a fixed-length representation vector, in regardless of what candidate ads are. The use of fixed-length vector will be a bottleneck, which brings difficulty for Embedding&MLP methods to capture user's diverse interests effectively from rich historical behaviors. In this paper, we propose a novel model: Deep Interest Network (DIN) which tackles this challenge by designing a local activation unit to adaptively learn the representation of user interests from historical behaviors with respect to a certain ad. This representation vector varies over different ads, improving the expressive ability of model greatly. Besides, we develop two techniques: mini-batch aware regularization and data adaptive activation function which can help training industrial deep networks with hundreds of millions of parameters. Experiments on two public datasets as well as an Alibaba real production dataset with over 2 billion samples demonstrate the effectiveness of proposed approaches, which achieve superior performance compared with state-of-the-art methods. DIN now has been successfully deployed in the online display advertising system in Alibaba, serving the main traffic.

Adversarial Multimodal Representation Learning for Click-Through Rate Prediction

“…Ni et al [20] adopt LSTM and the attention mechanism to model the user behavior sequence. Compared to sequence-independent approaches, these methods can significantly improve the CTR prediction accuracy and most of these techniques have been deployed in real-world applications [20,32,38,39].…”

“…Utilizing multiple modality features is often effective to improve the performance of CTR tasks. A straightforward way [20,38,39] is to concatenate the multiple modality features, which is equivalent to giving a fixed importance weight to each modality regardless of different items. A conceivable improvement [14,31] is to dynamically distinguish the contributions of different modalities through an attention mechanism.…”

Sequential Recommendation with Dual Side Neighbor-based Collaborative Relation Modeling

“…And hereby we introduce our Interactive Attention Mechanism. Unlike the attention mechanism in [43] and [44] which uses the target item to query the interacted items sequence, we utilize dual sequences information at the same time interactively to weigh across different time slice. The attention value of each time slice β t is calculated as,…”

“…The inference procedure is illustrated in Figure 4. As for the loss function, we take an end-to-end training and introduce (i) the widely used cross entropy loss L ce [25,43,44] over the whole training dataset and (ii) the parameter regularization L r . We utilize Adam algorithm for optimization.…”

Product Knowledge Graph Embedding for E-commerce

“…Secondly, we resort to the multi-task learning with multi-modal data to handle the sparsity issue. It has become a common practice for industrial applications to leverage the useful information across related tasks to make up for the data sparsity in individual task [12,29,37]. In e-commerce, available data sources often include customer view, purchase, search, substitution records, as well as product descriptions and hierarchical category information.…”