DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction

Abstract

Learning feature interactions is important to the model performance of online advertising services. As a result, extensive efforts have been devoted to designing effective architectures to learn feature interactions. However, we observe that the practical performance of those designs can vary from dataset to dataset, even when the order of interactions claimed to be captured is the same. That indicates different designs may have different advantages and the interactions captured by them have non-overlapping information. Motivated by this observation, we propose DHEN -a deep and hierarchical ensemble architecture that can leverage strengths of heterogeneous interaction modules and learn a hierarchy of the interactions under different orders. To overcome the challenge brought by DHEN's deeper and multi-layer structure in training, we propose a novel co-designed training system that can further improve the training efficiency of DHEN. Experiments of DHEN on large-scale dataset from CTR prediction tasks attained 0.27% improvement on the Normalized Entropy (NE) of prediction and 1.2x better training throughput than state-of-the-art baseline, demonstrating their effectiveness in practice.

Causality-based CTR prediction using graph neural networks

“…(3) Deep learning models are generally utilized to capture high-order feature interactions for CTR prediction, including standard long short-term memory (LSTM) (Hochreiter and Schmidhuber, 1997), convolutional neural network (CNN) (Zhang et al, 2022c), factorization machine supported neural network (FNN) (Zhang et al, 2016). In order to capture feature interactions of multiple orders flexibly, a bunch of ensemble models have been reported integrating deep learning models with (either low-order or high-order) explicit components, e.g., Wide & Deep (Cheng et al, 2016), DeepFM (Guo et al, 2018), Deep & Cross network (DCN) (Wang et al, 2017) and xDeepFM (Lian et al, 2018).…”