Exact-K Recommendation via Maximal Clique Optimization

Abstract

This paper targets to a novel but practical recommendation problem named exact-K recommendation. It is different from traditional top-K recommendation, as it focuses more on (constrained) combinatorial optimization which will optimize to recommend a whole set of K items called card, rather than ranking optimization which assumes that "better" items should be put into top positions. Thus we take the first step to give a formal problem definition, and innovatively reduce it to Maximum Clique Optimization based on graph. To tackle this specific combinatorial optimization problem which is NP-hard, we propose Graph Attention Networks (GAttN) with a Multi-head Self-attention encoder and a decoder with attention mechanism. It can end-to-end learn the joint distribution of the K items and generate an optimal card rather than rank individual items by prediction scores. Then we propose Reinforcement Learning from Demonstrations (RLfD) which combines the advantages in behavior cloning and reinforcement learning, making it sufficientand-efficient to train the model. Extensive experiments on three datasets demonstrate the effectiveness of our proposed GAttN with RLfD method, it outperforms several strong baselines with a relative improvement of 7.7% and 4.7% on average in Precision and Hit Ratio respectively, and achieves state-of-the-art (SOTA) performance for the exact-K recommendation problem.

Supervised Advantage Actor-Critic for Recommender Systems

“…Moreover, research on slate-based recommendation has also been conducted in [1,2,5,15], where actions are considered to be sets (slates) of items. This setting leads to an exponentially increased action space.…”

Value Penalized Q-Learning for Recommender Systems

“…There are several difficulties in building RLRS, such as the state representation (Lei et al 2020;Liu et al 2020;Wang et al 2020b), the slated-based recommendation setting (Sunehag et al 2015;Swaminathan et al 2017;Chen et al 2019;Gong et al 2019;Ie et al 2019) and the largescale recommendation problem (Dulac-Arnold et al 2015;Zhao et al 2019). However, there has been limited attention paid to the negative impact of offline setting, i.e., training RLRS agents from static datasets.…”

Co-Displayed Items Aware List Recommendation

“…Bello et al [16] exploit Pointer Network [31], in which they can gradually select items from the ranked items to construct the final lists. For the method in [18], it tries to maximize the click probability of the whole recommended list, in which the impact of multiple clicks is neglected. Ie et al [19] make an assumption that a user consumes a single item from a list only each time, based on which they propose a Q-learning [23] based method to solve the list aware recommendation task.…”

“…Some recent works [10], [15]- [17] propose re-ranking methods, in which they try to find better permutations for the items already ranked by the system. We independently develop a similar idea as [17], where the main difference is that we try to replace the original ranking mechanism in the system with new strategies to generate lists directly as [18]- [20]. The work in [18] is trying to optimize a simplified objective, and the one in [19] makes additional assumptions when solving the list recommendation task.…”

“…We independently develop a similar idea as [17], where the main difference is that we try to replace the original ranking mechanism in the system with new strategies to generate lists directly as [18]- [20]. The work in [18] is trying to optimize a simplified objective, and the one in [19] makes additional assumptions when solving the list recommendation task. For the method in [20], it is based on conditional variational auto-encoder (CVAE) [21].…”