Multi-level Recommendation Reasoning over Knowledge Graphs with Reinforcement Learning

Wang, Xiting; Liu, Kunpeng; Wang, Dongjie; Wu, Le; Fu, Yanjie; Xie, Xing

doi:10.1145/3485447.3512083

Cited by 65 publications

(13 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, self-explainable [12] models perform recommendation and explanation generation jointly. Representative methods of this family include those performing reasoning either on the paths within the KG [19,16,15,11] or via neural symbolic techniques [6,17]. Though the resulting explanation is aligned to the associated recommendation, the utility of the recommendations could decrease and the explanations could not be aligned to the expectation of the users.…”

Section: Introductionmentioning

confidence: 99%

XRecSys: A framework for path reasoning quality in explainable recommendation

et al. 2022

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

XRecSys: A framework for path reasoning quality in explainable recommendation

et al. 2022

View full text Add to dashboard Cite

“…In recent years, RL has been applied to many domains (e.g. spatial-temporal data mining, recommended systems) and achieves great achievements [30,32].…”

Section: Related Workmentioning

confidence: 99%

Group-wise Reinforcement Feature Generation for Optimal and Explainable Representation Space Reconstruction

Wang¹,

Fu²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Representation (feature) space is an environment where data points are vectorized, distances are computed, patterns are characterized, and geometric structures are embedded. Extracting a good representation space is critical to address the curse of dimensionality, improve model generalization, overcome data sparsity, and increase the availability of classic models. Existing literature, such as feature engineering and representation learning, is limited in achieving full automation (e.g., over heavy reliance on intensive labor and empirical experiences), explainable explicitness (e.g., traceable reconstruction process and explainable new features), and flexible optimal (e.g., optimal feature space reconstruction is not embedded into downstream tasks). Can we simultaneously address the automation, explicitness, and optimal challenges in representation space reconstruction for a machine learning task? To answer this question, we propose a group-wise reinforcement generation perspective. We reformulate representation space reconstruction into an interactive process of nested feature generation and selection, where feature generation is to generate new meaningful and explicit features, and feature selection is to eliminate redundant features to control feature sizes. We develop a cascading reinforcement learning method that leverages three cascading Markov Decision Processes to learn optimal generation policies to automate the selection of features and operations and the feature crossing. We design a group-wise generation strategy to cross a feature group, an operation, and another feature group to generate new features and find the strategy that can enhance exploration efficiency and augment reward signals of cascading agents. Finally, we present extensive experiments to demonstrate the effectiveness, efficiency, traceability, and explicitness of our system.

show abstract

“…In recent years, RL has been applied to many domains (e.g. spatial-temporal data mining, recommended systems) and achieves great achievements [11], [27]. In this paper, we formulate the selection of feature groups and operation as MDPs and propose a new cascading agent structure to resolve these MDPs.…”

Section: Related Workmentioning

confidence: 99%

“…In the preliminary work [11], we propose a novel principled framework based on a Traceable Group-wise Reinforcement Generation Perspective for addressing the automation, explicitness, and optimal issues in representation space reconstruction. Specifically, we view feature space reconstruction as a traceable iterative process of the combination of feature generation and feature selection.…”

Section: Introductionmentioning

confidence: 99%

Self-Optimizing Feature Transformation

Xiao¹,

Wang²,

Fu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Feature transformation aims to extract a good representation (feature) space by mathematically transforming existing features. It is crucial to address the curse of dimensionality, enhance model generalization, overcome data sparsity, and expand the availability of classic models. Current research focuses on domain knowledge-based feature engineering or learning latent representations; nevertheless, these methods are not entirely automated and cannot produce a traceable and optimal representation space. When rebuilding a feature space for a machine learning task, can these limitations be addressed concurrently? In the preliminary work, we present a self-optimizing framework for feature transformation that employs three cascading reinforced agents to learn effective policies for selecting candidate features and operations for feature crossing. While it has achieved good performance, the effectiveness of the reconstructed feature space can be improved by: (1) obtaining an advanced state representation for enabling reinforced agents to comprehend the current feature set better; and (2) resolving Q-value overestimation in reinforced agents for learning unbiased and effective policies. Thus, in the journal version, to fulfill goal 1, it is assumed that an effective state representation should not only contain the entire feature set knowledge, but also capture feature-feature correlations. First, we build a complete graph based on feature-feature correlations. Then, using message-passing, we aggregate feature knowledge based on the learned graph structure to obtain a graph embedding. The graph embedding can recover the original graph, which is the state representation. To fulfill goal 2, we undertake Q-learning using two Q-networks: one for action selection and the other for Q-value estimation. We divide the learning process in each Q-Network into two streams: state value estimation and action advantage value estimation. The Q-value is the combination of the two streams' estimations. Finally, to make experiments more convincing than the preliminary work, we conclude by adding the outlier detection task with five datasets, evaluating various state representation approaches, and comparing different training strategies. Extensive experiments and case studies show that our work is more effective and superior.

show abstract

Multi-level Recommendation Reasoning over Knowledge Graphs with Reinforcement Learning

Cited by 65 publications

References 25 publications

XRecSys: A framework for path reasoning quality in explainable recommendation

XRecSys: A framework for path reasoning quality in explainable recommendation

Group-wise Reinforcement Feature Generation for Optimal and Explainable Representation Space Reconstruction

Self-Optimizing Feature Transformation

Contact Info

Product

Resources

About