Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning

Lei, Deren; Jiang, Gangrong; Gu, Xueping; Sun, Kexuan; Mao, Yuning; Ren, Xiang

doi:10.18653/v1/2020.emnlp-main.688

Cited by 41 publications

(19 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Metrics are not uniformly calculated across individual algorithm implementations and common software libraries. This leads to unfair comparisons in reported results (Sun et al, 2020;Berrendorf et al, 2021). Therefore, we implement a independent and standardised approach to fairly evaluate predictions.…”

Section: Evaluation Strategymentioning

confidence: 97%

“…PoLo's results came from testing a limited number (151) of Compoundtreats-Disease links and the evaluation strategy was not standardised as pruning was not tested across all models. The ambiguity of how metrics are calculated also creates the possibility for unfair comparisons (Sun et al, 2020;Berrendorf et al, 2021). To overcome the limitations of these studies, we extend their work to larger biomedical KGs and offer a set of standardised evaluations.…”

Section: Reinforcement Learning Reasoningmentioning

confidence: 99%

See 1 more Smart Citation

Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs

Edwards¹,

Nilsson²,

Rózemberczki³

et al. 2021

Preprint

View full text Add to dashboard Cite

For Artificial Intelligence to have a greater impact in biology and medicine, it is crucial that recommendations are both accurate and transparent. In other domains, a neurosymbolic approach of multi-hop reasoning on knowledge graphs has been shown to produce transparent explanations. However, there is a lack of research applying it to complex biomedical datasets and problems. In this paper, the approach is explored for drug discovery to draw solid conclusions on its applicability. For the first time, we systematically apply it to multiple biomedical datasets and recommendation tasks with fair benchmark comparisons. The approach is found to outperform the best baselines by 21.7% on average whilst producing novel, biologically relevant explanations.

show abstract

Section: Evaluation Strategymentioning

confidence: 97%

Section: Reinforcement Learning Reasoningmentioning

confidence: 99%

Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs

Edwards¹,

Nilsson²,

Rózemberczki³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, spurious paths may mislead the agent to learn a wrong policy and further harm the generalization ability of the model. Although some studies have noticed the spurious path problem and provided instinctive solutions, such as action drop (Lin et al, 2018) and rule guider (Lei et al, 2020;Hou et al, 2021), a quantitative estimation is still absent.…”

Section: Introductionmentioning

confidence: 99%

Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning

Jiang,

Zhu,

Zhou

et al. 2023

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Multi-hop reasoning, a prevalent approach for query answering, aims at inferring new facts along reasonable paths over a knowledge graph. Reinforcement learning (RL) methods can be adopted by formulating the problem into a Markov decision process. However, common suffering within RL-based reasoning models is that the agent can be biased to spurious paths which coincidentally lead to the correct answer with poor explanation. In this work, we take a deep dive into this phenomenon and define a metric named Path Spuriousness (PS), to quantitatively estimate to what extent a path is spurious. Guided by the definition of PS, we design a model with a new reward that considers both answer accuracy and path reasonableness. We test our method on five datasets and experiments reveal that our method considerably enhances the agent's capacity to prevent spurious paths while keeping comparable to state-of-the-art performance.

show abstract

“…approaches of neural-symbolic reasoning aim at training an agent to infer path and object entity over KGs, which called walk-based reasoning. Recently, numerous walkbased reasoning methods based on RL approaches [28][29][30][31][32][33][34][35] solve KGR as a sequential decision process of reasoning. Both effectiveness and interpretability are demonstrated in these multi-hop methods.…”

Section: Introductionmentioning

confidence: 99%

AInvR: Adaptive Learning Rewards for Knowledge Graph Reasoning Using Agent Trajectories

Zhang,

Lu,

Qin

et al. 2023

Tsinghua Sci. Technol.

View full text Add to dashboard Cite

Multi-hop reasoning for incomplete Knowledge Graphs (KGs) demonstrates excellent interpretability with decent performance. Reinforcement Learning (RL) based approaches formulate multi-hop reasoning as a typical sequential decision problem. An intractable shortcoming of multi-hop reasoning with RL is that sparse reward signals make performance unstable. Current mainstream methods apply heuristic reward functions to counter this challenge.However, the inaccurate rewards caused by heuristic functions guide the agent to improper inference paths and unrelated object entities. To this end, we propose a novel adaptive Inverse Reinforcement Learning (IRL) framework for multi-hop reasoning, called AInvR. (1) To counter the missing and spurious paths, we replace the heuristic rule rewards with an adaptive rule reward learning mechanism based on agent's inference trajectories; (2) to alleviate the impact of over-rewarded object entities misled by inaccurate reward shaping and rules, we propose an adaptive negative hit reward learning mechanism based on agent's sampling strategy; (3) to further explore diverse paths and mitigate the influence of missing facts, we design a reward dropout mechanism to randomly mask and perturb reward parameters for the reward learning process. Experimental results on several benchmark knowledge graphs demonstrate that our method is more effective than existing multi-hop approaches.

show abstract

Learning Collaborative Agents with Rule Guidance for Knowledge Graph Reasoning

Cited by 41 publications

References 14 publications

Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs

Explainable Biomedical Recommendations via Reinforcement Learning Reasoning on Knowledge Graphs

Path Spuriousness-aware Reinforcement Learning for Multi-Hop Knowledge Graph Reasoning

AInvR: Adaptive Learning Rewards for Knowledge Graph Reasoning Using Agent Trajectories

Contact Info

Product

Resources

About