Robust cross-lingual knowledge base question answering via knowledge distillation

Abstract

PurposePrevious knowledge base question answering (KBQA) models only consider the monolingual scenario and cannot be directly extended to the cross-lingual scenario, in which the language of questions and that of knowledge base (KB) are different. Although a machine translation (MT) model can bridge the gap through translating questions to the language of KB, the noises of translated questions could accumulate and further sharply impair the final performance. Therefore, the authors propose a method to improve the robustness of KBQA models in the cross-lingual scenario.Design/methodology/approachThe authors propose a knowledge distillation-based robustness enhancement (KDRE) method. Specifically, first a monolingual model (teacher) is trained by ground truth (GT) data. Then to imitate the practical noises, a noise-generating model is designed to inject two types of noise into questions: general noise and translation-aware noise. Finally, the noisy questions are input into the student model. Meanwhile, the student model is jointly trained by GT data and distilled data, which are derived from the teacher when feeding GT questions.FindingsThe experimental results demonstrate that KDRE can improve the performance of models in the cross-lingual scenario. The performance of each module in KBQA model is improved by KDRE. The knowledge distillation (KD) and noise-generating model in the method can complementarily boost the robustness of models.Originality/valueThe authors first extend KBQA models from monolingual to cross-lingual scenario. Also, the authors first implement KD for KBQA to develop robust cross-lingual models.

Improving anti-money laundering in bitcoin using evolving graph convolutions and deep neural decision forest

“…The idea of distilling the knowledge in a neural network was first put forth by Hinton as a way to improve the performance of a machine learning algorithm by compressing the knowledge in a complex model into a single simple model (Hinton et al , 2015). Though this idea was primarily thought of as a compression mechanism rather than as an enhancement, in recent times, this idea has evolved to fit the needs for a performance enhancement (Wang and Dang, 2021; Hongyuan et al , 2022). In datasets used for speech and object recognition tasks, training must extract structure from these very large, highly redundant datasets, which often leads to a cumbersome model.…”