Continual Learning in Task-Oriented Dialogue Systems

Abstract

Continual learning in task-oriented dialogue systems allows the system to add new domains and functionalities over time after deployment, without incurring the high cost of retraining the whole system each time. In this paper, we propose a first-ever continual learning benchmark for task-oriented dialogue systems with 37 domains to be learned continuously in both modularized and end-to-end learning settings. In addition, we implement and compare multiple existing continual learning baselines, and we propose a simple yet effective architectural method based on residual adapters. We also suggest that the upper bound performance of continual learning should be equivalent to multitask learning when data from all domain is available at once. Our experiments demonstrate that the proposed architectural method and a simple replay-based strategy perform better, by a large margin, compared to other continuous learning techniques, and only slightly worse than the multitask learning upper bound while being 20X faster in learning new domains. We also report several trade-offs in terms of parameter usage, memory size and training time, which are important in the design of a task-oriented dialogue system. The proposed benchmark is released to promote more research in this direction 1 .

Continual Sequence Generation with Adaptive Compositional Modules

“…To evaluate the overall performance on all tasks, we use the mean of all tasks' performance score following Sun et al (2019); Mi et al (2020); Madotto et al (2021). For each scenario (similar tasks and dissimilar tasks), we report the average of mean scores on all sequences as an overall metric.…”

“…Motivated by prior continual sequence generation work (Madotto et al, 2021) that uses Adapter (Houlsby et al, 2019) to insert new adapter module into every transformer layer for each new coming task, we propose to strategically decide whether we can reuse some adapter modules from old tasks before training on each new coming task, in a twostage manner: decision stage and training stage, where the former determines the architecture for new tasks and the later trains the model.…”

“…Li et al (2019) employ neural architecture search (Liu et al, 2018) to optimize on new task's structure before learning new tasks. In language domain, prior work often utilizes adapter (Houlsby et al, 2019;Madotto et al, 2021;Ermis et al, 2022), which could be considered as task-specific MLPs inserted into frozen transformer layers. However, since all adapter modules are designed for only one specific task, no knowledge transfer is directly allowed in this case.…”

“…ARPER (Mi et al, 2020) combines regularization on parameters (Kirkpatrick et al, 2017) with prioritized exemplar replay. Keeping the pretrained model frozen, Madotto et al (2021) added task-specific modules for each task together with a perplexity-based classifier, without taking into account the potential for knowledge transfer between different tasks. Instead of blindly adding new modules for new tasks, our approach can detect reusable modules and strategically add new adapter modules in those layers in which reusing old modules would lead to severe forgetting.…”

“…Impact of Task Sequence Length Prior work in continual learning (Madotto et al, 2021;Huang et al, 2021) suggests that the differences in sequence length could influence the performance of continual learning. To this end, we further investigated the impact of sequence length in Table 4, where we reported the average performance at every step and calculated Backward Transfer following Lopez-Paz and Ranzato (2017):…”