Transfer Hierarchical Attention Network for Generative Dialog System

Abstract

In generative dialog systems, learning representations for the dialog context is a crucial step in generating high quality responses. The dialog systems are required to capture useful and compact information from mutually dependent sentences such that the generation process can effectively attend to the central semantics. Unfortunately, existing methods may not effectively identify importance distributions for each lower position when computing an upper level feature, which may lead to the loss of information critical to the constitution of the final context representations. To address this issue, we propose a transfer learning based method named transfer hierarchical attention network (THAN). The THAN model can leverage useful prior knowledge from two related auxiliary tasks, i.e., keyword extraction and sentence entailment, to facilitate the dialog representation learning for the main dialog generation task. During the transfer process, the syntactic structure and semantic relationship from the auxiliary tasks are distilled to enhance both the wordlevel and sentence-level attention mechanisms for the dialog system. Empirically, extensive experiments on the Twitter Dialog Corpus and the PERSONA-CHAT dataset demonstrate the effectiveness of the proposed THAN model compared with the state-of-the-art methods.

An intelligent hierarchical residual attention learning‐based conjoined twin neural network for Alzheimer's stage detection and prediction

“…argument engendered with the attention channel 71,72 . The final output of both divisions has been provided for the multistage correlation filter.…”

“…end for 12: data augmentation {shift, flip, rotation zoom, shear} 13: end for 14: end procedure argument engendered with the attention channel. 71,72 The final output of both divisions has been provided for the multistage correlation filter. The primary purpose of using hierarchical initiated residual attention-aware learning was to create more effectual, propagative, and discriminative extracted features.…”

“…Residual attention learning is a neural network-based attention mechanism. 71,72 It is compatible with the most cutting-edge feed-forward network design and is used for end-to-end model training and discriminating features. We have used hierarchical residual attention learning-inspired multistage conjoined twin network for obtaining efficient attention-aware and generative features that are discriminative.…”

Research on Transfer Learning of Vision-based Gesture Recognition

“…The goal of this paper is to propose a method in the field of gesture recognition, which enables a model trained in the source domain to be used in the target domain directly. Therefore, the time for collecting data is reduced and the time for annotating data could be minimized or eliminated [9] .…”

“…Consequently, it is very profitable to develop models in a new field by using the learned information. Using transferable knowledge could decrease the collection of data, reduce the need for data annotation, and increase the learning speed [9] . There is very little work to transfer knowledge between two or more sensor models.…”