Effective weakly supervised semantic frame induction using expression sharing in hierarchical hidden Markov models

Abstract

We present a framework for the induction of semantic frames from utterances in the context of an adaptive command-and-control interface. The system is trained on an individual user's utterances and the corresponding semantic frames representing controls. During training, no prior information on the alignment between utterance segments and frame slots and values is available. In addition, semantic frames in the training data can contain information that is not expressed in the utterances. To tackle this weakly supervised classification task, we propose a framework based on Hidden Markov Models (HMMs). Structural modifications, resulting in a hierarchical HMM, and an extension called expression sharing are introduced to minimize the amount of training time and effort required for the user. The dataset used for the present study is patcor, which contains commands uttered in the context of a vocally guided card game, Patience. Experiments were carried out on orthographic and phonetic transcriptions of commands, segmented on different levels of n-gram granularity. The experimental results show positive effects of all the studied system extensions, with some effect differences between the different input representations. Moreover, evaluation experiments on held-out data with the optimal system configuration show that the extended system is able to achieve high accuracies with relatively small amounts of training data.

A weakly supervised sequence tagging and grammar induction approach to semantic frame slot filling

“…This paper explores two different retraining techniques: inductive sequence labeling (Section 4.1) and stochastic context-free grammar induction (Section 4.2). As described in van de Loo et al (2019), FramEngine can be shown to exhibit a more than decent performance on the task of frame-slot filling for the PATCOR data. For retraining purposes, we can use FramEngine to decode its own training set, linking each word in the training set with a frame slot/value label through Viterbi decoding of the utterance.…”

“…The training partitions in the P9 set are used incrementally as training data for FramEngine. Optimal parameters and information sources for Fra-mEngine were obtained during expansive experiments on the P1-8 (van de Loo et al, 2019). The trained hierarchical hidden markov model is then used to transcode the training utterances into slot-value tag sequences.…”

“…The results show that the system has a promising learning potential even with small amounts of training data, but that enhancements are needed in order to produce practically usable accuracies for more complex utterances, such as the ones found in PATCOR. Work described in van de Loo et al (2019) focused on improvements with respect to the system architecture. Extensions regarding the HMM structure and a technique called "expression sharing" were added to FramEngine's workings and were shown to significantly improve on the frame-slot filling abilities on transcribed PATCOR data.…”

“…Extensions regarding the HMM structure and a technique called "expression sharing" were added to FramEngine's workings and were shown to significantly improve on the frame-slot filling abilities on transcribed PATCOR data. van de Loo et al (2019) describes parameter selection experiments on P1-8, while a final evaluation was conducted on the basis of learning curve experiments on the P9 set. FramEngine was able to score over 95% frame-slot F-score using around 200 utterances as training data.…”