Label-Dependencies Aware Recurrent Neural Networks

Dupont, Yoann; Dinarelli, Marco

doi:10.1007/978-3-319-77113-7_4

Cited by 9 publications

(12 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The improved RNN proposed in this paper is based on a similar model described in [19,20], and later improved in [21,22].…”

Section: Ejordanmentioning

confidence: 99%

Label-Dependency Coding in Simple Recurrent Networks for Spoken Language Understanding

Dinarelli¹,

Vukotić²,

Raymond³

2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

Modeling target label dependencies is important for sequence labeling tasks. This may become crucial in the case of Spoken Language Understanding (SLU) applications, especially for the slot-filling task where models have to deal often with a high number of target labels. Conditional Random Fields (CRF) were previously considered as the most efficient algorithm in these conditions. More recently, different architectures of Recurrent Neural Networks (RNNs) have been proposed for the SLU slot-filling task. Most of them, however, have been successfully evaluated on the simple ATIS database, on which it is difficult to draw significant conclusions. In this paper we propose new variants of RNNs able to learn efficiently and effectively label dependencies by integrating label embeddings. We show first that modeling label dependencies is useless on the (simple) ATIS database and unstructured models can produce state-of-the-art results on this benchmark. On ATIS our new variants achieve the same results as state-of-the-art models, while being much simpler. On the other hand, on the MEDIA benchmark, we show that the modification introduced in the proposed RNN outperforms traditional RNNs and CRF models.

show abstract

“…The improved RNN proposed in this paper is based on a similar model described in [19,20], and later improved in [21,22].…”

Section: Ejordanmentioning

confidence: 99%

Label-Dependency Coding in Simple Recurrent Networks for Spoken Language Understanding

Dinarelli¹,

Vukotić²,

Raymond³

2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…Inspired by the work of Dupont et.al on document analysis [14], we propose to consider information of the previous temporal segment. We perform this by increasing the dimension of the human pose features of the current segment s by concatenating it with a one-hot context vector C s−1 corresponding to the classification of the previous temporal segment s − 1, as illustrated in Fig.…”

Section: ) Context Featuresmentioning

confidence: 99%

Recognition of Activities of Daily Living via Hierarchical Long-Short Term Memory Networks

Devanne

Papadakis

Nguyen

2019

2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)

View full text Add to dashboard Cite

In order to offer optimal and personalized assistance services to frail people, smart homes or assistive robots must be able to understand the context and activities of users. With this outlook, we propose a vision-based approach for understanding activities of daily living (ADL) through skeleton data captured using an RGB-D camera. Upon decomposition of a skeleton sequence into short temporal segments, activities are classified via a hierarchical two-layer Long-Short Term Memory Network (LSTM) allowing to analyse the sequence at different levels of temporal granularity. The proposed approach is evaluated on a very challenging daily activity dataset wherein we attain superior performance. Our main contribution is a multi-scale, temporal dependency model of activities, founded on a comparison of context features that characterize previous recognition results and a hierarchical representation with a low-level behaviour-unit recognition layer and a high-level units chaining layer.

show abstract

“…is performed. They are inspired from the Sequence-to-Sequence architecture for the overall architecture, and from the models proposed in our previous work (Dinarelli & Tellier, 2016a,b;Dinarelli et al, 2017;Dupont et al, 2017;Dinarelli & Grobol, 2019) for making predictions based on a bidirectional context on the output side (labels). Afterwards we added to this architecture some of the characteristics of the Transformer model.…”

Section: Neural Architecturesmentioning

confidence: 99%

“…The neural architecture described so far uses the same ideas introduced in (Dinarelli & Tellier, 2016a;Dinarelli et al, 2017;Dupont et al, 2017;Dinarelli & Grobol, 2019) for predicting labels using both representations of left (forward) and right (backward) contexts, and for both input-level information (words, characters, etc.) and labels.…”

Section: 5mentioning

confidence: 99%

Hybrid Neural Models For Sequence Modelling: The Best Of Three Worlds

Dinarelli,

Grobol

2019

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a neural architecture with the main characteristics of the most successful neural models of the last years : bidirectional RNNs, encoder-decoder, and the Transformer model. Evaluation on three sequence labelling tasks yields results that are close to the state-of-theart for all tasks and better than it for some of them, showing the pertinence of this hybrid architecture for this kind of tasks.

show abstract

Label-Dependencies Aware Recurrent Neural Networks

Cited by 9 publications

References 40 publications

Label-Dependency Coding in Simple Recurrent Networks for Spoken Language Understanding

Label-Dependency Coding in Simple Recurrent Networks for Spoken Language Understanding

Recognition of Activities of Daily Living via Hierarchical Long-Short Term Memory Networks

Hybrid Neural Models For Sequence Modelling: The Best Of Three Worlds

Contact Info

Product

Resources

About