Joint Discriminative Decoding of Words and Semantic Tags for Spoken Language Understanding

Deoras, Anoop; Tür, Gökhan; Sarikaya, Ruhi; Hakkani‐Tür, Dilek

doi:10.1109/tasl.2013.2256894

Cited by 16 publications

(15 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another interesting work (Deoras et al, 2013) is around joint decoding of words and semantic tags on word lattices. They demonstrated significant improvements in both recognition and semantic tagging accuracy over cascade approach.…”

Section: Using Graphs Of Words For Monolingual Slumentioning

confidence: 99%

Multilingual Spoken Language Understanding using graphs and multiple translations

Calvo

Hurtado

García

et al. 2016

Computer Speech & Language

View full text Add to dashboard Cite

In this paper, we present an approach to multilingual Spoken Language Understanding based on a process of generalization of multiple translations, followed by a specific methodology to perform a semantic parsing of these combined translations. A statistical semantic model, which is learned from a segmented and labeled corpus, is used to represent the semantics of the task in a language. Our goal is to allow the users to interact with the system using other languages different from the one used to train the semantic models, avoiding the cost of segmenting and labeling a training corpus for each language. In order to reduce the effect of translation errors and to increase the coverage, we propose an algorithm to generate graphs of words from different translations. We also propose an algorithm to parse graphs of words with the statistical semantic model. The experimental results confirm the good behavior of this approach using French and English as input languages in a spoken language understanding task that was developed for Spanish.

show abstract

Section: Using Graphs Of Words For Monolingual Slumentioning

confidence: 99%

Multilingual Spoken Language Understanding using graphs and multiple translations

Calvo

Hurtado

García

et al. 2016

Computer Speech & Language

View full text Add to dashboard Cite

show abstract

“…Then an SLU module that uses the output of the ASR is trained and optimized for the understanding performance. However, as it has been pointed out many times, the hypothesis that gives a better recognition performance does not always yield a better understanding performance [108,127,38]. If the end goal of SLS is to understand what the user means and respond accordingly, both modules can be optimized jointly such that the system "understands better what it recognizes" and "recognizes better what it understands".…”

Section: Motivationmentioning

confidence: 99%

“…Word confusion networks that are extracted from ASR lattice have been used in [54,123]. A joint decoding algorithm is proposed for jointly performing recognition and understanding in [38]. Re-ranking models that re-ranks the multiple hypotheses of a generative SLU system by using support vector machines is given in [40].…”

Section: Using Multiple Hypothesesmentioning

confidence: 99%

See 1 more Smart Citation

Semantic language models with deep neural networks

Bayer

Riccardi

2016

Computer Speech & Language

View full text Add to dashboard Cite

Spoken language systems (SLS) communicate with users in natural language through speech. There are two main problems related to processing the spoken input in SLS. The first one is automatic speech recognition (ASR) which recognizes what the user says. The second one is spoken language understanding (SLU) which understands what the user means. We focus on the language model (LM) component of SLS. LMs constrain the search space that is used in the search for the best hypothesis. Therefore, they play a crucial role in the performance of SLS.It has long been discussed that an improvement in the recognition performance does not necessarily yield a better understanding performance. Therefore, optimization of LMs for the understanding performance is crucial. In addition, long-range dependencies in languages are hard to handle with statistical language models. These two problems are addressed in this thesis.We investigate two different LM structures. The first LM that we investigate enable SLS to understand better what they recognize by searching the ASR hypotheses for the best understanding performance. We refer to these models as joint LMs. They use lexical and semantic units jointly in the LM. The second LM structure uses the semantic context of an utterance, which can also be described as "what the system understands", to search for a better hypothesis that improves the recognition and the understanding performance. We refer to these models as semantic LMs (SELMs). SELMs use features that are based on a well established theory of lexical semantics, namely the theory of frame semantics. They incorporate the semantic features which are extracted from the ASR hypothesis into the LM and handle long-range dependencies by using the semantic relationships between words and semantic context. ASR noise is propagated to the semantic features, to suppress this noise we introduce the use of deep semantic encodings for semantic feature extraction. In this way, SELMs optimize both the recognition and the understanding performance.

show abstract

“…In (Deoras et al 2012;Deoras et al 2013), the authors propose a method for working with CRF on word lattices via the construction of an expanded lattice where the nodes represent the left and right context in which a word appears. This expanded lattice is much bigger than the original one, both in terms of number of nodes and arcs.…”

Section: Linear Chain Conditional Random Fieldsmentioning

confidence: 99%

A Strategy for Multilingual Spoken Language Understanding Based on Graphs of Linguistic Units

Lance¹

View full text Add to dashboard Cite

This thesis has been developed under a FPU scholarship (AP2010-4193) granted by the Spanish Government, and in the framework of the projects TIMPANO: Technology for complex human-machine conversational interaction with dynamic learning (TIN2011-28169-C05-01), and ASLP-MULAN: Audio, speech and language processing for multimedia analytics (TIN2014-54288-C4-3-R).A mi familia. AgradecimientosPensaba que no, que esta parte no me costaría mucho de escribir, pero no podía estar más equivocado. A veces es muy difícil expresar con palabras lo que uno siente, y más cuando una etapa tan importante está a punto de terminar. Y, aunque no suelo ser de agradecimientos públicos, creo que es justo agradecer y reconocer aquí el apoyo y esfuerzo de muchas personas sin las cuales esta tesis probablemente no habría sido posible.En primer lugar quiero agradecer a mis directores Fernando García y Emilio Sanchis toda la ayuda que me han prestado a lo largo de este tiempo, y el haberse preocupado por que mi formación investigadora se completara. Todo empezó cuando Emilio Sanchis, que había sido mi profesor de algorítmica, me ofreció hacer mi proyecto final de carrera en el grupo de investigación ELiRF. En este grupo he trabajado durante más de seis años y siempre me he sentido valorado. Quiero agradecer a todos los miembros del grupo la confianza que siempre han mostrado en mí, el escuchar mis ideas por extrañas que a veces pudieran parecer, y el haberme ayudado a crecer como persona y como profesional. Pero, aparte de a mis directores, quiero agradecer en especial el apoyo, los conocimientos y la amistad que tres personas me han aportado siempre y que, de una forma u otra, han tenido un impacto decisivo en mi vida: Jon, Lluís y Salva.El día a día en el laboratorio durante estos años ha sido realmente agradable, y gran parte del mérito lo tienen los que han sido mis compañeros durante todo o parte de este tiempo: Joan, Ximo, Mario, Merche, Lucía, Sergio, Flores, Jorge, Paco... Algunos decían que nos lo pasábamos muy bien en el laboratorio, y la verdad es que tenían razón.Es muy posible que esta tesis jamás les llegue a ellos, pero me gustaría dar las gracias ahora, al final del camino académico, a algunos profesores que tuve antes de mi época universitaria, y sin los cuales no sería la persona que soy ahora mismo. Carmen, Doña Encarna, Don AbstractIn this thesis, the problem of multilingual spoken language understanding is addressed using graphs to model and combine the different knowledge sources that take part in the understanding process. As a result of this work, a full multilingual spoken language understanding system has been developed, in which statistical models and graphs of linguistic units are used. One key feature of this system is its ability to combine and process multiple inputs provided by one or more sources such as speech recognizers or machine translators.A graph-based monolingual spoken language understanding system was developed as a starting point. The input to this system is a set of sentences that is provided by...

show abstract

Joint Discriminative Decoding of Words and Semantic Tags for Spoken Language Understanding

Cited by 16 publications

References 26 publications

Multilingual Spoken Language Understanding using graphs and multiple translations

Multilingual Spoken Language Understanding using graphs and multiple translations

Semantic language models with deep neural networks

A Strategy for Multilingual Spoken Language Understanding Based on Graphs of Linguistic Units

Contact Info

Product

Resources

About