Dialogue Transformers

Власов, В. И.; Mosig, Johannes E. M.; Nichol, Alan

doi:10.48550/arxiv.1910.00486

Cited by 7 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from NLU tasks, the key dialogue management task is to select the most appropriate system response depending on the context. Rasa provides a Transformer Embedding Dialogue Policy (TED) component [12] to handle this task.…”

Section: A Transformer-based Dialogue Processing In Rasamentioning

confidence: 99%

Conversational AI and Knowledge Graphs for Social Robot Interaction

Wilcock¹,

Jokinen

2022

2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

View full text Add to dashboard Cite

The paper describes an approach that combines work from three fields with previously separate research communities: social robotics, conversational AI, and graph databases. The aim is to develop a generic framework in which a variety of social robots can provide high-quality information to users by accessing semantically-rich knowledge graphs about multiple different domains. An example implementation uses a Furhat robot with Rasa open source conversational AI and knowledge graphs in Neo4j graph databases.

show abstract

Section: A Transformer-based Dialogue Processing In Rasamentioning

confidence: 99%

Conversational AI and Knowledge Graphs for Social Robot Interaction

Wilcock¹,

Jokinen

2022

2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

View full text Add to dashboard Cite

show abstract

“…One of the major advantages of transformers is that they achieve independency while making predictions on different dialogue stages based on the selfattention mechanism. This mechanism is used to learn sentence representations by comparing different positions of that sentence and allows preselecting tokens that affect the current state of the encoder [48,49]. The authors of [50] developed the Recurrent Embedding Dialogue Policy (REDP) architecture that utilizes the attention mechanism to achieve a better performance while recovering from unexpected dialogue inputs.…”

Section: Dialogue Management Modulementioning

confidence: 99%

“…The authors of [50] developed the Recurrent Embedding Dialogue Policy (REDP) architecture that utilizes the attention mechanism to achieve a better performance while recovering from unexpected dialogue inputs. In their work [48], Vlasov et al simplified the architecture of REDP and introduced TED policy. TED policy tries to maximize a similarity function while jointly training embeddings for a dialogue state and system actions.…”

Section: Dialogue Management Modulementioning

confidence: 99%

“…Architecture of the DM system is represented in Figure 5. TED policy is one of the efficient machine learning based dialog management methods [48]. TED policy takes an intent and entities from the user input, the slot and form values, and bot actions from previous turns as features, feeds them into the transformer where the self-attention mechanism is applied.…”

Section: Dialogue Managementmentioning

confidence: 99%

See 1 more Smart Citation

Development of Dialogue Management System for Banking Services

2021

View full text Add to dashboard Cite

Rapid increase in conversational AI and user chat data lead to intensive development of dialogue management systems (DMS) for various industries. Yet, for low-resource languages, such as Azerbaijani, very little research has been conducted. The main purpose of this work is to experiment with various DMS pipeline set-ups to decide on the most appropriate natural language understanding and dialogue manager settings. In our project, we designed and evaluated different DMS pipelines with respect to the conversational text data obtained from one of the leading retail banks in Azerbaijan. In the work, the main two components of DMS—Natural language Understanding (NLU) and Dialogue Manager—have been investigated. In the first step of NLU, we utilized a language identification (LI) component for language detection. We investigated both built-in LI methods such as fastText and custom machine learning (ML) models trained on the domain-based dataset. The second step of the work was a comparison of the classic ML classifiers (logistic regression, neural networks, and SVM) and Dual Intent and Entity Transformer (DIET) architecture for user intention detection. In these experiments we used different combinations of feature extractors such as CountVectorizer, Term Frequency-Inverse Document Frequency (TF-IDF) Vectorizer, and word embeddings for both word and character n-gram based tokens. To extract important information from the text messages, Named Entity Extraction (NER) component was added to the pipeline. The best NER model was chosen among conditional random fields (CRF) tagger, deep neural networks (DNN), models and build in entity extraction component inside DIET architecture. Obtained entity tags fed to the Dialogue Management module as features. All NLU set-ups were followed by the Dialogue Management module that contains a Rule-based Policy to handle FAQs and chitchats as well as a Transformer Embedding Dialogue (TED) Policy to handle more complex and unexpected dialogue inputs. As a result, we suggest a DMS pipeline for a financial assistant, which is capable of identifying intentions, named entities, and a language of text followed by policies that allow generating a proper response (based on the designed dialogues) and suggesting the best next action.

show abstract

“…Song et al employs adversarial training to improve both the quality and diversity of generated texts [39]. Recently, the Transformer encoder-decoder framework [42] is also employed in text generation models [43] to boost coherence.…”

Section: Neural Text Generationmentioning

confidence: 99%

Triangular Bidword Generation for Sponsored Search Auction

Song¹,

Chen²,

Zhou³

et al. 2021

Proceedings of the 14th ACM International Conference on Web Search and Data Mining

View full text Add to dashboard Cite

Sponsored search auction is a crucial component of modern search engines. It requires a set of candidate bidwords that advertisers can place bids on. Existing methods generate bidwords from search queries or advertisement content. However, they suffer from the data noise in and pairs. In this paper, we propose a triangular bidword generation model (TRIDENT), which takes the high-quality data of paired as a supervision signal to indirectly guide the bidword generation process. Our proposed model is simple yet effective: by using bidword as the bridge between search query and advertisement, the generation of search query, advertisement and bidword can be jointly learned in the triangular training framework. This alleviates the problem that the training data of bidword may be noisy. Experimental results, including automatic and human evaluations, show that our proposed TRIDENT can generate relevant and diverse bidwords for both search queries and advertisements. Our evaluation on online real data validates the effectiveness of the TRIDENT's generated bidwords for product search.

show abstract

Dialogue Transformers

Cited by 7 publications

References 20 publications

Conversational AI and Knowledge Graphs for Social Robot Interaction

Conversational AI and Knowledge Graphs for Social Robot Interaction

Development of Dialogue Management System for Banking Services

Triangular Bidword Generation for Sponsored Search Auction

Contact Info

Product

Resources

About