Keyword Extraction and Clustering for Document Recommendation in Conversations

Abstract

This paper addresses the problem of keyword extraction from conversations, with the goal of using these keywords to retrieve, for each short conversation fragment, a small number of potentially relevant documents, which can be recommended to participants. However, even a short fragment contains a variety of words, which are potentially related to several topics; moreover, using an automatic speech recognition (ASR) system introduces errors among them. Therefore, it is difficult to infer precisely the information needs of the conversation participants. We first propose an algorithm to extract keywords from the output of an ASR system (or a manual transcript for testing), which makes use of topic modeling techniques and of a submodular reward function which favors diversity in the keyword set, to match the potential diversity of topics and reduce ASR noise. Then, we propose a method to derive multiple topically separated queries from this keyword set, in order to maximize the chances of making at least one relevant recommendation when using these queries to search over the English Wikipedia. The proposed methods are evaluated in terms of relevance with respect to conversation fragments from the Fisher, AMI, and ELEA conversational corpora, rated by several human judges. The scores show that our proposal improves over previous methods that consider only word frequency or topic similarity, and represents a promising solution for a document recommender system to be used in conversations.

An Efficient Approach for Keyphrase Extraction from English Document

“…Many research articles have been found in the literature for keyphrase extraction used machine learning or data mining approaches [23][24][25][26][27][28][29][30][31]. A Naive Bayes approach is used for keyword generation [23].…”

Multimedia Retrieval and Search Ranking Based on Domain Ontology

“…Firstly, query text should be processed in advance. This procedure results in a keyword set for the query text [11]. The keyword set is the initial expansion set.…”

A Collective Study for Document Recommendation Using Textual Conversation Keywords

“…Number of methods has been proposed to automatically extracting keywords from a text, which are also applicable for transcribed conversation. The earliest technique has used word frequencies and TFIDF values to rank words for extraction [1].…”