Persian Language Understanding in Task-Oriented Dialogue System for Online Shopping

Borhanifard, Zeinab; Basafa, Hossein; Razavi, Seyedeh Zahra; Faili, Heshaam

doi:10.1109/ikt51791.2020.9345639

Cited by 1 publication

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These sentences were annotated with 12 semantic labels in 7 main categories. In the research of Borhanifard et al [14] which is an online shopping system, they primarily focused on NLU module. They crafted a corpus for online shopping by initially crawling 500 items in 9 categories of available products, each with its features, from Digikala (one of the well-known Persian online shopping websites).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Data Augmentation and Preparation Process of PerInfEx: A Persian Chatbot With the Ability of Information Extraction

Safari,

Shamsfard

2024

IEEE Access

View full text Add to dashboard Cite

In this paper, we describe data preparation for our proposed chatbot PerInfEx (Persian Information Extraction chatbot). It aims to interactively chit-chat with users in Persian and by asking the least number of direct questions, extract as much personal information as possible such as user's age or occupation. Collecting data in considerable size and aligned with our system's specifics is a crucial step to train data-hungry modules of Natural Language Understating (NLU) and Natural Language Generating (NLG). Initially, for NLU module, we collect 99 free-discussion dialogues and crawl 74 English training conversations as more-general datasets while also manually translate 72 dialogues of ConvAI2 corpus. Moreover, we gamify collection by implementing a chatting website results in 94 dialogues. It detects direct questions and assigns random profiles to participants. They should guess the opponents profile. Also, we propose two augmentation methods: a semi-automatic and a novel fully automatic method, comprehensively evaluated on NLU benchmarks and applied on our datasets. Also, by prompting OpenAI's GPT-3.5 model, we automatically generate 304 dialogues. The first part of these datasets is manually annotated while we use an active learning method for annotating rest of them. Next, to evaluate data quality, we assess them extrinsically using NLU baseline which results in intent-accuracy=88.64, slot-F1=83.68 and exact-match=78.22. Also, for NLG module, we automatically translate almost the rest of ConvAI2 corpus (16,217 dialogues) and paraphrase previously sets for its fine-tuning using GPT-3.5 model. Their assessment using our NLG baseline results in perplexity of 15.74 on train and 52.17 on test set.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In table14, exact match means the percentage of cases where both slots and intent are correctly assigned. As you can see, due to the limited size of data, using a pretrained language model in Joint BERT significantly boosts the results, making…”

mentioning

confidence: 99%

Data Augmentation and Preparation Process of PerInfEx: A Persian Chatbot With the Ability of Information Extraction

Safari,

Shamsfard

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

Persian Language Understanding in Task-Oriented Dialogue System for Online Shopping

Cited by 1 publication

References 15 publications

Data Augmentation and Preparation Process of PerInfEx: A Persian Chatbot With the Ability of Information Extraction

Data Augmentation and Preparation Process of PerInfEx: A Persian Chatbot With the Ability of Information Extraction

Contact Info

Product

Resources

About