A Literature Survey of Recent Advances in Chatbots

Caldarini, Guendalina; Jaf, Sardar; McGarry, Kenneth

doi:10.20944/preprints202112.0265.v1

Cited by 10 publications

(8 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Understanding these models is difficult because they employ complex non-linear interactions in densely-connected layers and operate in a high-dimensional space. Despite this, they are already deployed in high-impact settings Caldarini et al, 2022), underscoring the urgency of understanding and anticipating possible model behaviors. Some researchers have argued that interpretability is critical for the safe deployment of advanced machine learning systems (Hendrycks & Mazeika, 2022).…”

Section: Introductionmentioning

confidence: 99%

Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

Wang¹,

Variengien²,

Conmy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Research in mechanistic interpretability seeks to explain behaviors of machine learning (ML) models in terms of their internal components. However, most previous work either focuses on simple behaviors in small models or describes complicated behaviors in larger models with broad strokes. In this work, we bridge this gap by presenting an explanation for how GPT-2 small performs a natural language task called indirect object identification (IOI). Our explanation encompasses 26 attention heads grouped into 7 main classes, which we discovered using a combination of interpretability approaches relying on causal interventions. To our knowledge, this investigation is the largest end-to-end attempt at reverse-engineering a natural behavior "in the wild" in a language model. We evaluate the reliability of our explanation using three quantitative criteria-faithfulness, completeness and minimality. Though these criteria support our explanation, they also point to remaining gaps in our understanding. Our work provides evidence that a mechanistic understanding of large ML models is feasible, pointing toward opportunities to scale our understanding to both larger models and more complex tasks. Code for all experiments is available at https://github.com/redwoodresearch/Easy-Transformer.

show abstract

Section: Introductionmentioning

confidence: 99%

Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

Wang¹,

Variengien²,

Conmy³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…By merging Sentiment Analysis, Automatic Natural Language Generation, and Artificial Intelligence Modelling Language, it is made possible. This paper [2] discusses about the intelligent conversational computer programs known as chatbots which are created to mimic human speech in order to provide automated online support and guidance. In order to give clients virtual support, chatbots have been widely used by numerous businesses due to their growing benefits.…”

Section: IImentioning

confidence: 99%

A Survey on Intelligent Chatbot for Entertainment Recommendation

Singh¹,

Kashid²,

Badhe³

et al. 2023

IJRASET

View full text Add to dashboard Cite

bstract: Chatbots, also called chatterbots, is a form of artificial intelligence (AI) that simulates text or voice commands and give answers accordingly. It is a computer application uses voice instructions, text dialogues, or both to mimic human communication. In order to reduce the problem of information overload, which has created a possible problem for many Internet users, it is necessary to filter, prioritize, and efficiently distribute essential information on the Internet, where the quantity of options is overwhelming. A chatbot can be employed to communicate with end users, respond to their questions, comprehend their feelings, and make suggestions that are relevant. The objective of our project is to build a chatbot that allows the user to interact with it and get movies or songs recommendation of his liking/choice. Our proposed system is a single page website which can be run on user's desktop as well as mobile and its main focus is to accurately comprehend the user's question in text or voice format and respond to the user with relevant responses. We used Natural Language Processing (NLP) to convert the human conversation in text or voice format into data that is decrypted using recurrent text and patterns and then shaped into automated answers and responses. In recommendation model, Content based filtering is used which works on the data that we take from the user. The system will suggest various movies or songs based on the user's interests, and the result will be shown to the user.

show abstract

“…In a recent survey, Caldarini et al (2022) reviewed advances in chatbots by using the common approach of acquiring scientific papers from search databases, based on certain search terms, and selecting a small subset from the lot for analysis, based on publications between 2007 and 2021. The databases they used are IEEE, ScienceDirect, Springer, Google Scholar, JS-TOR, and arXiv.…”

Section: Related Workmentioning

confidence: 99%

State-of-the-art in Open-domain Conversational AI: A Survey

Adewumi¹,

Liwicki²,

Liwicki³

2022

Preprint

View full text Add to dashboard Cite

We survey SoTA open-domain conversational AI models with the purpose of presenting the prevailing challenges that still exist to spur future research. In addition, we provide statistics on the gender of conversational AI in order to guide the ethics discussion surrounding the issue. Open-domain conversational AI are known to have several challenges, including bland responses and performance degradation when prompted with figurative language, among others. First, we provide some background by discussing some topics of interest in conversational AI. We then discuss the method applied to the two investigations carried out that make up this study. The first investigation involves a search for recent SoTA open-domain conversational AI models while the second involves the search for 100 conversational AI to assess their gender. Results of the survey show that progress has been made with recent SoTA conversational AI, but there are still persistent challenges that need to be solved, and the female gender is more common than the male for conversational AI. One main take-away is that hybrid models of conversational AI offer more advantages than any single architecture. The key contributions of this survey are 1) the identification of prevailing challenges in SoTA open-domain conversational AI, 2) the unusual discussion about opendomain conversational AI for low-resource languages, and 3) the discussion about the ethics surrounding the gender of conversational AI.

show abstract

A Literature Survey of Recent Advances in Chatbots

Cited by 10 publications

References 45 publications

Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

A Survey on Intelligent Chatbot for Entertainment Recommendation

State-of-the-art in Open-domain Conversational AI: A Survey

Contact Info

Product

Resources

About