Harnessing the Power of LLMs for Service Quality Assessment From User-Generated Content

Falatouri, Taha; Hrušecká, Denisa; Fischer, Thomas

doi:10.1109/access.2024.3429290

IEEE Access

2024

DOI: 10.1109/access.2024.3429290

|View full text |Cite

Harnessing the Power of LLMs for Service Quality Assessment From User-Generated Content

Taha Falatouri,

Denisa Hrušecká,

Thomas Fischer

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Comparing Machine Learning Models for Sentiment Analysis and Rating Prediction of Vegan and Vegetarian Restaurant Reviews

Hanić,

Bagić Babac,

Gledec

et al. 2024

Computers

View full text Add to dashboard Cite

The paper investigates the relationship between written reviews and numerical ratings of vegan and vegetarian restaurants, aiming to develop a predictive model that accurately determines numerical ratings based on review content. The dataset was obtained by scraping reviews from November 2022 until January 2023 from the TripAdvisor website. The study applies multidimensional scaling and clustering using the KNN algorithm to visually represent the textual data. Sentiment analysis and rating predictions are conducted using neural networks, support vector machines (SVM), random forest, Naïve Bayes, and BERT models. Text vectorization is accomplished through term frequency-inverse document frequency (TF-IDF) and global vectors (GloVe). The analysis identified three main topics related to vegan and vegetarian restaurant experiences: (1) restaurant ambiance, (2) personal feelings towards the experience, and (3) the food itself. The study processed a total of 33,439 reviews, identifying key aspects of the dining experience and testing various machine learning methods for sentiment and rating predictions. Among the models tested, BERT outperformed the others, and TF-IDF proved slightly more effective than GloVe for word representation.

show abstract

Comparing Machine Learning Models for Sentiment Analysis and Rating Prediction of Vegan and Vegetarian Restaurant Reviews

Hanić,

Bagić Babac,

Gledec

et al. 2024

Computers

View full text Add to dashboard Cite

show abstract

Leveraging Large Language Models in Tourism: A Comparative Study of the Latest GPT Omni Models and BERT NLP for Customer Review Classification and Sentiment Analysis

Roumeliotis,

Tselikas,

Nasiopoulos

2024

Information

View full text Add to dashboard Cite

In today’s rapidly evolving digital landscape, customer reviews play a crucial role in shaping the reputation and success of hotels. Accurately analyzing and classifying the sentiment of these reviews offers valuable insights into customer satisfaction, enabling businesses to gain a competitive edge. This study undertakes a comparative analysis of traditional natural language processing (NLP) models, such as BERT and advanced large language models (LLMs), specifically GPT-4 omni and GPT-4o mini, both pre- and post-fine-tuning with few-shot learning. By leveraging an extensive dataset of hotel reviews, we evaluate the effectiveness of these models in predicting star ratings based on review content. The findings demonstrate that the GPT-4 omni family significantly outperforms the BERT model, achieving an accuracy of 67%, compared to BERT’s 60.6%. GPT-4o, in particular, excelled in accuracy and contextual understanding, showcasing the superiority of advanced LLMs over traditional NLP methods. This research underscores the potential of using sophisticated review evaluation systems in the hospitality industry and positions GPT-4o as a transformative tool for sentiment analysis. It marks a new era in automating and interpreting customer feedback with unprecedented precision.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Harnessing the Power of LLMs for Service Quality Assessment From User-Generated Content

Cited by 2 publications

References 49 publications

Comparing Machine Learning Models for Sentiment Analysis and Rating Prediction of Vegan and Vegetarian Restaurant Reviews

Comparing Machine Learning Models for Sentiment Analysis and Rating Prediction of Vegan and Vegetarian Restaurant Reviews

Leveraging Large Language Models in Tourism: A Comparative Study of the Latest GPT Omni Models and BERT NLP for Customer Review Classification and Sentiment Analysis

Contact Info

Product

Resources

About