Comparative Analysis of Classical and Deep Learning-based Natural Language Processing for Prioritizing Customer Complaints

Abstract

Recent advancements in natural language processing have been shown to be very effective for different text mining tasks and thus have provided the opportunity to enhance service research. To improve the customer service experience, this paper compares several natural language processing approaches in order to automatically prioritize incoming customer complaints for service agents. This can help companies to reduce customers' friction and enable effective resource allocations. Our paper uses stateof-the-art feature engineering techniques (e.g., term frequency, TF-IDF and Word2Vec) to identify key words that could enable machine to prioritize complainers. We experimented with many classical machine learning classification algorithms, such as Random Forests, Support Vector Machines, Decision Trees and Logistic Regression, as well as with deep learning-based classifiers, such as convolutional neural networks, bidirectional long short-term memory, and the pre-trained language model BERT to compare the model performance. Our findings show that the pre-trained language model BERT and TF-IDF in combination with Logistic Regression yields the highest macro averaged F1-score across the multiple classes and is therefore most capable of predicting the priority group of incoming customer complaints.