Investigating the criticality of user‐reported issues through their relations with app rating

Abstract

App quality impacts user experience and satisfaction. As a consequence, both app ratings and user feedback reported in app reviews are directly influenced by the user-perceived app quality. Through an empirical study involving 210,517 reviews related to 317 Android apps, in this paper, we experiment with the combined usage of app rating and user reviews analysis (i) to investigate the most important factors influencing the perceived app quality, (ii) focusing on the topics discussed in user review that most relate with app rating. Besides, we investigate whether specific code quality metrics could be monitored to prevent the rising of negative user feedback (i.e., types of user review comments), connected with low ratings. Our study demonstrates that user comments reporting bugs are negatively correlated with the rating, while reviews reportingfeature requests do not. Interestingly, depending on the app category, we observed that different kinds of issues have rather different relationships with the rating and the user-perceived quality of the app. In particular, we observe that for specific app categories (e.g., communication), some code quality factors have significant relationships with the raising of certain types of feedback, which, in turn, are negatively connected with app ratings.

Analysing app reviews for software engineering: a systematic literature review

“…Statistical analysis is used in many papers to report research results (Martin et al 2015;Sänger et al 2016;Di Sorbo et al 2020), demonstrate their significance (Vasa et al 2012;Khalid et al 2016), and draw conclusions of a large population of reviews by analysing their tiny portion (Pagano and Maalej 2013;Mercado et al 2016;Wang et al 2020a). We observed an interest in use of descriptive and inferential techniques for Content Analysis e.g., Vasa et al (2012), Pagano and Maalej (2013), Mercado et al (2016), Guzman et al (2018), and Wang et al (2020a).…”

“…Classification is by far the most common type of app review analysis found in the literature: 58% of publications describe techniques for classifying reviews. Classification can be used to separate informative reviews from those that are uninformative (e.g., Oh et al 2013;Chen et al 2014;Di Sorbo et al 2016;Di Sorbo et al 2020), spam (Chandy and Gu 2012) or fake (Martens and Maalej 2019b). Informative reviews can be subsequently classified to detect user intentions (e.g., Zhou et al 2020) and discussion topics (e.g., Di van Vliet et al 2020).…”

“…Studies have shown that users discuss diverse topics in reviews (Pagano and Maalej 2013;Shams et al 2020), such as app features, qualities (Williams and Mahmoud 2018;Franzmann et al 2020), requirements or issues (Khalid 2013;Alqahtani and Orji 2019;Kalaichelavan et al 2020;Williams et al 2020). For example, using content analysis, researchers analysed recurring types of issues reported by users (McIlroy et al 2016;Wang et al 2020a;Shams et al 2020), their distribution in reviews as well as as relations between app issue type and other information such as price and rating (Iacob et al 2013b; or between issue type and code quality indicators (Di Sorbo et al 2020). Interestingly, studies have pointed out that users' perception for the same apps can vary per country (Srisopha et al 2019), user gender (Guzman and Paredes Rojas 2019), development framework (Malavolta et al 2015a), and app store (Ali et al 2017).…”

Predicting issue types on GitHub

“…Software maintenance involves tasks for mitigating potential defects in the code, as well as for evolving it according to the users' emerging needs [1]. Thus, it is crucial for the success of software projects.…”

An NLP-based Tool for Software Artifacts Analysis

“…Software developers intensively rely on of software repositories [1], [9], [29] and written communication channels [7], [24] for exchanging relevant information about the ongoing development tasks and the status of overall project progress. Development teams contributing to the software repositories [1], [9], [29] are globally distributed [7]: they often (i) communicate in unstructured form through mailing lists [5] and chats [2], (ii) discuss problems and related solutions over issue trackers [20], [21] and Question & Answer forums (e.g., Stack Overflow) [31], and (iii) acquire feedback posted by users on dedicated channels, as in the case of mobile apps (e.g., app stores) [14]. Therefore, valuable information is disseminated (and scattered) over semi-structured and unstructured software artifacts [7], [9], [24].…”