Studying the consistency of star ratings and the complaints in 1 & 2-star user reviews for top free cross-platform Android and iOS apps

Abstract

How users rate a mobile app via star ratings and user reviews is of utmost importance for the success of an app. Recent studies and surveys show that users rely heavily on star ratings and user reviews that are provided by other users, for deciding which app to download. However, understanding star ratings and user reviews is a complicated matter, since they are influenced by many factors such as the actual quality of the app and how the user perceives such quality relative to their expectations, which are in turn influenced by their prior experiences and expectations relative to other apps on the platform (e.g., iOS versus Android). Nevertheless, star ratings and user reviews provide developers with valuable information for improving the software quality of their app. In an effort to expand their revenue and reach more users, app developers commonly build cross-platform apps, i.e., apps that are available on multiple platforms. As star ratings and user reviews are of such importance in the mobile app industry, it is essential for developers of cross-platform apps to maintain a consistent level of star ratings and user reviews for their apps across the various platforms on which they are available. In this paper, we investigate whether cross-platform apps achieve a consistent level of star ratings and user reviews. We manually identify 19 cross-platform apps and conduct an empirical study on their star ratings and user reviews. By manually tagging 9,902 1 & 2-star reviews of the studied cross-platform apps, we discover that the distribution of the frequency of complaint types varies across platforms. Finally, we study the negative impact ratio of complaint types and find that for some apps, users have higher expectations on one platform. All our proposed techniques and our methodologies are generic and can be used for any app. Our findings show that at least 68% of the studied cross-platform apps do not have consistent star ratings, which suggests that different quality assurance efforts need to be considered by Abstract How users rate a mobile app via star ratings and user reviews is of utmost importance for the success of an app. Recent studies and surveys show that users rely heavily on star ratings and user reviews that are provided by other users, for deciding which app to download. However, understanding star ratings and user reviews is a complicated matter, since they are influenced by many factors such as the actual quality of the app and how the user perceives such quality relative to their expectations, which are in turn influenced by their prior experiences and expectations relative to other apps on the platform (e.g., iOS versus Android). Nevertheless, star ratings and user reviews provide developers with valuable information for improving the software quality of their app.In an effort to expand their revenue and reach more users, app developers commonly build cross-platform apps, i.e., apps that are available on multiple platforms. As star ratings and user reviews are of such importance in the mobil...

Analysing app reviews for software engineering: a systematic literature review

“…We observed an interest in use of descriptive and inferential techniques for Content Analysis e.g., Vasa et al (2012), Pagano and Maalej (2013), Mercado et al (2016), Guzman et al (2018), and Wang et al (2020a). Summary statistics, box plots, and cumulative distribution charts help to gain understanding of review characteristics like their vocabulary size (Hoon et al 2012;Vasa et al 2012), issue type distribution (McIlroy et al 2016;Hu et al 2018;Williams et al 2020), or topics these reviews convey (Pagano and Maalej 2013;Srisopha and Alfayez 2018). Scholars employ different statistical tests to test check their hypothesis (Khalid et al 2016;Guzman and Paredes Rojas 2019;Franzmann et al 2020), to examine relationship between reviews characteristics (Srisopha and Alfayez 2018; Guzman and Paredes Rojas 2019; Di Sorbo et al 2020), and to study how sampling bias affects the validity of research results (Martin et al 2015).…”

“…Interestingly, studies have pointed out that users' perception for the same apps can vary per country (Srisopha et al 2019), user gender (Guzman and Paredes Rojas 2019), development framework (Malavolta et al 2015a), and app store (Ali et al 2017). Content analysis can be also beneficial for software engineers to understand whether cross-platform apps achieve consistency of users' perceptions across different app stores (Hu et al 2018;Hu et al 2019), or whether hybrid development tools achieve their main purpose: delivering an app that is perceived similarly by users across platforms (Hu et al 2019). Finally, studying the dialogue between users and developers has shown evidences that the chances of users to update their rating for an app increase as result of developer's response to reviews (McIlroy et al 2015;.…”

Automatic API Usage Scenario Documentation from Technical Q&A Sites

“…We use Wilcoxon Mann-Whitney U test to determine whether the difference between any two settings for a given variable is statistically significant (i.e, 𝛼 ≤ 0.05). Mann-Whitney U test is non-parametric, which is used in software engineering literature to compare data that is not normally distributed [31]. We estimate the effect size between the distributions.…”

The Role of Steps and Game Elements in Gamified Fitness Tracker Apps: A Systematic Review

“…Since the range for the number of reviews was large, we used a logarithmic scale in the y-axis to aid differentiating apps in this dimension. Although apps generally tend to get better ratings on Android than in the iOS platform [34,35], gamified fitness tracker apps seemed to be breaking this trend as they were more favourably rated in the iOS platform than in the Android platform as shown in the left part of Figure 2. Our final dataset has been previously published and is freely available via an openaccess creative commons license [36].…”