A First Look at Human Values-Violation in App Reviews

Abstract

Ubiquitous technologies such as mobile software applications (mobile apps) have a tremendous influence on the evolution of the social, cultural, economic, and political facets of life in society. Mobile apps fulfil many practical purposes for users including entertainment, transportation, financial management, etc. Given the ubiquity of mobile apps in the lives of individuals and the consequent effect of these technologies on society, it is essential to consider the relationship between human values and the development and deployment of mobile apps. The many negative consequences of violating human values such as privacy, fairness or social justice by technology have been documented in recent times. If we can detect these violations in a timely manner, developers can look to better address them. To understand the violation of human values in a range of common mobile apps, we analysed 22,119 app reviews from Google Play Store using natural language processing techniques. We base our values violation detection approach on a widely accepted model of human values; the Schwartz theory of basic human values. The results of our analysis show that 26.5% of the reviews contained text indicating user perceived violations of human values. We found that benevolence and self-direction were the most violated value categories, and conformity and tradition were the least violated categories. Our results also highlight the need for a proactive approach to the alignment of values amongst stakeholders and the use of app reviews as a valuable additional source for mining values requirements.

Human Values Violations in Stack Overflow: An Exploratory Study

“…Shams et al [40] analysed 1,522 reviews from 29 Bangladeshi agricultural apps to understand both the desired and missing values that should be addressed in the development of such apps. Furthermore, Obie et al [31] introduced a dictionary-based natural language processing technique for detecting the violation of human values in app reviews. The result of their study showed that 26.5% of the analyzed 22,119 app reviews contained perceived violations of human values by the end-users.…”

“…As important steps towards addressing the violation of the values of mobile apps users in society, Obie et al [31] proposed the mining of values requirements from rich data sources, the alignment of values between stakeholders in SE projects, and the adoption of critical technical practice in mobile SE. A recent study further contends that careful consideration of domain context in the design and application of values instruments should be made during values requirements gathering, as the hierarchy of end-users values may vary depending on the end-users domain context [32].…”

“…To do so, we developed a list of regex keywords and phrases that are likely to be associated with human values violations concerns, such as "moral", "ethical", "human-cent", and "society". This list was designed based on our observations from the 300 comments and their corresponding posts labeled in the first step and consulting a dictionary of human values-related keywords and phrases developed in [31] for identifying human values violations in app user reviews. This list was adjusted over several query runs, reducing to 21 regex keywords, and was used to design a SQL SELECT query to identify comments that contained one of the keywords in the list.…”

“…The SQL SELECT query returned 10144 comments. Note that keywords have also been used in previous studies (e.g., [31,51]) to fine-tune data collection and minimize false positives. The list of 21 keywords and phrases is available in our replication package [25].…”

“…The ultimate goal is to develop software systems that better reflect and respect human values. Other lines of research aimed to understand which types of human values are discussed by developers in GitHub's issue tracking systems [30] and analyzed app reviews to determine which human values are ignored or violated by apps [31,40].…”