A 5-Step Framework for Measuring the Quality of Experience (QoE) of Internet of Things (IoT) Services

Abstract

The ultimate benefit of the Internet of Things (IoT) is to improve quality of life by supporting the automation and interconnection of various services; this interconnection includes public facilities in platforms such as smart cities. IoT implementation is believed to improve services and yield additional benefits. However, assessment of how the IoT brings such an experience to users is still a challenge that is not yet standardized. Given that little research has discussed this issue, this paper aims to formulate a framework for measuring the quality of experience (QoE) regarding IoT services on the basis of the Absolute Category Rating with Hidden Reference (ACR-HR) scale, which measures the perception of experience by having the IoT users provide scores for their experiences before and after IoT implementation. The proposed framework consists of 5 steps: setting up the focus of the IoT services and the QoE parameters being assessed, defining users of IoT services, conducting a mean opinion score (MOS) survey of the users, calculating the DMOS (differential MOS) on the basis of the ACR-HR quantitative scale, and finally providing the strategic implications to those who implement the service. To test the proposed framework, we have conducted a proof of concept by measuring the QoE for the IoT services in the Jakarta Smart City platform. Six institutional users operate the IoT for their public services, ranging from fire and rescue brigades to public bus transportation. The results imply that the IoT-based services provide a good experience to institutional users when compared to the state of services without IoT. A strategic implication is that some features of Jakarta Smart City must be evaluated and improved to increase the level of satisfaction of users. To conclude, our framework is sufficiently reliable in measuring the QoE for IoT-based services and can be recommended for use for any other IoT-type service.

Success Factors Influencing Citizens’ Adoption of IoT Service Orchestration for Public Value Creation in Smart Government

“…It is predicted that there could be a 4.6 trillion US dollar economy concerning IoT in the public sector in the next decade [27]. E-gov research policies concerning IoT are limited due to its early development phase in the public sector [28], and the results of the IoT services in the Jakarta Smart City platform highlight the worthy experience of such services [29]. Governments are looking at IoT from various dimensions, including how IoT systems can improve efficiency, intelligence, and decision making in e-gov services, which motivates this study to probe the success factors influencing users' adoption of IoT devices for public value creation.…”

Quality of Experience in Internet of Things: A Systematic Literature Review

“…Even though subjective quality evaluating techniques, in which the quality of a services/application is measured through users' feedback and tools such as MOS, differential MOS and the ACR-HR [21] [22] [23], are the most popular used techniques, recently, their efficiency is widely questioned by researchers. In IoT, there are some automatic applications, wherein human feedback is not always available, thus, only subjective metrics may not be suitable to evaluate their overall QoE.…”

Is It a Smart City a Creative Place?

“…taking care of health (Ek et al, 2018)) and sound environment (Kang et al, 2018) which influences the comfort of life of urban community. According to Chen and Lin (2019) smart city architecture takes into account 20 areas including in particular information infrastructure based on ICT, artificial intelligence, the IoT or solutions based on a cloud and fog computing (Du et al, 2019;Lytras et al, 2019;Milošević et al, 2019;Sánchez-Teba & Bermúdez-González, 2019), e-administration, social insurance, intelligent transportation (Suryanegara et al, 2019), e.g. electric and autonomous cars (Si Min Lim & Taeihagh, 2019), intelligent diagnosis and treatment as well as video-mapping (Yang, 2019).…”