A Robust Camera-Based Interface for Mobile Entertainment

Abstract

Camera-based interfaces in mobile devices are starting to be used in games and apps, but few works have evaluated them in terms of usability or user perception. Due to the changing nature of mobile contexts, this evaluation requires extensive studies to consider the full spectrum of potential users and contexts. However, previous works usually evaluate these interfaces in controlled environments such as laboratory conditions, therefore, the findings cannot be generalized to real users and real contexts. In this work, we present a robust camera-based interface for mobile entertainment. The interface detects and tracks the user’s head by processing the frames provided by the mobile device’s front camera, and its position is then used to interact with the mobile apps. First, we evaluate the interface as a pointing device to study its accuracy, and different factors to configure such as the gain or the device’s orientation, as well as the optimal target size for the interface. Second, we present an in the wild study to evaluate the usage and the user’s perception when playing a game controlled by head motion. Finally, the game is published in an application store to make it available to a large number of potential users and contexts and we register usage data. Results show the feasibility of using this robust camera-based interface for mobile entertainment in different contexts and by different people.

Collateral effects of the Kalman Filter on the Throughput of a Head-Tracker for Mobile Devices

“…FaceMe [RMMYV16] is a head-tracker interface for mobile devices that uses the information of the front camera to detect and track the user's nose position and translate its movements into interaction actions to the device (Figure 1). A version of SINA system [VMYP08], a camera-based head-tracker interface for desktop environment, was adapted and optimized for mobile devices.…”

“…Our tracking stage is able to run in real-time on current mobile devices with a variety of CPU platforms. A detailed description of the system is found in other sources [RMMYV16].…”

“…We have developed an image-based head-tracker interface for mobile devices [RMMYV16] that only uses the information of the front camera to detect and track the user's nose position and translate its movements into a pointing metaphor to the device. However, as already Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Estado del Arte en Neurotecnologías para la Asistencia y la Rehabilitación en España: Tecnologías Auxiliares, Trasferencia Tecnológica y Aplicación Clínica

“…La primera versión de este sistema permitía el control del cursor mediante el seguimiento de la nariz (Manresa-Yee et al, 2010) y la segunda versión mediante el movimiento de los ojos (Garcés et al, 2011). Recientemente, se ha actualizado el modelo de interacción en forma de head-tracker para dispositivos móviles por la evolución tecnológica y comercial que han tenido dichos dispositivos (Roig-Maimó et al, 2016a). Esto ha permitido a su vez realizar un estudio de su aplicabilidad a la accesibilidad de dispositivos móviles para usuarios con esclerosis múltiple (RoigMaimó et al, 2016b).…”

Sensors for Entertainment

“…Roig-Maimó et al [ 9 ] focus on user interaction with mobile devices by using the onboard camera as a pointing device. A game is created that can be controlled by head movements.…”