Towards using embedded magnetic field sensor for around mobile device 3D interaction

Ketabdar, Hamed; Roshandel, Mehran; Yüksel, Kamer Ali

doi:10.1145/1851600.1851626

Cited by 43 publications

(27 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, using simple IR proximity sensors fac-ing outwards [3] or upwards [23,24]. Magnetic field sensing has been used to track rigid motion around a device by pairing permanent magnets and magnetometers either by wearing both [4] or by using the device's built-in IMU (inertial measurement unit) [1,15,20]. Just like optical sensing solutions, magnetic field sensing can suffer from coarse sensing fidelity and input is limited to tracking a restricted number of discrete points and often with limited degrees-of-freedom (DoF).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

In-air gestures around unmodified mobile devices

Song

Sörös

Pece

et al. 2014

Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology

150

View full text Add to dashboard Cite

Figure 1: Touch input is expressive but can occlude large parts of the screen (A). We propose a machine learning based algorithm for gesture recognition expanding the interaction space around the mobile device (B), adding in-air gestures and hand-part tracking (D) to commodity off-the-shelf mobile devices, relying only on the device's camera (and no hardware modifications). We demonstrate a number of compelling interactive scenarios including bi-manual input to mapping and gaming applications (C+D). The algorithm runs in real time and can even be used on ultra-mobile devices such as smartwatches (E). ABSTRACTWe present a novel machine learning based algorithm extending the interaction space around mobile devices. The technique uses only the RGB camera now commonplace on off-the-shelf mobile devices. Our algorithm robustly recognizes a wide range of in-air gestures, supporting user variation, and varying lighting conditions. We demonstrate that our algorithm runs in real-time on unmodified mobile devices, including resource-constrained smartphones and smartwatches. Our goal is not to replace the touchscreen as primary input device, but rather to augment and enrich the existing interaction vocabulary using gestures. While touch input works well for many scenarios, we demonstrate numerous interaction tasks such as mode switches, application and task management, menu selection and certain types of navigation, where such input can be either complemented or better served by inair gestures. This removes screen real-estate issues on small touchscreens, and allows input to be expanded to the 3D space around the device. We present results for recognition accuracy (93% test and 98% train), impact of memory footprint and other model parameters. Finally, we report results from preliminary user evaluations, discuss advantages and limitations and conclude with directions for future work.

show abstract

Section: Related Workmentioning

confidence: 99%

“…For example using infrared (IR) proximity sensors [3,23] or handheld magnetic tags [20]. Others have attempted to address the input scarcity issue by leveraging the human body [11] or surfaces in the environment [10,34] as interactive platforms.…”

Section: Introductionmentioning

confidence: 99%

In-air gestures around unmodified mobile devices

Song

Sörös

Pece

et al. 2014

Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology

150

View full text Add to dashboard Cite

show abstract

“…Ketabdar et al [7] dealt with mobile phone interaction by using changes in magnetic field invoked by re-positioning of external magnetic material around the device. Machine learning algorithms were used for movement-based gestures classification.…”

Section: Related Workmentioning

confidence: 99%

Contact-free Interaction with Mobile Devices using Magnetic, Lighting and Infrared Sources

Ljubić

Arbula

2017

Int. J. Interact. Mob. Technol.

View full text Add to dashboard Cite

Abstract-In the area of Human-Computer Interaction (HCI), the concept of Around Device Interaction (ADI) is gaining special attention nowadays. It provides the possibility of interacting with the device by making use of gestures made in the surrounding space, as well as by detecting certain ambient changes, thus eliminating the need of direct physical contact. Contact-free interaction can be useful for wearable and smart devices, especially in cases where operating with small buttons and touchscreens appears to be particularly inconvenient. This paper presents alternative techniques for smartphone interaction based on different sensory inputs. Namely, external sources of magnetic field, light, and infrared (IR) signals are used to manipulate device sensors' readings which are in turn interpreted as the corresponding interaction commands. Proof-ofconcept mobile applications are developed and here presented, demonstrating both the potentials and utility of the contact-free interaction. Specifically, small neodymium magnet was used for smartphone text entry, typical battery flashlight served for sending information to device via Morse code, as well as for alarm triggering, and voice-enabled control device was utilized for invoking smartphone calls from a distance. The presented interaction techniques do not require smartphone hardware alterations. While the usage of built-in magnetic field and ambient light sensors is assumed, peripheral infrared receiver can be used in order to provide IR-based support.

show abstract

“…Other works reduce the hardware complexity even further by making use of the integrated sensors in the tablets. Ketabdar et al [7], for instance, exploit the magnetic (compass) sensor of the device, and interact around it using magnets. Unlike optical approaches, this solution is more robust to occlusion.…”

Section: Related Workmentioning

confidence: 99%

“…Probably for this reason most of them [1,7,8] require the interactions to take place close to the device. In a table-based MDE, this restriction could require that users lean over the table or even move towards the target device, which could be cumbersome and cause interference with interaction on the other devices.…”

Section: Related Workmentioning

confidence: 99%

Around-Device Interactions: A Usability Study of Frame Markers in Acquisition Tasks

Garcia-Sanjuan

Catalá

Fitzpatrick

et al. 2015

Human-Computer Interaction – INTERACT 2015

View full text Add to dashboard Cite

Abstract. Digital tabletops present numerous benefits in face-to-face collaboration environments. However, their integration in real settings is complicated by cost and fixed location. In this respect, building table-like environments using several handheld devices such as tablets or smartphones provides a promising alternative but is limited to touch interaction only. We propose instead another kind of "around-device" interaction (ADI) technique using the built-in front camera of these devices and fiducial frame markers, which presents advantages including better awareness and less interference. This paper contributes a first step in exploring the potential of this interaction technique by conducting a usability test comparing several ergonomic factors that may have an effect on the very first operation of the interaction: the acquisition of the marker.

show abstract

Towards using embedded magnetic field sensor for around mobile device 3D interaction

Cited by 43 publications

References 9 publications

In-air gestures around unmodified mobile devices

In-air gestures around unmodified mobile devices

Contact-free Interaction with Mobile Devices using Magnetic, Lighting and Infrared Sources

Around-Device Interactions: A Usability Study of Frame Markers in Acquisition Tasks

Contact Info

Product

Resources

About