A Review of 3d Gesture Interaction for Handheld Augmented Reality

Yusof, Cik Suhaimi; Bai, Huidong; Billinghurst, Mark; Sunar, Mohd Shahrizal

doi:10.11113/jt.v78.6923

Cited by 13 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been several surveys or review papers, but their focuses are not on interaction techniques for 3D object manipulation in handheld mobile AR [2]- [4], [6]- [8]. Some papers have reviewed only a single interaction technique [9], [10]. Hence, this paper reviews the field of primary interaction techniques for 3D object manipulation in handheld mobile AR which can be categorized as touch-based, midair gestures-based, and device-based interaction techniques.…”

Section: A Motivationmentioning

confidence: 99%

“…Since AR had been claimed can provides intuitive interaction experiences, mid-air gestures-based interaction technique which is the intuitive interaction way had been widely explored in AR field including handheld mobile AR interface discussed above and so Yusof et al [10] produced a short review about mid-air gestures-based interaction technique in handheld mobile AR due to the active responses of many researchers towards this interaction. They summarized some of the related studies and discussed the existing limitations of the mid-air gestures-based interaction such as the quality of depth sensors nowadays directly affect the recognition of the hand or finger postures, self-occlusion and lack accuracy still the main issues to be cured although there have some researches that provide potential solutions towards these issues such as 3D finger CAPE that proposed by Jang et al [99] against occlusion estimated the 3D finger joint and 3D fingertip locations to form fingertip detector based on depth images.…”

Section: Figurementioning

confidence: 99%

See 1 more Smart Citation

3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review

2019

Self Cite

View full text Add to dashboard Cite

Handheld mobile devices referring to mobile phones and tablets become the major output medium for augmented reality (AR), which have seen significant growth in popularity and usage among the public due to the growing release of consumer-oriented communication products nowadays, especially touchscreen smartphones. Unlike traditional desktop or tabletop AR (large display-based) and head-mounted display-based AR systems, small display-based AR (handheld mobile display) requires different interaction techniques that mostly utilize single-hand interaction as well as the limitation of small screen display and limited activity time due to the battery operation hour of handheld mobile devices. However, in handheld mobile AR, research is still lacking, especially research that focuses on 3D interaction in handheld mobile AR for virtual object manipulation. Thus, this paper provides an overview of 3D interaction techniques in handheld mobile AR with critical analysis. First, we describe three main interaction technique categories that applicable in handheld mobile AR, which is touch-based interaction, mid-air gestures-based interaction, and device-based interaction techniques, of their basic concepts on 3D object manipulation. Then, we classify and systematize the highlighted techniques and discuss the advantages and drawbacks of each. Previous studies for widely used techniques have been studied comprehensively. We then draw up a comparison among the different techniques based on the important elements considered in handheld mobile AR. The aim of this paper is to provide researchers with background information on AR and those who are interested in the field of 3D interaction, realizing each technique category. INDEX TERMS 3D object manipulation, device-based interaction technique, mid-air gestures-based interaction technique, handheld mobile AR, touch-based interaction technique. I. INTRODUCTIONAccording to Milgram and Kishino [1], augmented reality (AR) refers to all cases in which the display of an otherwise real environment is augmented by means of virtual (computer graphic) objects. Extended from this definition, Azuma [2] defined that AR is the variation of virtual environments or virtual reality; however, unlike virtual reality that immerses a user inside a synthetic environment and the user cannot see the real world around him/her, AR allows the user to see the real world, with virtual objects composited with the real world. Beyond that, some researchers have doneThe associate editor coordinating the review of this manuscript and approving it for publication was Waleed Alsabhan. a number of surveys or reviews about AR. For example, Krevelen and Poelman [3], drawing up conclusions from several previous works, defined the AR system as: 1) combining real and virtual objects in a real environment; 2) registering real and virtual objects with each other and 3) running interactively, in 3D and in real-time. From another point of view, Mekni and Lemieux [4] proposed to define AR as systems that have the following charact...

show abstract

Section: A Motivationmentioning

confidence: 99%

Section: Figurementioning

confidence: 99%

3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review

2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Since humans use their hands as the main instrument for the manipulation of objects in real life, free hand interaction should be employed for the design of AR interfaces that will be accepted by end users [55]. Gesture interaction for handheld AR in particular directly utilizes the user's natural behavior; however most research related to the matter has focused on investigating the movement of fingers rather than of hands [56]. In the process of designing Audio Legends, the adoption most of the existing gesture based interaction modes and the inclusion of new ones were taken into consideration.…”

Section: Approaches To Audio Augmentationmentioning

confidence: 99%

Audio Legends: Investigating Sonic Interaction in an Augmented Reality Audio Game

Rovithis

Moustakas

Floros

et al. 2019

MTI

View full text Add to dashboard Cite

Augmented Reality Audio Games (ARAG) enrich the physical world with virtual sounds to express their content and mechanics. Existing ARAG implementations have focused on exploring the surroundings and navigating to virtual sound sources as the main mode of interaction. This paper suggests that gestural activity with a handheld device can realize complex modes of sonic interaction in the augmented environment, resulting in an enhanced immersive game experience. The ARAG “Audio Legends” was designed and tested to evaluate the usability and immersion of a system featuring an exploration phase based on auditory navigation, as well as an action phase, in which players aim at virtual sonic targets and wave the device to hit them or hold the device to block them. The results of the experiment provide evidence that players are easily accustomed to auditory navigation and that gestural sonic interaction is perceived as difficult, yet this does not affect negatively the system’s usability and players’ immersion. Findings also include indications that elements, such as sound design, the synchronization of sound and gesture, the fidelity of audio augmentation, and environmental conditions, also affect significantly the game experience, whereas background factors, such as age, sex, and game or music experience, do not have any critical impact.

show abstract

“…The drawbacks in handheld displays are that it only allows 2D pointing and tapping on touch screen [3]. Meanwhile, the other problem is when the user needs to hold the device with one hand and stretching out the other to manipulate the virtual, causing fatigue upon holding the device for long [4] besides it is a lack of intuitiveness.…”

Section: Introductionmentioning

confidence: 99%

An Inertial Device-based User Interaction with Occlusion-free Object Handling in a Handheld Augmented Reality

Goh¹,

Sunar²,

Ismail³

et al. 2018

IJIE

View full text Add to dashboard Cite

A Review of 3d Gesture Interaction for Handheld Augmented Reality

Cited by 13 publications

References 28 publications

3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review

3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review

Audio Legends: Investigating Sonic Interaction in an Augmented Reality Audio Game

An Inertial Device-based User Interaction with Occlusion-free Object Handling in a Handheld Augmented Reality

Contact Info

Product

Resources

About