Hands in Space: Gesture Interaction with Augmented-Reality Interfaces

Billinghurst, Mark; Piumsomboon, Thammathip; Bai, Huidong

doi:10.1109/mcg.2014.8

Cited by 56 publications

(25 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…According to [35] more researches have to be carried out on a multimodal combined strategy and how to combine gesture inputs and speeches so that the intent of the users can be conveyed to AR applications. The authors in [36] have stated in their study that the multimodal interface can be extended further to other AR application domains to see the benefits seen in the virtual environment whether it can be expanded to different fields.…”

Section: Advances In Multimediamentioning

confidence: 99%

User Satisfaction for an Augmented Reality Application to Support Productive Vocabulary Using Speech Recognition

Hashim

Majid

Arshad

et al. 2018

Advances in Multimedia

View full text Add to dashboard Cite

The aim of this paper is to evaluate user satisfaction based on System Usability Scale (SUS) questionnaire on Augmented Reality (AR) application for productive vocabulary using speech recognition. There is still lack of research focus on user satisfaction towards the use of AR-based app using speech recognition for vocabulary learning in early education. The first objective of this paper is to develop an AR application for children using speech recognition to enhance productive vocabulary learning that integrates visual script (orthography) and audio (phonology). The second objective is to evaluate and identify the user satisfaction in enhancing productive vocabulary methods by combining AR technology and speech recognition to the parents and teachers by doing a mixed method testing such as questionnaire and interview. To achieve this, an interview session was conducted with the experts and SUS questionnaire was given to the teachers and parents of the student to evaluate user satisfaction. The results show that the research hypotheses of this study were successfully achieved. It revealed that the teachers and students were satisfied with the application based on SUS score (SUS score > 68). Since the total SUS score is 80.3, which is above average, it shows that they were satisfied with the overall application. Apart from that, the mean of the usability study based on questionnaire also shows that the users have found that this application is usable in terms of learnability study and that this application is easy to use.

show abstract

Section: Advances In Multimediamentioning

confidence: 99%

User Satisfaction for an Augmented Reality Application to Support Productive Vocabulary Using Speech Recognition

Hashim

Majid

Arshad

et al. 2018

Advances in Multimedia

View full text Add to dashboard Cite

show abstract

“…Citra yang dihasilkan dari depth camera diolah dan kemudian diklasifikasikan menggunakan algoritma Support Vector Machine (SVM). Selain itu, terdapat penelitian mengenai penggunaan tangan sebagai interaksi alamiah dengan Augmented-Reality Interfaces [3] dan penelitian mengenai pengenalan bahasa isyarat yaitu pengenalan American Sign Language (ASL) berdasarkan nilai estimasi pose tangan dari depth sensor secara real time [4]. Penggunaan Kinect dalam penelitian teah banyak dilakukan diantaranya penelitian mengenai metodeuntuk mendeteksi tangan dan gerakan tangan menggunakan depth sensor Kinect [5][6], penelitian untuk memperkirakan pose kepala secara real time menggunakan depth sensor [7], dan penelitian mengenai skema untuk mengekstraksi tangan dari depth image [8].…”

Section: Pendahuluanunclassified

Pengenalan Pose Tangan Menggunakan HuMoment

Utami¹,

Ichwan²

2017

INFOTEL

View full text Add to dashboard Cite

Abstrak -Computer vision yang didasarkan pada pengenalan bentuk memiliki banyak potensi dalam interaksi manusia dan komputer. Pose tangan dapat dijadikan simbol interaksi manusia dengan komputer seperti halnya pada penggunaan berbagai pose tangan pada bahasa isyarat. Berbagai pose tangan dapat digunakan untuk menggantikan fungsi mouse, untuk mengendalikan robot, dan sebagainya. Penelitian ini difokuskan pada pembangunan sistem pengenalan pose tangan menggunakan HuMoment. Proses pengenalan pose tangan dimulai dengan melakukan segmentasi citra masukan untuk menghasilkan citra ROI (Region of Interest), yaitu area telapak tangan. Selanjutnya, dilakukan proses deteksi tepi. Kemudian, dilakukan ekstraksi nilai HuMoment. Nilai HuMoment dikuantisasikan ke dalam bukukode yang dihasilkan dari proses pelatihan menggunakan K-Means. Proses kuantisasi dilakukan dengan menghitung nilai Euclidean Distance terkecil antara nilai HuMoment citra masukan dan bukukode. Berdasarkan hasil penelitian, nilai akurasi sistem dalam mengenali pose tangan adalah 88,57%. Various hand poses can be used to replace the mouse, motion control robots, and others. This study focused on the development of a hand pose recognition system using HuMoment quantization. The process begins with image segmentation to generate the image ROI (Region of Interest). Then the edge detection proess is performed. After that, the value of HuMoment is extracted. The HuMoment value quantized into codebook which is resulted from the training process using K-Means. Quantization process is done by calculating the smallest Euclidean Distance between the HuMoment of input image and codebook. Based on this research, the recognition accuracy is 88.57%.Keywords -Hand Pose Recognition; Computer Vision; HuMoment; Depth Image; Kinect I. PENDAHULUANDengan semakin meningkatnya penggunaan komputer dalam kehidupan sehari-hari, maka muncul tantangan baru bagaimana memberikan kenyamanan dan kemudahan dalam menggunakan komputer, salah satunya adalah kebutuhan akan metode interaksi dan komunikasi yang lebih alami antara pengguna dan computer.Penggunaan pose tangan merupakan salah satu solusi untuk memenuhi kebutuhan manusia dalam interaksi manusia dan komputer (Computer Human Interaction) yang lebih intuitif, cepat, dan sejalan dengan fungsi natural anggota tubuh manusia. Tangan dapat menghasilkan beragam pose. Setiap pose memiliki maksud dan makna tersendiri sesuai dengan kesepakatan umum ataupun kesepakatan diantara yang melakukan komunikasi. Oleh karena itu pose tangan dapat digunakan untuk menggantikan fungsi mouse, untuk mengontrol gerak robot, dan sebagainya.Depth-sensing dan gesture-tracking seperti Microsoft Kinect atau Leap Motion Controller merupakan contoh inovasi teknologi dalam metode interaksi manusia dan komputer. Teknologi tersebut memberikan peluang kepada pengguna untuk berinteraksi dengan komputer menggunakan gerakan tangan atau badan. Namun, sistem pengenalan gerakan yang digunakan berdasakan pada posisi skeleton bukan pose atau bentuk.

show abstract

“…The Samsung Galaxy features "Air Gesture" control interfaces, 15 which interpret waving hand gestures, using a front camera and proximity sensor. Billinghurst et al used smartphones to acquire hand gestures captured by a mobile camera (Billinghurst et al 2014), overlaying mixed reality graphics stabilized by internal position sensors. This interface is somewhat different from our system, which uses smartphone position data directly, but which effects are displayed across the network.…”

Section: Related Toys Devices Apps and Systemsmentioning

confidence: 99%

“Twhirleds”: Spun and whirled affordances controlling multimodal mobile-ambient environments with reality distortion and synchronized lighting to preserve intuitive alignment

Cohen

Ranaweera

Ryskeldiev

et al. 2017

Sci Phone Appl Mob Devices

View full text Add to dashboard Cite

The popularity of the contemporary smartphone makes it an attractive platform for new applications. We are exploring the potential of such personal devices to control networked displays. In particular, we have developed a system that can sense mobile phone orientation to support two kinds of juggling-like play styles: padiddle and poi. Padiddling is spinning a flattish object (such as a tablet or board-mounted smartphone) on the tip of one's finger. Poi involves whirling a weight (in this case the smartphone itself) at the end of a tether. Orientation of a twirled device can be metered, and with a communications infrastructure, this streamed azimuthal data can be used to modulate various distributed, synchronous, multimodal displays, including panoramic and photospherical imagery, diffusion of pantophonic and periphonic auditory soundscapes, and mixed virtuality scenes featuring avatars and props animated by real-world twirling. The unique nature of the twirling styles allows interestingly fluid perspective shifts, including orbiting "inspection gesture" virtual cameras with self-conscious ambidextrous avatars and "reality distortion" fields with perturbed affordance projection.

show abstract

Hands in Space: Gesture Interaction with Augmented-Reality Interfaces

Abstract: Researchers at the Human Interface Technology Laboratory New Zealand (HIT Lab NZ) are investigating free-hand gestures for natural interaction with augmented-reality interfaces. They've applied the results to systems for desktop computers and mobile devices.

Cited by 56 publications

References 5 publications

User Satisfaction for an Augmented Reality Application to Support Productive Vocabulary Using Speech Recognition

User Satisfaction for an Augmented Reality Application to Support Productive Vocabulary Using Speech Recognition

Pengenalan Pose Tangan Menggunakan HuMoment

“Twhirleds”: Spun and whirled affordances controlling multimodal mobile-ambient environments with reality distortion and synchronized lighting to preserve intuitive alignment

Contact Info

Product

Resources

About