Augmented reality dialog interface for multimodal teleoperation

Pereira, André Grahl; Carter, Elizabeth; Leite, Iolanda; Mars, John; Lehman, Jill Fain

doi:10.1109/roman.2017.8172389

Cited by 14 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to the researchers, this system can be integrated to use for other fields like home care, healthcare, eHealth and education. Pereira, Carter [31] developed a teleoperation robot using verbal and nonverbal behavioural interaction of a single or dual user. Wang, Zheng [24] combined HMD and robotic technologies to manipulate the objects into the smart home.…”

Section: Related Workmentioning

confidence: 99%

A Smart Multimodal Augmented Reality Application Skill Training For Preoperative Procedures

Taçgin

Taçgın

2020

Bilişim Teknolojileri Dergisi

View full text Add to dashboard Cite

Ameliyat öncesi hazırlıkta cerrahi aletlerin mayo masasına dizilimi için temel kurallar bulunsa da belirli bir standart bulunmamaktadır. Hemşirelik öğrencileri yüksek riskli durumlar ve öğrenmeleri sırasında yetersiz geribildirim alma gibi sınırlılıklar sebebiyle gereken tecrübeyi kazanmak için yeterli olanaklara sahip değildir. Bu çalışma söz konusu bu sorunun çözümü için geliştirilen akili bir çoklu model artırılmış gerçeklik uygulamasını tanıtmaktadır. Sistem uzman cerrahi hemşirelerin dizilimlerini kullanarak oluşturulan 10 doğru ve 10 hatalı dizilimle eğitilmiştir. Bu uygulamayı kullanan öğrenciler ameliyat öncesi hazinlik aşamalarını gerçekleştirdikten sonra kendi dizilimlerinin uygunluğuna ilişkin bireysel geribildirim alabilmektedir. Fiziksel ek hareketleri ile etkileşimin sağlandığı bu uygulama, Meta2 artırılmış gerçeklik gözlüğü kullanılarak geliştirilmiştir. Bu çalışma, geliştirilen çoklu model artırılmış gerçeklik uygulamasının ara yüzü, özellikleri ve akıllı sistem yapısını açıklamaktadır. Anahtar Kelimeler-çoklu model arayüz, artırılmış gerçeklik, hemşirelik eğitimi, akıllı sistemler.

show abstract

Section: Related Workmentioning

confidence: 99%

A Smart Multimodal Augmented Reality Application Skill Training For Preoperative Procedures

Taçgin

Taçgın

2020

Bilişim Teknolojileri Dergisi

View full text Add to dashboard Cite

show abstract

“…Several works have shown effectiveness of using projector based approaches to augment robot's intentions into the shared workspace [2,4,16,20]. In particular, Andersen et al [2] proposed an objectaware projection technique which takes into account the 3D nature of the environment.…”

Section: Figurementioning

confidence: 99%

“…Similarly, because participants need to look away from the workspace and back at the monitor in MO, this will likely increase the number of errors. Our reasoning for establishing H3 and H5 is drawn from previous research showing that mixed reality applications can improve user experience [16,19]. H4 is argued for by reasoning that the augmented reality condition will enable participants to dedicate full attention to the workspace.…”

Section: Hypothesesmentioning

confidence: 99%

“…An alternative approach consists of employing visualisation techniques such as augmented [2] or mixed reality [8] to augment the scene with the robot's or human's intentions. While the first few works in this direction have started to appear [4,16,19], the effects of augmenting the workspace to disambiguate user verbal requests are still unknown.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction

Sibirtseva

Kontogiorgos

Nykvist

et al. 2018

2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)

Self Cite

View full text Add to dashboard Cite

Picking up objects requested by a human user is a common task in human-robot interaction. When multiple objects match the user's verbal description, the robot needs to clarify which object the user is referring to before executing the action. Previous research has focused on perceiving user's multimodal behaviour to complement verbal commands or minimising the number of follow up questions to reduce task time. In this paper, we propose a system for reference disambiguation based on visualisation and compare three methods to disambiguate natural language instructions. In a controlled experiment with a YuMi robot, we investigated real-time augmentations of the workspace in three conditions -mixed reality, augmented reality, and a monitor as the baseline -using objective measures such as time and accuracy, and subjective measures like engagement, immersion, and display interference. Significant differences were found in accuracy and engagement between the conditions, but no differences were found in task time. Despite the higher error rates in the mixed reality condition, participants found that modality more engaging than the other two, but overall showed preference for the augmented reality condition over the monitor and mixed reality conditions.

show abstract

“…By displaying critical information to the user it can increase their workspace awareness and productivity [4]. Novel input modes offered by AR systems, like gesture and voice commands, provide intuitive means of communication [5]. Increased user safety provided by AR can lead to reduced task completion time and costs.…”

Section: Introductionmentioning

confidence: 99%

Intuitive and Safe Interaction in Multi-User Human Robot Collaboration Environments through Augmented Reality Displays

Tsamis

Chantziaras

Giakoumis

et al. 2021

2021 30th IEEE International Conference on Robot &Amp; Human Interactive Communication (RO-MAN)

View full text Add to dashboard Cite

As autonomous collaborative robots are more widely used in work environments alongside humans it is of great importance to facilitate the communication between people and robotic systems, in a way that promotes safety and productivity. To this end, we propose an Augmented Reality (AR) based system that allows workers in a human-robot collaborative environment to interact with a robot while also receiving information regarding the robot state and plans that relate to the human's safety and trust, such as the intended movement of the robotic arm or the navigation plan of the mobile platform. To evaluate the effectiveness of the proposed system we conducted experiments with 13 participants, where two users had to work in the same workspace while being assisted by a mobile manipulator. We measured the task completion time as well as the robot idle time using our ARbased human-robot interaction system and compared them to a conventional setup without the use of augmented reality. Additional, subjective evaluations related to user satisfaction, system usability, perceived safety and trust showed that users assessed the system in a positive way and preferred AR visualization over more traditional interfaces.

show abstract

Augmented reality dialog interface for multimodal teleoperation

Cited by 14 publications

References 17 publications

A Smart Multimodal Augmented Reality Application Skill Training For Preoperative Procedures

A Smart Multimodal Augmented Reality Application Skill Training For Preoperative Procedures

A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction

Intuitive and Safe Interaction in Multi-User Human Robot Collaboration Environments through Augmented Reality Displays

Contact Info

Product

Resources

About