Multimodal interfaces and communication cues for remote collaboration

Kim, Seungwon; Billinghurst, Mark; Kim, Kangsoo

doi:10.1007/s12193-020-00346-8

Cited by 39 publications

(19 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MR remote collaboration is a type of mediated collaboration between a local worker and a remote helper (a.k.a. remote exert) to solve a physical task [17][44] [45]. The local worker is the person viewing the hand gesture to get a help from a remote helper and the remote helper is the person performing the hand gesture to help the local worker [46] [47].…”

Section: Methodsmentioning

confidence: 99%

“…However, in recent collaboration systems, each collaborator mostly has a first-person view and they have different viewpoint positions and perspective rather than identical view in a large task space [4] [17]. In the second study of our previous work [4] where both the gesture performer and the viewer had different perspective views and the performer was away from the task objects in a large task space, the HiA hand gesture was difficult to use.…”

Section: A Hand Gesture In Collaboration Systemmentioning

confidence: 97%

“…Hand gesture communication has been studied in many MR interfaces for collaboration [17]. In this section, we review previous studies of hand gesture communication, then explain the research gap between these previous studies and our work.…”

Section: Related Workmentioning

confidence: 98%

See 2 more Smart Citations

Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration

et al. 2020

Self Cite

View full text Add to dashboard Cite

Section: Methodsmentioning

confidence: 99%

Section: A Hand Gesture In Collaboration Systemmentioning

confidence: 97%

See 1 more Smart Citation

Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration

et al. 2020

Self Cite

View full text Add to dashboard Cite

“…A synchronized first-person view offers the same perspective and enables video conferencing or online meetings across devices or platforms through, for example, Webex ® , Google Meet ® , Skype ® , and Zoom ® . In remote collaboration, different AR system approaches have been developed to improve communication efficiency by applying gazevisualization platforms for physical tasks [62], 3D gesture and CAD models of mixed reality for training tasks [63], essential factors for remote collaboration [64], image-or live-video-based AR collaborations for industrial applications [65], remote diagnosis for complex equipment [66], 360 • video cameras (360 cameras) in augmented virtual teleportation (AVT) for high-fidelity telecollaboration [67], web-based extended reality (XR) for physical environments with physical objects [68], Industry 4.0 environments [59], 2D/3D telecollaboration [69], and indoor construction monitoring [70].…”

Section: Video Conferencing and Remote Collaboration With Armentioning

confidence: 99%

AR-Based 3D Virtual Reconstruction of Brick Details

Shih

2022

Remote Sensing

View full text Add to dashboard Cite

Building heritage contributes to the historical context and industrial history of a city. Brick warehouses, which comprise a systematic interface between components, demand an interactive manipulation of inspected parts to interpret their construction complexity. The documentation of brick details in augmented reality (AR) can be challenging when the relative location needs to be defined in 3D. This study aimed to compare brick details in AR, and to reconstruct the interacted result in the correct relative location. We applied photogrammetry modeling and smartphone AR for the first and secondary 3D reconstruction of brick warehouse details and compared the results. In total, 146 3D AR database models were created. The AR-based virtual reconstruction process applied multiple imagery resources from video conferencing and broadcast of models on the Augment® platform through a smartphone. Tests verified the virtual reconstruction in AR, and concluded the deviation between the final secondary reconstructed 3D model and the first reconstructed model had a standard deviation of less than 1 cm. AR enabled the study and documentation of cross-referenced results in comparison with the simplified reconstruction process, with structural detail and visual detail suitable for 3D color prints.

show abstract

“…Remote collaboration has been studied for more than three decades [8] with a major focus on increasing awareness: understanding what is going on in the task space and understanding collaborators' activities [5]. This is because awareness may be reduced in remote collaboration compared to co-located collaboration [9][10][11].…”

Section: Introductionmentioning

confidence: 99%

View Types and Visual Communication Cues for Remote Collaboration

Kim¹,

Huang²,

Oh³

et al. 2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Over the last several years, remote collaboration has been getting more attention in the research community because of the COVID-19 pandemic. In previous studies, researchers have investigated the effect of adding visual communication cues or shared views in collaboration, but there has not been any previous study exploring the influence between them. In this paper, we investigate the influence of view types on the use of visual communication cues. We compared the use of the three visual cues (hand gesture, a pointer with hand gesture, and sketches with hand gesture) across two view types (dependent and independent views), respectively. We conducted a user study, and the results showed that hand gesture and sketches with the hand gesture cues were well matched with the dependent view condition, and using a pointer with the hand gesture cue was suited to the independent view condition. With the dependent view, the hand gesture and sketch cues required less mental effort for collaborative communication, had better usability, provided better message understanding, and increased feeling of co-presence compared to the independent view. Since the dependent view supported the same viewpoint between the remote expert and a local worker, the local worker could easily understand the remote expert's hand gestures. In contrast, in the independent view case, when they had different viewpoints, it was not easy for the local worker to understand the remote expert's hand gestures. The sketch cue had a benefit of showing the final position and orientation of the manipulating objects with the dependent view, but this benefit was less obvious in the independent view case (which provided a further view compared to the dependent view) because precise drawing in the sketches was difficult from a distance. On the contrary, a pointer with the hand gesture cue required less mental effort to collaborate, had better usability, provided better message understanding, and an increased feeling of co-presence in the independent view condition than in the dependent view condition. The pointer cue could be used instead of a hand gesture in the independent view condition because the pointer could still show precise pointing information regardless of the view type.

show abstract

Multimodal interfaces and communication cues for remote collaboration

Cited by 39 publications

References 47 publications

Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration

Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration

AR-Based 3D Virtual Reconstruction of Brick Details

View Types and Visual Communication Cues for Remote Collaboration

Contact Info

Product

Resources

About