Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

Dima, Elijs; Sjöström, Mårten

doi:10.1109/access.2021.3086894

Cited by 7 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A less explored area in AT is view augmentation through selective content removal from the presented view. The AROSS [11] is an AT system with augmentation through selective content removal and Novel Perspective view generation. The interplay and the relative benefits of these factors towards operator QoE, especially for the mining context, have not been investigated previously.…”

Section: Qoe For At In Remote Controlmentioning

confidence: 99%

“…The total time was estimated at approximately 75 minutes per participant. [11]. During Exp 1, an autostereoscopic 4K display was used for presenting the interface of the remote operation system.…”

Section: Experiments 1 Proceduresmentioning

confidence: 99%

“…In this study, we explored how in-view augmentations and interface distributions influence the remote operator's ability to perceive wall patterns, and estimate the 3D environment through an Augmented Telepresence (AT) interface. The study was performed with the usage of our Augmented Remote Operating System for Scaling in Mining (AROSS) [11]. The AROSS was initially built to demonstrate the technical feasibility of introducing real-time in-view augmentations (object removal, i.e., "Disocclusion Augmentation") and Novel Perspective view generation for the mining context.…”

Section: Introductionmentioning

confidence: 99%

“…"NP" denote views showing the scene from positions outside of the camera array at the remote location. Further technical details of AROSS are given in[11]. During Exp 1, an autostereoscopic 4K display was used for presenting the interface of the remote operation system.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Augmented remote operating system for scaling in smart mining applications: Quality of experience aspects

Shirin¹,

Elijs²,

Mårten³

et al. 2022

Self Cite

View full text Add to dashboard Cite

Remote operation and Augmented Telepresence are fields of interest for novel industrial applications in e.g., construction and mining. In this study, we report on an ongoing investigation of the Quality of Experience aspects of an Augmented Telepresence system for remote operation. The system can achieve view augmentation with selective content removal and Novel Perspective view generation. Two formal subjective studies have been performed with test participants scoring their experience while using the system with different levels of view augmentation. The participants also gave free-form feedback on the system and their experiences. The first experiment focused on the effects of in-view augmentations and interface distributions on wall patterns perception. The second one focused on the effects of augmentations on the depth and 3D environment understanding.The participants' feedback from experiment 1 showed that the majority of participants preferred to use the original camera views and the Disocclusion Augmentation view instead of the Novel Perspective views. Moreover, the Disocclusion Augmentation, that was shown in combination with other views seemed beneficial. When the views were isolated in experiment 2, the impact of the Disocclusion Augmentation view was found to be lower than the Novel Perspective views.

show abstract

Section: Qoe For At In Remote Controlmentioning

confidence: 99%

Section: Experiments 1 Proceduresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Augmented remote operating system for scaling in smart mining applications: Quality of experience aspects

Shirin¹,

Elijs²,

Mårten³

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

Investigation of human interaction with an augmented remote operating system for scaling in mining applications

Rafiei,

Brunnström,

Andersson

et al. 2024

Qual User Exp

View full text Add to dashboard Cite

Thanks to the advent of telepresence applications, we can remotely take control and operate industrial machinery. Teleoperation removes operators from hazardous workplaces such as mining and plays an essential role in the safety of workers. In addition, augmented telepresence can introduce information that helps the user understand the remote scene. However, remote operation presents challenges since the information received is more limited than what could be perceived by being physically present, such as accurately judging depth. This study investigates how well operators interact with an augmented remote operation scaling system (AROSS) in a mining context when different computer-generated visual interfaces are provided. The system can achieve five visual interfaces: Disocclusion Augmentation view using selective content removal, Novel Perspective view generation, Lidar view, Right (Original) view, and Left (Original) view. We performed two experiments in a mine-like laboratory to analyze human interaction with the designed prototype by applying a mixed research methodology that used questionnaires, interviews, and observations. This mixed methodology consisted of quality of experience methods to discover the users’ requirements from a technological standpoint and user experience methods (i.e., user-centric approaches). We investigated 10 and 11 users’ interactions in two experimental studies. The first experiment focused on identifying small patterns (e.g., cracks in the mine wall), and the second focused on depth and three-dimensional understanding. We considered the first experiment a feasibility test to understand how to conduct the second experiment. Therefore, we designed the second test to assess the technical readiness of AROSS from the users’ perspective. The overall conclusion yields a comprehensive understanding of users’ perceptions and experiences. The quality of experience results favored Left and Right (Original) views for remote control, indicating a preference among remote operators using natural (Original) views due to their facilitation of environmental comprehension. User experience analysis revealed the reason why other views were less favored and what their potential benefits are. Specifically, Novel Perspective and Lidar views were found helpful for depth perception, and Disocclusion Augmentation view functionality could be enhanced if robot arm position tracking is enabled. These insights inform design recommendations, emphasizing the value of incorporating Disocclusion Augmentation and Novel Perspective views and suggesting improvements to enhance system usability.

show abstract

Navigating contemporary challenges and future prospects in digital industry evolution

Gallab,

Di Nardo,

Naciri

2024

Discov Appl Sci

View full text Add to dashboard Cite

The advent of the digital industry, also known as Industry 4.0 is a transformation period in manufacturing, where the integration of digital technologies with physical systems is underlined. This transformation is crucial for the pillars of cyber-physical systems (CPS), cyber resilience protection, and workers’ safety, which collectively from the cornerstone of Industry 4.0. Ensuring the secure exploitation of Industry 4.0’s advantages necessitates a dedicated focus on workers’ safety, cyber resilience protection, and the security and privacy of human-centric CPS systems. However, the full realization of Industry 4.0’s potential hinges upon effectively addressing these challenges and aligning the benefits of digitalization with the exigencies of worker well-being and safeguarding critical infrastructure. This study is a comprehensive literature review on this digital era, focusing on CPS, resilience, and workers’ safety. The review aims to summarise current research and advancements, offering insights for researchers, practitioners, and decision-makers. By identifying gaps in knowledge, the study lays the foundations for additional research and supports ongoing progress in the digital industry. Industry 4.0 embodies a new era of manufacturing integrating digital technologies with a focus on sustainability and human-centric design, supported by CPS, resilience, and worker’s safety. This necessitate addressing challenges to ensure benefits align with worker needs and infrastructure protection.

show abstract

Camera and Lidar-Based View Generation for Augmented Remote Operation in Mining Applications

Cited by 7 publications

References 37 publications

Augmented remote operating system for scaling in smart mining applications: Quality of experience aspects

Augmented remote operating system for scaling in smart mining applications: Quality of experience aspects

Investigation of human interaction with an augmented remote operating system for scaling in mining applications

Navigating contemporary challenges and future prospects in digital industry evolution

Contact Info

Product

Resources

About