Using HoloLens to create a virtual operator station for mobile robots

Kot, Tomáš; Novák, Petr; Bajak, Jan

doi:10.1109/carpathiancc.2018.8399667

Cited by 13 publications

(7 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In case of sensor data, VAs mainly consist of graphic elements that exclusively display proprioceptive sensors data [2] [10]. Only in [3] some information about remote terrain (namely slope) is given to teleoperators.…”

Section: B Mixed Reality Representationsmentioning

confidence: 99%

“…Only in [3] some information about remote terrain (namely slope) is given to teleoperators. Other interfaces propose the use of stereo cameras to increase spatial awareness, but only recent works combine stereoscopic views with head-mounted displays (HMDs) [10] [12][31]- [36].…”

Section: B Mixed Reality Representationsmentioning

confidence: 99%

“…better representations of sensors information [3], wider range of available commands and options [4], and more intuitive dashboards including the use of three-dimensional displays [5]. Despite the use of immersive displays and mixed reality (MR) representations has often been discussed [6]- [10], it is rare to find it on commercial products. There is nonetheless consensus about its potential and advantage in increasing operator's sense of presence in remote workspaces.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Intuitive Robot Teleoperation Through Multi-Sensor Informed Mixed Reality Visual Aids

et al. 2021

View full text Add to dashboard Cite

The work carried out by the University of Catania is in the framework of the project "Safe and Smart Farming with Artificial Intelligence and Robotics-programma ricerca di ateneo UNICT 2020-22 linea 2" ABSTRACT Mobile robotic systems have evolved to include sensors capable of truthfully describing robot status and operating environment as accurately and reliably as never before. This possibility is challenged by effective sensor data exploitation, because of the cognitive load an operator is exposed to, due to the large amount of data and time-dependency constraints. This paper addresses this challenge in remote-vehicle teleoperation by proposing an intuitive way to present sensor data to users by means of using mixed reality and visual aids within the user interface. We propose a method for organizing information presentation and a set of visual aids to facilitate visual communication of data in teleoperation control panels. The resulting sensor-information presentation appears coherent and intuitive, making it easier for an operator to catch and comprehend information meaning. This increases situational awareness and speeds up decision-making. Our method is implemented on a real mobile robotic system operating outdoor equipped with on-board internal and external sensors, GPS, and a reconstructed 3D graphical model provided by an assistant drone. Experimentation verified feasibility while intuitive and comprehensive visual communication was confirmed through an assessment, which encourages further developments.

show abstract

Section: B Mixed Reality Representationsmentioning

confidence: 99%

Section: B Mixed Reality Representationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Intuitive Robot Teleoperation Through Multi-Sensor Informed Mixed Reality Visual Aids

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Therefore, it should be mentioned that three other types of AR glasses were tested, as during preliminary tests, namely Meta 2, Epson Moverio BT-200 (used e.g., by Angrisani et al, 2018 [ 22 ]), Epson Moverio BT-300, and the Google Glass. Only HoloLens was chosen for the in-depth study (used e.g., by Ingeson et al, 2019 [ 15 ]; Müller et al, 2018 [ 17 ]; Kot et al, 2018b [ 24 ]; Vorraber et al, 2019 [ 28 ]; Sluganovic et al, 2017 [ 58 ]; Xue et al, 2019 [ 62 ]) and presented the most advanced technology with the widest vision field and 3D space, giving the most user-friendly interface, especially in accordance with the assumptions made in this research. Because of the assumptions mentioned above, the AR support system was not adapted and installed on other models of AR glasses.…”

Section: Technology Imperfections Limitations and Future Developmentioning

confidence: 99%

Assessment of Augmented Reality in Manual Wiring Production Process with Use of Mobile AR Glasses

Szajna

Stryjski

Woźniak

et al. 2020

Sensors

View full text Add to dashboard Cite

Digitalization of production environment, also called Industry 4.0 (the term invented by Wahlster Wolfgang in Germany) is now one of the hottest topics in the computer science departments at universities and companies. One of the most significant topics in this area is augmented reality (AR). The interest in AR has grown especially after the introduction of the Microsoft HoloLens in 2016, which made this technology available for researchers and developers all around the world. It is divided into numerous subtopics and technologies. These wireless, see-through glasses give a very natural human-machine interface, with the possibility to present certain necessary information right in front of the user’s eyes as 3D virtual objects, in parallel with the observation of the real world, and the possibility to communicate with the system by simple gestures and speech. Scientists noted that in-depth studies connected to the effects of AR applications are presently sparse. In the first part of this paper, the authors recall the research from 2019 about the new method of manual wiring support with the AR glasses. In the second part, the study (tests) for this method carried out by the research team is described. The method was applied in the actual production environment with consideration of the actual production process, which is manual wiring of the industrial enclosures (control cabinets). Finally, authors deliberate on conclusions, technology’s imperfections, limitations, and future possible development of the presented solution.

show abstract

“…To be more precisethere are different research areas, where control and improvement of how to better use and visualise different information, about the VR and Augmented Reality (AR). VR simulation could also be used in manufacturing, robotics and control systems [1][2][3][4] by simulating different algorithms and control methods. Manufacturing, robotics, and control Architecture [5] of varying scale rooms and buildings, for finding the most efficient way of design.…”

Section: Introductionmentioning

confidence: 99%

Digital Twin Based Synchronised Control and Simulation Of the Industrial Robotic Cell Using Virtual Reality

Kuts

Otto

Tähemaa

et al. 2019

Journal of Machine Engineering

View full text Add to dashboard Cite

During the years common understanding of the possibilities and perspectives of Virtual Reality (VR) usage has been changed. It is thought that VR is mainly used in entertainment purposes, but it is being used already for many years in different industries, and now with easier access to the hardware it became a helpful and accessible tool that could be used and developed in any field of human activities. In manufacturing, immersive technologies are mainly used nowadays for the visualisation of processes and products combining those visuals into the factory Digital Twin (DT) which is possible to view from the inside look. This feature is already being used in several manufacturing simulation tools, which enable to view onto industrial line / robotic cells via Virtual Reality glasses. However, the potential of using simulations with VR in manufacturing is not fully uncovered. The main aim of this, industrial robotics targeted research is to enable besides simulation also universal control algorithms through Virtual Reality experience, produced by game engine Unity3D, which can be easily modified for a wide range of industrial equipment. The primary outcome of this work is the development of the synchro-nisation model of real and virtual industrial robots and experimental testing the developed model in Virtual Reality and shop floor labs

show abstract

Using HoloLens to create a virtual operator station for mobile robots

Cited by 13 publications

References 5 publications

Intuitive Robot Teleoperation Through Multi-Sensor Informed Mixed Reality Visual Aids

Intuitive Robot Teleoperation Through Multi-Sensor Informed Mixed Reality Visual Aids

Assessment of Augmented Reality in Manual Wiring Production Process with Use of Mobile AR Glasses

Digital Twin Based Synchronised Control and Simulation Of the Industrial Robotic Cell Using Virtual Reality

Contact Info

Product

Resources

About