Innovative mixed reality approach to rock mass mapping in underground mining

Abstract

Traditional methods of mapping in mines use a geological compass for orientation measurements, a tape measure for length measurements and pen/paper for sketching and taking notes. This has been improved in recent years with a change from compass to smart phones/tablets and the use of tablets for sketching photographic images. A significant improvement is the use of remote sensing e.g. LiDAR and photogrammetry to collect data with subsequent office-based data processing. However, the collected 3D data is usually displayed on 2D monitors which is a very different experience in comparison to fieldwork. The perception of 3D geometry and scale are lost, which may result in biased and limited interpretations. The Microsoft HoloLens is an augmented reality headset comprising a computer with a transparent screen and a 3D scanner. This device allows the creation of a 3D map of the real-world almost in real-time. Using this map, it becomes possible to place virtual objects within the real-world and to know the location of the HoloLens user. As the screen is transparent HoloLens users can see the real-world along with virtual objects, such as previous mapping and drilling completed anywhere within the underground mining operation. Advantage of state-of-the-art new VR/MR technology was taken and a Microsoft HoloLens application which can be used to annotate/draw directly on a rock face was developed and the necessary spatial data in the field for use during mapping is made available. After field work has been undertaken, the collected data can be easily exported to other software without further post-processing. Holographic mapping procedures for virtual rock outcrops constructed from remote sensing data, e.g. ground and UAV based LiDAR and photogrammetry was also developed. This allows office-based mapping of virtual outcrops. The developed software will improve direct outcrop mapping and office-based mapping using remote sensing data.

Using Mixed Reality for the Visualization and Dissemination of Complex 3D Models in Geosciences—Application to the Montserrat Massif (Spain)

“…In [14] the HoloLens 2 UWP management is described. The applications for the Ho-loLens are created by using Unity game engine software [51] and the Mixed Reality Toolkit, an open-source project supported by Microsoft that provides primary scripts to accelerate the development of MR applications.…”

“…In structural geology, traditional mapping methods have evolved from the geological compass, tape and paper field book to smart phones for the dip/dip direction logging, and the use of tablets for sketching photographic images. Additionally, Terrestrial Laser Scanning (TLS) and digital photogrammetry are now available to assist logging in wider areas, with an improvement in the completeness and representativeness of the results [14].…”

“…However, despite these scientific and technical advances, it is still a common practice to simplify the 3D data in 2D static images, losing part of its communicative potential [14,15]. Because of technical limitations, the collected 3D data is usually displayed in 2D on monitors and in PDF or paper documents (static plan and elevation views, with some sections), leading to a notably different experience when compared to fieldwork.…”

“…Prof. D. Stead (SFU, Burnaby, BC, Canada) spoke about "Geovisualization of mixed reality holographic models" in the final part of his ISRM lecture [23], giving a set of inspiring examples. At SFU, the Engineering Geology and Resource Geotechnics Research Group has used MR technology in numerous engineering projects including open pits and landslide investigation, developing several applications based on MR HMD, such as "Virtual Core Logging", "Virtual rock outcrop", or "EasyMap/EasyMine" [14,34,35]. For instance, in [14], the developers used state-of-the-art new VR/MR technology to provide in the field the virtual tools to be used for real outcrop mapping (virtual compass, virtual tape and so on).…”

“…At SFU, the Engineering Geology and Resource Geotechnics Research Group has used MR technology in numerous engineering projects including open pits and landslide investigation, developing several applications based on MR HMD, such as "Virtual Core Logging", "Virtual rock outcrop", or "EasyMap/EasyMine" [14,34,35]. For instance, in [14], the developers used state-of-the-art new VR/MR technology to provide in the field the virtual tools to be used for real outcrop mapping (virtual compass, virtual tape and so on). They also developed holographic mapping procedures for office-based mapping of virtual rock outcrops acquired with ground-and UAV-based LiDAR and with digital photogrammetry.…”