Classification and Integration of Massive 3d Points Clouds in a Virtual Reality (Vr) Environment

Abstract

Abstract. With the increasing volume of 3D applications using immersive technologies such as virtual, augmented and mixed reality, it is very interesting to create better ways to integrate unstructured 3D data such as point clouds as a source of data. Indeed, this can lead to an efficient workflow from 3D capture to 3D immersive environment creation without the need to derive 3D model, and lengthy optimization pipelines. In this paper, the main focus is on the direct classification and integration of massive 3D point clouds in a virtual reality (VR) environment. The emphasis is put on leveraging open-source frameworks for an easy replication of the findings. First, we develop a semi-automatic segmentation approach to provide semantic descriptors (mainly classes) to groups of points. We then build an octree data structure leveraged through out-of-core algorithms to load in real time and continuously only the points that are in the VR user's field of view. Then, we provide an open-source solution using Unity with a user interface for VR point cloud interaction and visualisation. Finally, we provide a full semantic VR data integration enhanced through developed shaders for future spatio-semantic queries. We tested our approach on several datasets of which a point cloud composed of 2.3 billion points, representing the heritage site of the castle of Jehay (Belgium). The results underline the efficiency and performance of the solution for visualizing classifieds massive point clouds in virtual environments with more than 100 frame per second.

Unsupervised Segmentation of Indoor 3d Point Cloud: Application to Object-Based Classification

“…We delineated F1-scores close to the sharpness results of the segmentation. Down the line, it is used for extracting the surface of ceilings, walls, or floors if one wants to make digital quotations (Poux et al, 2017b); it can provide a basis for extracting semantic spaces (sub-spaces) organised regarding their function (Poux et al, 2017a(Poux et al, , 2016; it can be used to provide a basis for floor plans, cut, section creation or visualisation purposes (Kharroubi et al, 2019). These are currently investigated as part of future works.…”

Initial User-Centered Design of a Virtual Reality Heritage System: Applications for Digital Tourism

“…The data gathered from reality vary in forms and utility scope. Three-dimensional point clouds and 3D mesh are the most common geospatial data types within VR applications, with examples in [24,35,36].…”

“…Then, the compression and structuration of such datasets are heavy constraints that limit the integration of point clouds directly in VR scenarios [39]. Some work, such as [35,36,40], permit the integration of massive point clouds data, but an effective visualization and interaction are still limited. Three-dimensional mesh representation is more commonly found within the industry workflows, first propelled by the history of obtaining 3D information without reality capture devices.…”

A Built Heritage Information System Based on Point Cloud Data: HIS-PC

“…2020, 9, 588 of 30 and non-specialists. Based on our experience and previous works in the field [7,75,76], we wish to add such visualisation techniques to HIS-PC to extend the uses of cultural heritage digital content (tourism, teaching, cultural mediation . .…”