Automatic 3D Reconstruction From Unstructured Videos Combining Video Summarization and Structure From Motion

Abstract

Social media and collection of large volumes of multimedia data such as images, videos and the accompanying text is of prime importance in today's society. This is stimulated by the power of the humans to communicate with one another. A useful paradigm of exploitation of such a huge amount of multimedia volumes is the 3D reconstruction and modeling of sites, historical cultural cities/regions or objects of interest from the short videos captured by simple users mainly for personal or touristic purposes. The main challenge in this research is the unstructured nature of the videos and the fact that they contain much information which is not related with the object the 3D model we ask for but for personal usage such as humans in front of the objects, weather conditions, etc. In this article, we propose an automatic scheme for 3D modeling/reconstruction of objects of interest by collecting pools of short duration videos that have been captured mainly for touristic purposes. Initially a video summarization algorithm is introduced using a discriminant Principal Component Analysis (d-PCA). The goal of this innovative scheme is to extract the frames so that bunches within each video cluster that contains videos of content referring to the same object present the maximum coherency of image data while content across bunches the minimum one. Experimental results on cultural objects indicate the efficiency of the proposed method to 3D reconstruct assets of interest using an unstructured image content information.

Wild Image Retrieval with HAAR Features and Hybrid DBSCAN Clustering For 3D Cultural Artefact Landmarks Reconstruction

“…In [15] this article, we propose an automatic scheme for 3D modeling/reconstruction of objects of interest by collecting pools of short duration videos that have been captured mainly for touristic purposes. Initially a video summarization algorithm is introduced using a discriminant Principal Component Analysis (d-PCA).…”

3D Building Façade Reconstruction Using Deep Learning

“…Image data have been exploited, besides as a texture source, to generate dense depth point-clouds, using an approach known as Structure from Motion (SFM) [8]. The image-driven point-clouds are then, either combined with the laser scanning data [9][10][11] or used solely [12,13] to construct dense and photo-realistic reconstructions.…”

“…Compared to the aforementioned approaches, our current and earlier works on the topic of photo-realistic 3D building façade reconstruction, are amongst the few that attempt to exclude the dependence on laser scanning data [12,13]. Our framework combines image and georeferenced data to extract meaningful attributes for the façade's structural elements (positional, appearance and depth-related features).…”