Limitations and Review of Geometric Deep Learning Algorithms for Monocular 3D Reconstruction in Architecture

Abstract

This paper aims to test algorithms for 3D reconstruction from a single image specifically for building envelopes. This research shows the current limitations of these approaches when applied to classes outside of the initial distribution. We tested solutions with differentiable rendering, implicit functions, and other end-to-end geometric deep learning approaches. We recognize the importance of generating a 3D reconstruction from a single image for many different industries, not only for Architecture, Engineering, and Construction (AEC) industry but also for robotics, autonomous driving, gaming, virtual and augmented reality, drone delivery, 3D authoring, improving 2D recognition and many others. Henceforth, engineers and computer scientists could benefit, not only from having the 3D representations but also from the Building Information Model (BIM) at their disposal. With further development of these algorithms it could be possible to access specific properties such as thermal, physical, maintenance, cost, and other parameters embedded in the class.