Point cloud data are used to create an as-built building information model (as-built BIM) that reflects the actual status of any building, whether being constructed or already completed. However, indoor clutter objects in the point cloud data, such as people, tools, and materials, should be effectively eliminated to create the as-built BIM. In this study, the authors proposed a novel method to automatically remove indoor clutter objects based on the Manhattan World assumption and object characteristics. Our method adopts a two-dimensional (2D) projection of a 3D point cloud approach and utilizes different properties of indoor clutter objects and structural elements in the point cloud. Voxel-grid downsampling, density-based spatial clustering (DBSCAN), the statistical outlier removal (SOR) filter, and the unsupervised radius-based nearest neighbor search algorithm were applied to our method. Based on the evaluation of our proposed method using six actual scan datasets, we found that our method achieved a higher mean accuracy (0.94), precision (0.97), recall (0.90), and F1 core (0.93) than the commercial point cloud processing software. Our method shows better results than commercial point cloud processing software in classifying and removing indoor clutter objects in complex indoor environments acquired from construction sites. As a result, assumptions about the different properties of indoor clutter objects and structural elements are being used to identify indoor clutter objects. Additionally, it is confirmed that the parameters used in the proposed method could be determined by the voxel size once it is decided during the downsampling process.