Perishable surveying, mapping, and post-disaster damage data typically require efficient and rapid field collection techniques. Such datasets permit highly detailed site investigation and characterization of civil infrastructure systems. One of the more common methods to collect, preserve, and reconstruct three-dimensional scenes digitally, is the use of an unpiloted aerial system (UAS), commonly known as a drone. Onboard photographic payloads permit scene reconstruction via structure-from-motion (SfM); however, such approaches often require direct site access and survey points for accurate and verified results, which may limit its efficiency. In this paper, the impact of the number and distribution of ground control points within a UAS SfM point cloud is evaluated in terms of error. This study is primarily motivated by the need to understand how the accuracy would vary if site access is not possible or limited. In this paper, the focus is on two remote sensing case studies, including a 0.75 by 0.50-km region of interest that contains a bridge structure, paved and gravel roadways, vegetation with a moderate elevation range of 24 m, and a low-volume gravel road of 1.0 km in length with a modest elevation range of 9 m, which represent two different site geometries. While other studies have focused primarily on the accuracy at discrete locations via checkpoints, this study examines the distributed errors throughout the region of interest via complementary light detection and ranging (lidar) datasets collected at the same time. Moreover, the international roughness index (IRI), a professional roadway surface standard, is quantified to demonstrate the impact of errors on roadway quality parameters. Via quantification and comparison of the differences, guidance is provided on the optimal number of ground control points required for a time-efficient remote UAS survey.