Terrestrial laser scanning (TLS) provides a rapid remote sensing technique to model 3D objects but can also be used to assess the surface condition of structures. In this study, an effective image processing technique is proposed for crack detection on images extracted from the octree structure of TLS data. To efficiently utilize TLS for the surface condition assessment of large structures, a process was constructed to compress the original scanned data based on the octree structure. The point cloud data obtained by TLS was converted into voxel data, and further converted into an octree data structure, which significantly reduced the data size but minimized the loss of resolution to detect cracks on the surface. The compressed data was then used to detect cracks on the surface using a combination of image processing algorithms. The crack detection procedure involved the following main steps: (1) classification of an image into three categories (i.e., background, structural joints and sediments, and surface) using K-means clustering according to color similarity, (2) deletion of non-crack parts on the surface using improved subtraction combined with median filtering and K-means clustering results, (3) detection of major crack objects on the surface based on Otsu’s binarization method, and (4) highlighting crack objects by morphological operations. The proposed technique was validated on a spillway wall of a concrete dam structure in South Korea. The scanned data was compressed up to 50% of the original scanned data, while showing good performance in detecting cracks with various shapes.