Recently, the construction industry has benefited from the increased application of smart construction led by the core technologies of the fourth industrial revolution, such as BIM, AI, modular construction, and AR/VR, which enhance productivity and work efficiency. In addition, the importance of “Off-Site Construction (OSC)”, a factory-based production method, is being highlighted as modular construction increases in the domestic construction market as a means of productivity enhancement. The problem with OSC construction is that the quality inspection of Precast Concrete (PC) members produced at the factory and brought to the construction site is not carried out accurately and systematically. Due to the shortage of quality inspection manpower, a lot of time and money is wasted on inspecting PC members on-site, compromising inspection efficiency and accuracy. In this study, the major inspection items to be checked during the quality inspection are classified based on the existing PC member quality inspection checklist and PC construction specifications. Based on the major inspection items, the items to which AI technology can be applied (for automatic quality inspection) were identified. Additionally, the research was conducted focusing on the detection of cracks, which are one of the major types of defects in PC members. However, accurate detection of cracks is difficult since the inspection mostly relies on a visual check coupled with subjective experience. To automate the detection of cracks for PC members, video images of cracks and non-cracks on the surface were collected and used for image training and recognition using Convolutional Neural Network (CNN) and object detection, one of the deep learning technologies commonly applied in the field of image object recognition. Detected cracks were classified according to set thresholds (crack width and length), and finally, an automated PC member crack detection system that enables automatic crack detection based on mobile and web servers using deep learning and imaging technologies was proposed. This study is expected to enable more accurate and efficient on-site PC member quality inspection. Through the smart PC member quality inspection system proposed in this study, the time required for each phase of the existing PC member quality inspection work was reduced. This led to a reduction of 13 min of total work time, thereby improving work efficiency and convenience. Since quality inspection information can be stored and managed in the system database, human errors can be reduced while managing the quality of OSC work systematically and accurately. It is expected that through optimizing and upgrading our proposed system, quality work for the precise construction of OSC projects can be ensured. At the same time, systematic and accurate quality management of OSC projects is achievable through inspection data. In addition, the smart quality inspection system is expected to establish a smart work environment that enables efficient and accurate quality inspection practices if applied to various construction activities other than the OSC projects.