Identifying crack and predicting crack propagation are critical processes for the risk assessment of engineering structures. Most traditional approaches to crack modeling are faced with issues of high computational costs and excessive computing time. To address this issue, we explore the potential of deep learning (DL) to increase the efficiency of crack detection and forecasting crack growth. However, there is no single algorithm that can fit all data sets well or can apply in all cases since specific tasks vary. In the paper, we present DL models for identifying cracks, especially on concrete surface images, and for predicting crack propagation. Firstly, SegNet and U-Net networks are used to identify concrete cracks. Stochastic gradient descent (SGD) and adaptive moment estimation (Adam) algorithms are applied to minimize loss function during iterations. Secondly, time series algorithms including gated recurrent unit (GRU) and long short-term memory (LSTM) are used to predict crack propagation. The experimental findings indicate that the U-Net is more robust and efficient than the SegNet for identifying crack segmentation and achieves the most outstanding results. For evaluation of crack propagation, GRU and LSTM are used as DL models and results show good agreement with the experimental data.