This paper proposes a new intelligent recognition method for concrete ultrasonic detection based on wavelet packet transform and a convolutional neural network (CNN). To validate the proposed data-based method, a case study is presented where the K-fold cross-validation was adopted to produce the performance analysis and classification experiments. Moreover, three evaluation indicators, precision, recall, and F-score, are calculated for analyzing the classification performance of the trained models. As a result, the obtained four-classifying CNN reaches more than 99% detection accuracy while the lowest recognition accuracy is not less than 92.5% on the testing dataset for the six-classifying CNN model. Compared with the existing stochastic configuration network (SCN) models, the presented method achieves the design objective with better recognition performance. The calculation results of the six-classifying and five-classifying models and related research clearly indicate the remaining challenging tasks for intelligent recognition algorithms in extracting features and classifying mass data from various concrete defects precisely and efficiently.