Ultrasonic testing of concrete structures using the pitch-catch method is an effective technique for testing concrete structures that cannot be accessed on two opposing surfaces. However, the ultrasonic signals so measured are extremely noisy and contain a complicated pattern of multiple frequency-coupled reflections that makes interpretation a difficult task. In this investigation, a neural network modeling approach is used to classify ultrasonically tested concrete specimens into one of two classes: defective or nondefective. Different types of neural nets are used, and their performance is evaluated. It was found that correct classification of the individual ultrasonic signals could be achieved with an accuracy of 75 percent for the test set and 95 percent for the training set. These recognition rates lead to the correct classification of all the individual test specimens. The study shows that although some neural net architectures may show high performance using a particular training data set, their results might not be consistent. In this paper, the consistency of the network performance was tested by shuffling the training and testing data sets.