Dynamic analysis of structures is very important for structural design and health monitoring. Conventional numerical or experimental methods often suffer from the great challenges of analyzing the responses of linear and nonlinear structures, such as high cost, poor accuracy, and low efficiency. In this study, the recurrent neural network (RNN) and long short-term memory (LSTM) models were used to predict the responses of structures with or without nonlinear components. The time series k-means (TSkmeans) algorithm was used to divide label data into different clusters to enhance the generalization of the models. The models were trained with different cluster acceleration records and the corresponding structural responses obtained by numerical methods, and then predicted the responses of nonlinear and linear structures under different seismic waves. The results showed that the two deep learning models had a good ability to predict the time history response of a linear system. The RNN and LSTM models could roughly predict the response trend of nonlinear structures, but the RNN model could not reproduce the response details of nonlinear structures (high-frequency characteristics and peak values).