Significant wave height (SWH) prediction can effectively improve the safety of marine activities and reduce the occurrence of maritime accidents, which is of great significance to national security and the development of the marine economy. In this study, we comprehensively analyzed the SWH prediction performance of the recurrent neural network (RNN), long short-term memory network (LSTM), and gated recurrent unit network (GRU) by considering different input lengths, prediction lengths, and model complexity. The experimental results show that (1) the input length impacts the prediction results of SWH, but it does not mean that the longer the input length, the better the prediction performance. When the input length is 24h, the prediction performance of RNN, LSTM, and GRU models is better. (2) The prediction length influences the SWH prediction results. As the prediction length increases, the prediction performance gradually decreases. Among them, RNN is not suitable for 48h long-term SWH prediction. (3) The more layers of the model, the better the SWH prediction performance is not necessarily. When the number of layers is set to 3 or 4, the model’s prediction performance is better.