Vehicle speed prediction can facilitate many applications, such as optimizing vehicle propulsion systems and designing advanced driver assistance control systems. In a complex and variable traffic environment, many dynamic factors affect vehicle speed and make it difficult to predict accurately. The development of intelligent transportation systems and machine learning methods makes it possible to predict short-term vehicle speed accurately. A novel vehicle speed prediction model is proposed in this paper to improve prediction accuracy based on a deep learning method. A practical temporal and channel attention module (TCAM) is designed for convolutional neural networks (CNNs) to strengthen meaningful information and reduce the amount of unnecessary information. A gated recurrent unit (GRU) network with an attention mechanism is constructed to explore significant hidden relationships among time-series data with its memory function. These two subprediction models are fused to enhance the performance of vehicle speed prediction. Simulation experiments using IPG Carmaker software validate that the proposed model provides better predictive accuracy than traditional and existing vehicle speed prediction methods based on deep learning.