Sunspots have a significant impact on human activities. In this study, we aimed to improve solar activity prediction accuracy. To predict the sunspot number based on different aspects, such as extracted features and relationships among data, we developed a hybrid model that includes a one-dimensional convolutional neural network (1D-CNN) for extracting the features of sunspots and bidirectional long short-term memory (BiLSTM) embedded with a multi-head attention mechanism (MHAM) to learn the inner relationships among data and finally predict the sunspot number. We evaluated our model and several existing models according to different evaluation indicators, such as mean absolute error (MAE) and root mean square error (RMSE). Compared with the informer, stacked LSTM, XGBoost-DL, and EMD-LSTM-AM models, the RMSE and MAE of our results were more than 42.5% and 65.1% lower, respectively. The experimental results demonstrate that our model has higher accuracy than other methods.