The multi-classification of finger movements is a difficult challenge at present. In this paper, a multi-task Siamese network is proposed. Based on the basic Siamese network architecture, this study designs the position relationships between sample pairs and constructs single-sample subtasks for the left and right input samples. The position relationships between samples not only additionally supplement the spatial factors between electroencephalogram (EEG) signals that facilitate the generalization performance of position network learning, but also turn the EEG signals set structured so that the multi-task Siamese network is used to effectively learn multiple relationships between samples. During the model testing phase, a balanced matching algorithm of single test sample VS multi-training sample is used to improve the robustness of the multi-task Siamese network model. The experimental results on the 5F motor imagery data set from The Largest SCP of Motor-Imagery, an EEG data signal repository, and data set4 of the 4th brain-computer interface (BCI) competition show that compared with the single-subtask Siamese network, EEGNet, ResNet-50 and the public experimental results of the original dataset, the multi-task Siamese network in this paper can obtain better multiclass motor imagery (MI) classification accuracy and can better realize the multi-classification of MI-EEG signals. This study could be helpful to promote the development of finger-based MI-BCI system.