Chromosomal karyotype is important to determine whether a newborn has a genetic disorder. There are two main categories of chromosomal abnormalities, structural abnormalities in which the chromosome structure is altered, and chromosome number abnormalities. Manual karyotyping is complex and takes a lot of time because it requires a high degree of domain expertise. Based on this investigation proposes a new method of chromosome defect detection based on deep learning with 20,299 chromosome images from Dongguan Kanghua Hospital as data and integrates the diversity of chromosome features, and trains to propose a classifier model based on feature fusion for chromosome abnormality detection. We put forward a feature fusion classifier with dynamic weights (FFCDW) for chromosomal abnormality detection, after data augmentation with three deep learning networks, ResNet, SENet, and VGG19, the three trained models are combined using a dynamic weighting approach. Experiments prove the FFCDW method outperforms these mainstream models of ResNet, SENet, and VGG19. The proposed method based on FFCDW achieves precision of 0.8902 and F1-score of 0.8805 with a small standard deviation (0.00903 and 0.00892, respectively). In addition, the algorithm can automatically assign weights based on the results of a single model, and the strategy with dynamic weights outperforms the strategy with fixed weights in the proposed feature fusion classifier.