The occurrence of fault in rotating machinery is random and complex, and the diagnosis of the compound faults has been a challenge in industrial production. Accurate diagnosis of the compound faults can be of significant help to practical maintenance and management. However, most existing intelligent diagnostic methods typically require abundant data for training, which is often difficult to collect for compound faults. In this paper, a novel method called impact feature-based decoupling capsule network (IFDCN) is proposed for diagnosing compound faults with incomplete datasets. In this model, an improved Laplace wavelet kernel capsule neural network is proposed to extract and enhance the impact features of vibration signal. A decoupling classifier is designed to decouple the compound faults in the diagnostic process so as to identify the sub-faults contained in the compound faults. In using this proposed model for incomplete datasets, the compound fault data is not trained and is not labeled. Through training on single-fault data, the proposed model is capable of classifying and decoupling the fault types. The feature extraction capability of the network is visualized by heat maps, and the physical significance of feature extraction is explained by deep learning network. The effectiveness of IFDCN is verified through different experimental of gear and bearing and the experiment results indicate that the proposed model has higher identifying precision and can accurately decouple the compound faults without compound fault samples.