The variable working conditions and frequent turns make the aircraft actuator system prone to failure, seriously threatening flight safety. The identification of the airplane actuator system is critical for flight decisions and safety. Most fault diagnosis methods of actuators only focus on the actuators themselves, ignoring the disturbance caused by the fault of the actuator position sensor, which may easily lead to wrong decisions. In order to distinguish the actuator fault from its position sensor fault and identify the fault type accurately, an offline diagnosis method of convolutional neural network (CNN) with novel topology for processing time series is proposed. A new shift layer is added after the input layer, which avoids the loss of a large number of features due to the direct connection between the time series and the convolutional layer. A local topological network learning complex pattern with inception module is designed to improve the diagnostic accuracy in different working conditions. The wide residual structure is introduced to expand the convolutional channel, which allows the network features at the bottom level to propagate directly to the top level to prevent network degradation. Simulation results show that this method can accurately diagnose the actuator fault and its position sensor, with an average accuracy of 96.8%. Compared with the current mainstream data-driven methods, the precision and recall are increased by 6.3% and 6.7% respectively.