The inter-shaft bearing is the core component of a high-pressure rotor support system of a high-thrust aero engine. One of the most challenging tasks for a PHM is monitoring its working condition. However, considering that in the bearing rotor system of a high-thrust aero engine bearings are prone to wear failure due to unbalanced or misaligned faults of the rotor system, especially in harsh environments, such as those at high operating loads and high rotation speeds, bearing wear can easily evolve into serious faults. Compared with aero engine fault diagnosis and RUL prediction, relatively little research has been conducted on bearing condition monitoring. In addition, considering how to evaluate future performance states with limited time series data is a key problem. At the same time, the current deep neural network model has the technical challenge of poor interpretability. In order to fill the above gaps, we developed a new framework of a residual space–time feature fusion focusing module named RTCA-Net, which focuses on solving the key problem. It is difficult to accurately monitor the wear state of aero engine inter-shaft bearings under special working conditions in practical engineering. Specifically, firstly, a residual space–time structure module was innovatively designed to capture the characteristic information of the metal dust signal effectively. Secondly, a feature-focusing module was designed. By adjusting the change in the weight coefficient during training, the RTCA-Net framework can select the more useful information for monitoring the wear condition of inter-shaft bearings. Finally, the experimental dataset of metal debris was verified and compared with seven other methods, such as the RTC-Net. The results showed that the proposed RTCA-Net framework has good generalization, superiority, and credibility.