As the operating conditions (also known as domains) of rotating machinery become increasingly diverse, fault diagnosis has garnered growing attention. However, fault diagnosis frequently encounters challenges such as long-tailed data distributions, domain shifts in monitoring data, and the unavailability of target-domain data. Existing approaches can only address some of these challenges, limiting their applications. To address these challenges concurrently, we introduce a novel learning paradigm called long-tailed multi-domain generalized fault diagnosis (LMGFD) and propose a two-stage learning framework for LMGFD, comprising domain-invariant feature learning and balanced classifier learning. In the first stage, we leverage a balanced multi-order moment matching (BMMM) module to align subdomains with long-tailed distributions. Additionally, a balanced prototypical supervised contrastive (BPSC) module is developed to effectively alleviate the contrastive imbalance. The combination of BMMM and BPSC enables the effective learning of long-tailed domain-invariant features. In the second stage, we extend the focal loss to a multi-class version and re-weight it using effective sample numbers to strengthen tailed-class loss, thereby mitigating the overfitting problem. Experimental results on both a public dataset and a private dataset support the competitiveness and effectiveness of the proposed method. The findings suggest that we present a promising solution for fault diagnosis of rotating machinery under variable operating conditions.