Intelligent on-site fault diagnosis and professional vibration analysis are essential for the safety and stability of rotating machinery operation. This paper represents a fault diagnosis scheme based on two-stage compressed sensing for triaxial vibration data, which realizes fault diagnosis for rotating machinery based on compressed data and data reconstruction for professional vibration analysis. In the 1st stage, the triaxial vibration signals are compressed using a pre-designed hybrid measurement matrix; these compressed data can be used both for time-frequency transform and for vibration data reconstruction. In the 2nd stage, the frequency spectra of the triaxial vibration signals are fused and further compressed using another pre-designed joint measurement matrix, which inhibits the high-frequency noises simultaneously. Finally, the fused spectra are employed as feature vectors in sparse-representation-based classification, where the proposed batch matching pursuit (BMP) algorithm is utilized to calculate the sparse vectors. The two-stage compression scheme and the BMP algorithm minimize the computational cost of on-site fault diagnosis, which is suitable for edge computing platforms. Meanwhile, the compressed vibration data can be reconstructed, which provides evidence for professional vibration analysis. The method proposed in this study is validated by two practical case studies, in which the accuracies are 99.73% and 96.70%, respectively.