The intelligent maintenance of railway equipment plays a pivotal role in advancing the sustainability of transportation and manufacturing. Railway turnouts, being an essential component of railway infrastructure, often encounter various faults, which present operational challenges. Existing fault diagnosis methods for railway turnouts primarily utilize vectorized monitoring data, interpreted either through vector-based models or distance-based measurements. However, these methods exhibit limited interpretability or are heavily reliant on standard curves, which impairs their performance or restricts their generalizability. To address these limitations, a railway turnouts fault diagnosis method with monitoring signal images and support matrix machine is proposed herein. In addition, a pinball loss-based multiclass support matrix machine (PL-MSMM) is designed to address the noise sensitivity limitations of the multiclass support matrix machine (MSMM). First, the time-series monitoring signals in one dimension are transformed into images in two dimensions. Subsequently, the image-based feature matrix is constructed. Then, the PL-MSMM model is trained using the feature matrix to facilitate the fault diagnosis. The proposed method is evaluated using a real-world operational current dataset, achieving a fault identification accuracy rate of 98.67%. This method outperforms the existing method in terms of accuracy, precision, and F1-score, demonstrating its superiority.