In the context of increasing demand for flexible operation of hydropower plants, this study analyses the factors leading to instability in a prototype pump turbine under turbine mode, based on experimental data collected on-site. Pressure sensors and accelerometers were installed to compare pressure signals and vibration acceleration signals. The test results show that the characteristics of pressure fluctuation are dominated by three specific factors under different output power. Vibration reaches its maximum value at approximately 30% of the rated output and its minimum value at around 70% of the rated output, and the causes of twice increases in vibration is not the same. These characteristics often indicate potential unsteady flow, vortex-induced vibration, and flow separation, subsequently affecting the stability of the pump turbine. Understanding the conditions and severity of the occurrence of instability characteristics in the unit is of great significance for power plant personnel to adjust the unit’s operation strategy and improve operational efficiency.