High stability in axial direction is required for rocket pumps operated under extremely low-temperature and high-pressure conditions, turbopumps therefore uses balance piston (BP) system for balancing their axial thrust. The BP system is stable under quasi-static conditions. However, BP system might become dynamically unstable under some condition. Thus it is fundamental for stability evaluation of turbopumps to predict static/dynamic characteristics in axial direction of BP system. Furthermore, we focus on characteristic change by cavitation which often occurs in the pump inlet. In this paper, an experimental study of a model turbopump which had an unshrouded impeller equipped with BP system was carried out. We experimented it with an active magnetic bearing (AMB) test facility in order forcibly to oscillate it with an optional amplitude and frequency. In addition, we examined characteristics of BP system by three-dimensional computational fluid dynamics (3D-CFD) simulations. The results of 3D-CFD simulations were in good agreement with these tendency of BP system, and were effective in predicting its static/dynamic characteristics. Some cases showed that dynamic characteristic of BP system became unstable by growth of cavitation, therefore we suggest that the influence of cavitation must also be considered in the design of turbopump.