A piston piezoelectric (PZT) pump has many advantages for the use of light actuators. How to deal with the contradiction between the piston amount and oil delivery quality is an essence when designing the pump. In order to depress the pressure pulsation and flow pulsation in a single-piston PZT pump, a two-stage attenuator is proposed. It involves an elastic cavity group and an unloading valve. Unsteady flow inside the pump is numerically calculated and analyzed to reveal its delivery characteristic in the whole pumping cycle. The distributing process of the passive valves is obtained through the dynamic mesh technique. Influences of key design and operation factors on the delivery performance of the pump are analyzed. The results indicate that the flexible cavity group and the unloading valve arranged at the delivery port can reduce the flow pulsation by 45%. The design can effectively provide stable flow for the actuator in a certain frequency range.