The acceleration based force control is very important component in motion control. Looking ahead the development of the robotics system in the future, realization with high accuracy of the force control is essential. Since it is controlled so as to follow the external environment, the control performance completely depends on an impedance of the contact object. In generally, the velocity feedback stabilization is used to solve the problem. This paper tries to improve the fast-response while stabilizing the whole system based on the velocity feedback. The virtual actuator system is introduced in the proposed method. The control input for stabilizing is calculated by considering the interaction between the virtual system and the real system that contacts to objects. As a result, the velocity feedback works only in a contact state. With the proposed method, both of stabilization and quick response of the force control are achieved. The effectiveness of the proposed method is confirmed through the simple experiments.