Electro-hydrostatic actuators (EHAs) are hydraulic actuators that are flexible and exhibit high backdrivability. Flexible operation and accurate detection of reaction forces are required for robots to be able to perform in environments in which they will be cohabiting with humans. However, nonlinear elements that degrade detection accuracy, such as static friction, backlash, and oil leakage, are present in hydraulic systems. In addition, pressure sensors in hydraulic systems are not very accurate at estimating reaction forces, because they cannot estimate internal forces and viscous friction. In this study, we propose a combination of control algorithms for accurately estimating reaction forces. Static friction is compensated by using feedback modulators. In addition, we use a backlash and oil leakage compensator, which do not require any models, to suppress the relative velocity between the motor-side and load-side. Then, the use of a reaction force observer (RFOB) that exploits both pressure sensors and encoders is proposed. The RFOB can be implemented because disturbances are linearized by the compensators. Experimental results show that reaction forces can be estimated with very high accuracy using the proposed RFOB. In addition, we implemented force control using the RFOB and evaluate the force tracking performances by improving the estimation accuracy.