Robotic exoskeletons with different actuators have been extensively investigated for decades, but designing a robot joint with compact structure, sufficient output torque, and satisfactory kinematic performance still remains a challenge. The paper introduces the mechanical design and control scheme of a novel joint with a circular rotary electro-hydraulic actuator in order to provide walking assistance for the hip joint. First, measurement and analysis techniques are applied in the gait recognition to determine an accurate swing range of the target joint and provide essential information for the structure design of the hydraulic rotary drive joint. Secondly, the functional structure and working principle of the hydraulic rotary joint are described in detail and the corresponding mathematical model is established for the valve-controlled hydraulic servo system. In addition, an adaptive sliding mode control (ASMC) method is proposed for position-tracking and improving the anti-disturbance ability of the controlled system. The global asymptotic stability and finite-time convergence of the closed-loop system are proved by Lyapunov stability theory. Finally, the feasibility and position-tracking performance of the designed ASMC controller were experimentally verified. Compared to the PID controller and the conventional sliding mode control scheme, the proposed controller improved the position-tracking control accuracy by 20%.