This article is focused on the high-performance trajectory tracking control of single actuator of a hydraulic excavator. A novel adaptive neural finite-time controller without tedious offline parameter identification and the complex backstepping scheme is put forward. By employing a coordinate transform, the original system can be represented in a canonical form. Consequently, the control objective is retained by controlling the transformed system, which allows a simple controller design without using backstepping. To estimate the immeasurable states of the transformed system, a high-order sliding mode observer is employed, of which observation error is guaranteed to be bounded in finite time. To guarantee finite-time trajectory tracking performance, an adaptive neural finite-time controller based on neural network approximation and terminal sliding mode theory is synthesized. During its synthesis, an echo state network is used to approximate the lumped uncertain system functions, and it guarantees an improved approximation with online-updated output weights. Besides, to handle the lumped uncertain nonlinearities resulting from observation error and neural approximation error, a robust term is employed. The influences of the uncertain nonlinearities are restrained with a novel parameter adaption law, which estimates and updates the upper bound of the lumped uncertain nonlinearities online. With this novel controller, the finite-time trajectory tracking error convergence is proved theoretically. The superior performance and the practical applicability of the proposed method are verified by comparative simulations and experiments.