Robot manipulators are now used in various domains and environments, where they can be subjected to random vibrations. Random vibrations mainly affect the torque control signal, and a torque controller is therefore required to be designed for stabilization purposes. However, for security or intellectual property protection reasons, most commercialized robots are manufactured with unknown and inaccessible torque controller interface such that the user can only design a position/velocity controller. This paper proposes an adaptive task-space velocity controller free from the inner controller's structure and exhibiting stochastic and deterministic disturbances rejection to deal with these issues. To deal with the unknown inner controller, the paper exploits the fact that most torque controllers use a velocity feedback term, and it considers the other terms as an unknown functions vector. To cope with random disturbances, it is demonstrated that the random excitation matrix can be linearly parameterized, and therefore, a direct adaptive method is constructed. Using radial basis function neural network (RBF NN), an indirect adaptive method is developed to cope with deterministic uncertainties. Through Lyapunov theory, the paper proves that all the closed-loop signals are bounded in probability. The effectiveness of the proposed approach is further demonstrated through simulation comparisons.INDEX TERMS Robot manipulators, Adaptive task-space control, Closed inner controller, Random vibrations, Neural networks.