In a tele-abrasive task, it is principally human arm movements that cause variation in the position of the abrasive nozzle, thereby resulting in high operating costs and low productivity. It is difficult to design a system that can minimize the variation that accrues from operators behaving differently, which is difficult to predict. Although skilled operators can reduce this variation, becoming a skillful operator requires a lengthy training period. In this work, a two-stage variation streaming technique was used to extract variation sources in a tele-abrasive system. Furthermore, we propose an integrated human–computer approach to control variation in these systems—an approach that applies an innovative human arm movement pattern incorporated with a Kalman filter into a standard system. A virtual tele-abrasive system was used to validate our approach. Furthermore, compared with conventional systems, the proposed approach will help operators to perform abrasive tasks more comfortably and require a shorter training period.