Computer vision tracking systems are used in many medical and industrial applications. Understanding and modeling the tracking errors for a given system aids in the correct implementation and operation for optimal measurement results. This project aims to simulate and experimentally validate a tracking system for medical imaging. In this work, we developed a validation strategy for a target-based vision tracking system with an industrial robot. The simulation results show that the system can be accurately modeled, and the error assessment strategy is robust. Experimental verification with an EPSON C3 robot shows the reliability of the vision tracking system to obtain the target position and pose accurately. The general-purpose performance assessment strategy can be used as a vision tracking evaluation mechanism to ensure the system performance is adequate for a given application.