Object tracking has been researched for many years as an important topic in machine learning, robot vision and many other fields. Over the years, various tracking methods are proposed and developed in order to gain a better tracking effect. Among them the mean-shift algorithm turns out to be robust and accurate compared other algorithms after different kinds of tests. But due to its limitations, the changes in scale and rotational motion of an object cannot be effectively processed. This problem occurs when the object of interest moves towards or away from the video camera. Improving over the previously proposed method such as scale and orientation adaptive mean shift tracking, which performs well with scaling change but not for the rotation, in this paper, the proposed method modifies the continuously adaptive mean shift tracking method so that it can handle effectively for changes in size and rotation in motion, simultaneously. The simulation results yield a successful tracking of moving objects even when the object undergoes scaling in size and rotation in motion in comparison to the conventional ones.