Success of a tracking method depends largely on choosing the suitable window size as soon as the target size changes in image sequences. To achieve this goal, we propose a fast tracking algorithm based on adaptively adjusting tracking window. Firstly, tracking window is divided into four edge subwindows, and a background subwindow around it. Then, by calculating the spatiotemporal gradient power ratios of the target in each subwindow, four proper expansion vectors are associated with any tracking window sides such that the occupancy rate of the target in tracking window should be maintained within a specified range. In addition, since temporal changing of target is evaluated in calculating these vectors, we estimate overall target displacement by sum of expansion vectors. Experimental results using various real video sequences show that the proposed algorithm successfully track an unknown textured target in real time, and is robust to dynamic occlusions in complex noisy backgrounds.