In order to further improve fractional-pel interpolation image quality of video sequence with different resolutions and reduce algorithm complexity, the fractional-pel interpolation algorithm based on adaptive filter (AF_FIA) is proposed. This algorithm adaptively selects the interpolation filters with different orders according to the three video sequence regions with different resolutions; in the three video sequence regions with different resolutions, the high-order interpolation filter is replaced by low-order interpolation filter according to the correlation between pixels to realize the adaptive selection of filter. The complexity analysis results show that compared with other algorithms, this algorithm reduces space complexity and computation complexity, thus reducing the storage access and coding time. The simulation results indicate that compared with other algorithms, this algorithm has good coding performance and robustness for video sequences with different resolutions.