High Efficiency Video Coding (HEVC) is the latest video coding standard. The residual quadtree (RQT) coding structure which provides variable block size for transform coding, is adopted in HEVC to achieve high coding efficiency. However, compared with previous standards, encoding complexity is increased significantly in HEVC due to the advanced encoding structure. A fast transform unit (TU) mode decision algorithm using residual difference is proposed in this paper to reduce the computational complexity. The proposed algorithm utilized the residual difference to determine the criterion of early TU termination and early TU skip. The threshold was trained from the beginning samples of each sequence. Experimental results showed that the proposed algorithm saves up to 75.64% and on average 64% TU encoding time compared with HM 15.0 in low delay P configuration and the loss of average BD-BR is less than 0.5%.