Monitoring respiratory movements is an effective way to improve radiotherapy treatments of thoracic and abdominal tumors, but the current approach is limited to measuring specific points in the chest and abdomen. In this paper, a dynamic three-dimensional (3D) measurement approach of the human chest and abdomen surface is proposed, which can infer tumor movement more accurately, so the radiotherapy damage to the human body can be reduced. Firstly, color stripe patterns in the RGB color model are projected, then after color correction, the collected stripe image sequences are separated into the three RGB primary color stripe image sequences. Secondly, a fringe projection approach is used to extract the folded phase combined 3D Fourier transform with 3D Gaussian filtering. By the relationship between adjacent fringe images in the time sequence, Gaussian filter parameters with individual characteristics are designed and optimized to improve the accuracy of wrapped phase extraction. In addition, based on the difference between the fractional parts of the folded phase error, one remainder equation can be determined, which is used for time-phase unwrapping. The simulation model and human experiments show that the proposed approach can obtain the 3D image sequences of the chest and abdomen surface in respiratory motion effectively and accurately with strong anti-interference ability.