With the flexibly designed three variations of the framework, a real-time frame rate ranging from 30fps (frames per second) to 130fps has been achieved. Also, a reference updating scheme is employed to compensate the decorrelation when the deformation becomes large. Finally, a real-time reference-based dynamic phase retrieval algorithm called G-LS3U is proposed to extract phase distributions from fringe or speckle patterns. Different parallel computing strategies are developed and applied to both the least-squares fitting and the windowed Fourier filtering (WFF) processes. G-LS3U achieved a remarkably high processing rate at 131+ fps, making G-LS3U the fastest reference-based dynamic phase retrieval algorithm reported heretofore.