This paper presents a new framework for particle tracking based on a Gaussian Mixture Model (GMM). It is an extension of the state-of-the-art iterative reconstruction of individual particles by a continuous modeling of the particle trajectories considering the position and velocity as coupled quantities. The proposed approach includes an initialization and a processing step. In the first step, the velocities at the initial points are determined after iterative reconstruction of individual particles of the first four images to be able to generate the tracks between these initial points. From there on, the tracks are extended in the processing step by searching for and including new points obtained from consecutive images based on continuous modeling of the particle trajectories with a Gaussian Mixture Model. The presented tracking procedure allows to extend existing trajectories interactively with low computing effort and to store them in a compact representation using little memory space. To demonstrate the performance and the functionality of this new particle tracking approach, it is successfully applied to a synthetic turbulent pipe flow, to the problem of observing particles corresponding to a Brownian motion (e.g., motion of cells), as well as to problems where the motion is guided by boundary forces, e.g., in the case of particle tracking velocimetry of turbulent Rayleigh–Bénard convection.