Particle filters are often used for tracking objects within a scene. As the prediction model of a particle filter is often implemented using basic movement predictions such as random walk, constant velocity or acceleration, these models will usually be incorrect. Therefore, this paper proposes a new approach, based on a Canonical Correlation Analysis (CCA) tracking method which provides an object specific motion model. This model is used to construct a proposal distribution of the prediction model which predicts new states, increasing the robustness of the particle filter. Results confirm an increase in accuracy compared to state-of-the-art methods.