Tracking human pose using observations from less than three cameras is a challenging task due to ambiguity in the available image evidence. This work presents a method for tracking using a pre-trained model of activity to guide sampling within an Annealed Particle Filtering framework. The approach is an example of model-based analysis-bysynthesis and is capable of robust tracking from less than 3 cameras with reduced numbers of samples. We test the scheme on a common dataset containing ground truth motion capture data and compare against quantitative results for standard Annealed Particle Filtering. We find lower absolute and relative error scores for both monocular and 2-camera sequences using 80% fewer particles.