This research investigates the impact of scene context knowledge on tracking vehicles in an urban environment based on video image change detection. The scene context consists of knowledge of the road network and 3D building properties. Airborne sensor position information relative to a 3D model of the context enables calculation of building occlusions of ground locations. From this context, probability of detection maps that include regions of interest and smoothed lines-of-sight are developed that assist the change detection algorithm in reducing false alarms.