This paper introduces a systematic constraint-based approach to specify complex tasks of general sensorbased robot systems consisting of rigid links and joints. The approach integrates both instantaneous task specification and estimation of geometric uncertainty in a unified framework. Major components are the use of feature coordinates, defined with respect to object and feature frames, which facilitate the task specification, and the introduction of uncertainty coordinates to model geometric uncertainty. While the focus of the paper is on task specification, an existing velocity based control scheme is reformulated in terms of these feature and uncertainty coordinates. This control scheme compensates for the effect of time varying uncertainty coordinates. Constraint weighting results in an invariant robot behavior in case of conflicting constraints with heterogeneous units.The approach applies to a large variety of robot systems (mobile robots, multiple robot systems, dynamic human-robot interaction, etc.), various sensor systems, and different robot tasks. Ample simulation and experimental results are presented.