Providing a compositional interpretation procedure for discourses in which descriptions of complex dependencies between interrelated objects are incrementally built is a key challenge for formal theories of natural language interpretation. This paper examines several quantificational phenomena and argues that we need richly structured contexts of interpretation that are passed on between different parts of the same sentence and also across sentential boundaries to account for these phenomena. The main contribution of the paper is showing how we can add structure to contexts in an incremental way, starting with the basic notion of context in classical first-order logic, i.e., interpretation contexts formalized as single total variable assignments.