Demand response is receiving increasing interest as a new form of flexibility within low-carbon power systems. Energy models are an important tool to assess the potential capability of demand side contributions. This paper critically reviews the assumptions in current models and introduces a new conceptual framework to better facilitate such an assessment. We propose three dimensions along which change could occur, namely technology, activities and service expectations. Using this framework, the socio-technical assumptions underpinning 'bottom-up' activity-based energy demand models are identified and a number of shortcomings are discussed. First, links between appliance usage and activities are not evidence-based. We propose new data collection approaches to address this gap. Second, aside from thermal comfort, service expectations, which can be an important source of flexibility, are underrepresented and their inclusion into demand models would improve their predicative power in this area. Finally, flexibility can be present over a range of time scales, from immediate responses, to longer term trends. Longitudinal time use data from participants in demand response schemes may be able to illuminate these. The recommendations of this paper seek to enhance the current state-of-the-art in activity-based models and to provide useful tools for the assessment of demand response.