The assessment of an energy retrofit necessarily requires an energy measurement campaign before (base year energy consumption) and after (post retrofit energy consumption) the retrofit. Only in this way is it possible to reach a safe conclusion, on the true retrofit impact. In addition, a number of adjustments are necessary to secure that the retrofit impact on energy consumption is effectively isolated, i.e., which we report on the true retrofit impact and not, for example, on external variations, such as a more mild winter. This paper introduces a conceptual framework for taking account, in the retrofit impact assessment, of three external parameters: weather, indoor comfort and space occupancy. The broader strategy behind this work is to develop a comprehensive methodology that would allow a cost efficient, fast and accurate assessment of energy retrofits in buildings. This would allow insight, on the investor side, as to the prudence of his investment and, and in this way, could help the proliferation of the practice of energy retrofits. The adjustment methodology, introduced here, is a first step in this direction.