This review article investigates the methods proposed for disaggregating the space heating units' load from the aggregate electricity load of commercial and residential buildings. It explores conventional approaches together with those that employ traditional machine learning, deep supervised learning and reinforcement learning. The review also outlines corresponding data requirements and examines the suitability of a commonly utilised toolkit for disaggregating heating loads from lowfrequency aggregate power measurements. It is shown that most of the proposed approaches have been applied to high-resolution measurements and that few studies have been dedicated to low-resolution aggregate loads (e.g. provided by smart meters). Furthermore, only a few methods have taken account of special considerations for heating technologies, given the corresponding governing physical phenomena. Accordingly, the recommendations for future works include adding a rigorous preprocessing step, in which features inspired by the building physics (e.g. lagged values for the ambient conditions and values that represent the correlation between heating consumption and outdoor temperature) are added to the available input feature pool. Such a pipeline may benefit from deep supervised learning or reinforcement learning methods, as these methods are shown to offer higher performance compared to traditional machine learning algorithms for load disaggregation.