Energy consumption in buildings is responsible for 40 % of the final energy consumption in the European Union and the United States of America. In addition to thermal energy, buildings require electricity for all kinds of appliances. Regulatory constraints such as energy labels aim at increasing the energy efficiency of large appliances such as fridges and washing machines. However, they only partially cover plug-loads. The amount of electricity consumption of unregulated plug-loads such as mobile phones, USB chargers and kettles is continuously increasing. For European households, their share of electricity consumption reached 25 % in 2018. Additional data about the plug-loads usage can help decrease the energy consumption of buildings by improving energy management systems, applying peak-shaving or demand-side management. People live and work in buildings, making such data privacy sensitive. Federated Learning (FL) helps to leverage these data without violating regulatory frameworks such as the General Data Protection Regulation. We use a high-frequency energy data set of office appliances (BLOND) to train four appliance classifiers (CNN, LSTM, ResNet and DenseNet). We investigate the effect of different data distributions (entire dataset, IID and non-IID) and training methods on four performance metrics (accuracy, F1 score, precision and recall). The results show that a non-IID setup decreases all performance metrics for some model architectures by 44 %. However, our LSTM model even with a non-IID labels achieves similar F1 scores compared to central training. Additionally, we show the importance of client selection in FL architectures to reduce the overall training time and we quantify the decrease in network traffic compared to a central training approach, the energy consumption and scalability.