The burgeoning Internet of Energy (IoE) paradigm, a fusion of the Internet of Things (IoT) and Smart Grid (SG) technologies, holds the promise of significantly enhancing the reliability and efficiency of energy production, transmission, and consumption across the entire energy chain, from generation to the end user. Two central technical aspects that enable this innovation are the advent of smart consumer electronics and the establishment of bidirectional IoT communications. These developments have facilitated the incorporation of novel applications into the Smart Grid, including smart metering, Consumer Demand-Response (CDR) management, and prepayment. In this study, our focus lies primarily on the development of a secure and efficient key management system for CDR communications. It is demonstrated herein that a previous key graph-based scheme, called EDR, is susceptible to collusion attacks and lacks support for broadcast CDR communications. In response to these vulnerabilities, we propose a novel key management scheme, referred to as Secure and Efficient key management scheme for CDR communications (SE-CDR). This scheme retains the strengths of the EDR while introducing a modified multi-group key graph technique, designed to ensure the secure, efficient, and scalable management of unicast, multicast, and broadcast CDR communications. The presented security analysis and performance evaluation results establish the robust security of the SE-CDR scheme. Moreover, a comparative analysis revealed that this new approach offers significant improvements in terms of storage and communication efficiency, outperforming existing state-of-the-art methods. This study thus presents a promising advancement in the realm of secure and efficient key management for the Internet of Energy paradigm.