The current paper presents a comprehensive review analysis of Multi-agent control methodologies for Integrated Building Energy Management Systems (IBEMSs), considering combinations of multi-diverse equipment such as Heating, Ventilation, and Air conditioning (HVAC), domestic hot water (DHW), lighting systems (LS), renewable energy sources (RES), energy storage systems (ESS) as well as electric vehicles (EVs), integrated at the building level. Grounded in the evaluation of key control methodologies—such as Model Predictive Control (MPC) and reinforcement learning (RL) along with their synergistic hybrid integration—the current study integrates a large number of impactful applications of the last decade and evaluates their contribution to the field of energy management in buildings. To this end, over seventy key scholarly papers from the 2014–2024 period have been integrated and analyzed to provide a holistic evaluation on different areas of interest, including the utilized algorithms, agent interactions, energy system types, building typologies, application types and simulation tools. Moreover, by analyzing the latest advancements in the field, a fruitful trend identification is conducted in the realm of multi-agent control for IBEMS frameworks, highlighting the most prominent solutions to achieve sustainability and energy efficiency.