Building energy flexibility (BEF) is getting increasing attention as a key factor for building energy saving target besides building energy intensity and energy efficiency. BEF is very rich in content but rare in solid progress. The battery energy storage system (BESS) is making substantial contributions in BEF. This review study presents a comprehensive analysis on the BEF with BESS, in terms of the current study statues, connection to building energy management, integration with renewable energy sources and electric vehicles. In addition, electric vehicle (EV) is found to be flexible by itself for BEF. This study delivers some new insights and findings that the synchronized aging of building and battery and complex optimization with multiple indexes, is still challenging. Additionally, the opportunities behind the challenges are also highlighted that the higher level of BEF can be achieved by tackling the higher level of system complexity. Further, the suggestions about distributed energy control for distributed energy system, as well as coupling building energy pattern with users’ behavior pattern, are also proposed. The review study may further facilitate the BEF with BESS and push forward the synergistic improvement of building energy efficiency, intensity, and flexibility.